Shift left testing done early catches production bugs

Tech professionals just hope their next deployment won’t cause any errors. But they’re often unsure about their code cleanliness. Which reminds me of what happened to me one day.

My boss stormed my digital workspace with an ASAP request for me to perform testing. He needed it done for a user-to-store marketplace app that was bound to launch in two days.

Perfect timing. It happened early in my “career” so, without catching a breath, I said I’d take care of it. It still hurts when I think about that situation.

Two days to run tests for a live app without documentation or defined requirements felt like a workweek without lunches. I started with 4 hours of exploration tests during which I produced two A4 pages with questions and a list of errors that included type 500 errors, typos, and element overlapping.

Early the next day, I reported to the project team seeking answers. Imagine what the conversation could have led to.

This is why DevOps recommend shift left testing in software development

Bad implementation of some of the app’s functions caused the issues I found. One of the critical errors concerned product ordering. If a user requested a package pickup, they wouldn’t be able to order anything from then on. Our job was then to unclog this.

As a QA who joined in the last days of the project, I had to say if I believed the world should see the app we produced. For sure it shouldn’t. We needed an emergency repair plan to bring the software to a usable state.

The gang avoiding quality tests because “deadlines”. Source: Giphy

The revelation shook the client, but they knew we needed extra quality assurance work to deliver A-grade software. It was quite problematic.

• Project costs flew up because of the time required for additional implementation
• The software’s launch date had to be moved, which forced most departments at the company to stay on hold with promotion

Faced with these consequences, the client insisted on having a QA around to carry out tests, prep documentation, and provide app-health reports. That specialist was then enrolled in the next project to bless the team with more predictable development.

Why shift left testing should matter more

I assume you either faced such a situation or at least heard about it. This experience of mine isn’t unusual as in my lifetime, there have been several news stories revealing how the lack of early-stage software testing led to disasters. Inflexible time to market deadlines are the common modus operandi here.

Consider what happened with the Ariane 5 rocket in 1996.

After 10 years of manufacturing that ate up $7B, the rocket blew up live on air just dozens of seconds into the launch. Engineering decided to copy/paste the software from Ariane 4 for this newer edition without testing for compatibility.

This is how a 7B dollar explosion caused by a lack of QA testing looks like. Source: capcomespace.net

Then, there was the Boeing 737 MAX ocean crash that took 346 lives in a tragic accident. Cause?

On one side, an investigation revealed that the software was bugged. Boeing made a cheap move by working with less-skilled developers from a fourth-world country. That’s still understandable, but their American software engineers didn’t provide enough oversight for the MAX project.

Then, Boeing didn’t bother to test the production-ready software with pilots, which caused many of them to notice a critical issue with nose control while doing commercial flights.

So, these are just two examples, right? Oh, wait, there was something about Tesla and Samsung. See if you remember what went wrong there.

Unsuspecting customers shouldn’t pay the price of business incompetence. Save your project by following the shift left testing approach.

If you’re wondering what is shift left testing

If late or skipped quality-check skips cause trouble, why not start testing earlier?

That’s the idea behind shift left testing or “testing to the left” that was popularized by Larry Smight around 2001. He believed that bugs reported early in the development process cost less to fix, and soon enough you’ll see the data that proves so.

Smith’s philosophy helps teams develop software that’s safe, reliable, well-adjusted, and secure.

Shift-left testing is an approach to software and system testing in which testing is performed sooner in the life cycle.

“Left” in “Shift left testing” is a kind of programming slang that means “early”. It suggests where the testing task should be on the Kanban board. Often, it’s held somewhere on the board’s right side, which by rule groups tasks for later.

Shift it left, early and often, and it will be done sooner in the software development lifecycle.

Professionals stuck in the old school of programming leave tests for the 4th stage in the software development life cycle. See that on the chart below.

In my experience, that’s too damn late, because there are many opportunities for errors to multiply even before development finishes.

The cost of late-stage testing is stunning

Bug removal costs less earlier in the development. The later we bring a QA into play, the more hurtful the costs are. The idea of testing early is backed research.

Data from the Ponemon Institute reveals that bug fixes cost $80 in the early stages of development and a whooping $7600 in the case of production bugs.

Throughout my career, I’ve noticed that there is a strong trend of not hiring a QA until the product initiates self-destruct mode.

The cost of that decision is on the CTO or PO who believe that their developers won’t leave any issues unreported. But are they actively looking for software errors while they code? Not so much. You’ve heard the development mantra “If it’s not broke, don’t fix it”. Does it still sound believable?

But worry not, because if you believe there should be more extensive testing in your development cycle, shifting left will get easier in time.

Shift left testing benefits are quite logical

Shift left testing is mostly about introducing a new mindset to your development process without the need for a massive project re-alignment. It’s like adding one more employee to a recurring Zoom call.

Senior developers that you know should confirm that most software projects have greater success if there’s a QA on board from the start. That’s what we believe in at The Software House. By knowing how the product was built from scratch, the QA might even predict where most bugs can occur, which helps the team deliver milestones without delays.

If you’d like to limit or even avoid costly production bugs, you want a QA present from day one in your next project.

• You’ll have better project documentation that saves hours of guesswork
• The QA will ensure core functions work as intended through requirement-based testing
• Your developers will feel empowered to code thanks to the QA’s practical product knowledge
• With fewer production bugs to bother you, more users should adopt your software.
• QA support helps to ship reliable code “components” faster thanks to API and GUI tests.
• They also help to manage the project’s progress by identifying and responding to potential roadblocks in no time
• In the end, your application will reach higher quality in a shorter time

While the QA specialist can lead continuous testing for the project, developers should stay vigilant about writing clean code that passes their unit tests.

How to implement shift left testing

Once the QA becomes your new friend in the project, you’d like to estimate which stages of the development process need the utmost attention. Your PO can request a review of fixes for that last project. 

Analysis

• Reviewing the client’s business goals, application requirements, and the user group’s profile
• Determining the testing process and scheduling tests

Design

• Providing static requirement analysis
• Preparing data for tests and the testing environment
• Creating the test process and its standards
• Selecting the most efficient tools for test case management etc
• Providing a reasonable Definition of Ready and Definition of Done

Development

• Reviewing the application code
• Preparing unit and integration tests
• Managing TDD and BDD

Testing

• Running various quality tests
• Implementing automated tests

Deployment

• Ensuring Continuous Integration
• Improving the deployment process — also with the use of automation.

Maintenance

• Monitoring logs for anything sus.
• Running A/B tests.

Now, after seeing that list, maybe you realized QA contributes to the work of nearly every professional on the team.

While that helping hand might be needed, be sure the development team understands the left shift in testing and the QA’s role in it before they get swarmed with comments about their work 🙂

If you trust in what a QA is doing, every problem in development will soon have a reasonable solution to it, because that’s what they’re trained to do.

QAs help your product become truly competitive

Early testing helps companies in saturated markets ship software that might be the next breakthrough.

That added contribution from the QA to many development areas helps with the production of better quality software on time, which can quickly become a reported fact if you keep track of your project data. Expect better control over the project, its finances, and the code quality. You might also find that teams working arm-in-arm with a QA specialist mature faster as they learn many more software development practices to follow.