White box testing vs black box testing. Which is better according to QA specialists?

Black box testing vs white box testing explained in one sentence

The easiest definitions at this point? White box testing is based on knowledge about system structure. Black box testing is based on knowledge about system specifications.

However, our “parcels” don’t limit to a simple definition but rather the entire series of activities ensuring improved quality. That is why this article will dig deeper. You will find out:

• What’s the purpose of black box and white box testing?
• What are the differences between white box and black box tests?
• Which approach should you choose in your software project?

So let’s open the boxes and see what’s inside!

What is black box testing?

Quality Assurance specialists test the application without knowing the internal code structure, but with very good knowledge of the requirements. And black box testing is a validation of functional requirements. Black box tests are performed based on the software documentation, including a description of implemented functionalities and their requirements.

Black box testing can be broken down into:

• functional testing – software functionality testing a.k.a. “what the system does”,
• non-functional testing – performance, usability and security testing, a.k.a. “how the system works”.

Experience-based testing (also known as error guessing) is also performed to complement the scope. Quality Assurance experts validate software by using their experience with similar projects in the past and intuition for potential bugs that could creep into the project they are working on now.

What is the purpose of black box testing?

The main purpose of black-box tests is to check whether the software has been implemented according to the requirements, meets the user’s expectations, and/or contains errors.

During black box testing, you need to get positive answers to a few questions:

• Do all functionalities work properly?
• Is the user able to seamlessly go through the flow?
• Are all the paths complete?
• Does the app look great?
• Is all presented data correct?

How to perform black box testing?

Black box testing requires a lot of preparations, so it’s very important to include an experience Quality Assurance specialist who will help with:

• Analysing of software requirements and specifications,
• Getting to know the application as a whole,
• Preparing test cases,

Determining the data needed to go through all user paths (it’s super important to include positive test scenarios and negative paths).

When all aforementioned points are done and done, a Quality Assurance expert should perform test cases, come up with a detailed analysis of results, report any occurring errors and retest everything when errors are corrected.

It seems much, but all these activities have an important goal: comparing the product with requirements. If any differences are found, every single person in the entire project will receive this information and will be able to make the necessary fixes much faster and more efficiently. And we all know that time equals money, and we want to save as much of both as possible!

How to prepare data and test cases for black box testing?

Analysing input and output data and preparing test cases allows reducing the number of cases performed with precise values (determined based on the requirements).

The most common techniques for designing test cases in functional testing are:

Boundary Values

Boundary values are exact input or output values (that the user enters into the application’s fields) that are at the boundary or in the immediate vicinity of the equivalence classes.

Equivalence classes

Creating test cases based on sets of input or output data that will result in the same behaviour of the module or system. So the test will be positive for values ABC but negative for XYZ. 

Phew, that’s a lot of complicated theory, so let’s break it down through a practical example.

You test an app dedicated to 20-30-year-olds. The form contains an input field accepting integers. What are the equivalence classes and the boundary values?

Equivalence classes:

• 20-30 – positive test
• -infinite to 19 and 31 to +infinity – negative test

Boundary values: 19, 20, 21, 29, 30, 31

• 20, 21, 29, 30 – positive test
• 19, 31 – negative test 

Decision table

Creating cases from a table showing the combination of inputs and outputs with the expected results.

Let’s see that in an example too.

Your favourite online store offers discounts and free shipping. When you place an order above $100 you get a 10% discount. When the order exceeds $150 you get a 15% discount. When your order is over $200 you get a 20% discount and free shipping. What will be the decision table in this case?

[Y – yes; N – no]

Benefits of black box testing

• Performed from the user’s perspective,
• Finding differences between implementation and specification,
• QA specialists don’t have to dig deep into the internal structure of the application,
• Creating documentation that includes the test cases (two birds with one stone, you’d have to prepare that anyway)
• Test cases can be automated. 

Disadvantages of black box testing

• Requirements or specifications may be unclear, making it very difficult to write test cases,
• No certainty that all areas have been tested (it’s not possible to test everything but every additional test reduces the risk of faulty software so the more the better).

Now, moving onto white box testing…

What is white box testing?

An insight into your code and into its structure. White box tests (also called structural tests, or glass box tests) verify whether the source code has been correctly prepared, so it’s very important that the person testing the code has a high level of programming knowledge. Often it’s the developer who wrote the code (finally, a positive answer to the “Can’t developer test it?” question).

There are two basic types of white box structural testing:

• statement coverage testing – providing the tested data in a way to ensure that every instruction in your code has been executed at least once,
• decision coverage testing – a more accurate method that covers with tests every decision that software can make.

How to prepare white box testing?

Many tools are offering the possibility of preparing simple but satisfying white box tests. We will focus on the JavaScript JEST library here. There’s not much philosophy behind this choice, other than it’s so easy to write the code with.

So, we have a simple function below. You probably noticed that some decisions will have to be made depending on the input data provided.

To make it easier to understand your app’s decisions, use another tool – code2flow. It will transform your code into a much more readable graph. So, following the example above…

Once you’ve prepared a data graph, you can proceed to prepare your first white box test.

How to perform white box testing?

Statement coverage testing

You need to select the data here so that each instruction is called at least once. You can see that two tests are enough to achieve 100% statement coverage. For the purpose of our example, we will use the following inputs:

TC 1 [3; -4; 5; 0]

TC 2 [-1; 5; 0; 7]

All our tests passed (yay!), however, some lines of code are not covered by tests (boo!).

Decision coverage testing

It’s clear that lines 4, 9, 16 (red font) are not covered. In order to fix this error, add additional tests to cover those possible decisions too. In this case, your input data should look like this:

TC 1: [1;2;0;0],

TC 2: [3;-4;5;0],

TC 3: [2;-1;1;2],

TC 4: [-1;5;0;3],

TC 5: [-1;1;0;3],

…and your code will look like this:

Now the test report will be solid green.

 Benefits of white box testing

• very precise,
• simple to automate,
• it is possible to determine EXACTLY where the error occurred,
• optimising code by finding hidden bugs,
• can be prepared even when the user interface is still missing.

 Disadvantages of white box testing

• testers must have advanced programming skills to make sense of the code’s spaghetti structure,
• sometimes just not possible to test every existing condition,
• focusing on the existing code may result in omitting some functionality deficiencies. It’s hard to pick up a situation when a developer misunderstood User Story and have something miscoded. So the functionality works fine in the dev’s eyes, but in reality, it doesn’t.

Which is better? White box testing vs black box testing

The world of software testing is not black and white (pun intended) so there’s no best approach. There are some similarities between black box and white box testing – both can be automated.

However, the biggest difference is the app’s area under testing, and who can perform the tests. White box testing requires a lot of technical knowledge (therefore, they are most often conducted by developers who wrote the code,  and black box testing can be performed by QAs without such skills. So if you’re one or the other – here’s your answer.

Having said that, the best approach is to combine the power of devs and QAs and introduce both white box and black box testing. This will guarantee full coverage of the app with test cases, a thorough check of the internal code structure, and all requirements.

Good luck!