Quality Assurance strategy: questions and answers from the webinar

Today, our QA experts that conducted the webinar are going to answer some of the questions that showed up both during and after the online event.

Topics discussed in the webinars

Experts from The Software House continue to hold online webinars to talk about the latest and most interesting in the world of software development. One of our latest events was dedicated to the issue of the Quality Assurance strategy.

Conducted by TSH’s Head of QA Dawid Krala and an experienced QA Automation Engineer Tomasz Górski, it included topics such as:

• developing quality software and staying with the budget,
• choosing the most optimal testing tools,
• practical differences between quality assurance and quality control,
• the Test Pyramid vs Testing Trophy,
• the extended standard approach to software testing.

Bummed out you missed it? No worries, you’ll find both the webinar and the presentation in this article. Don’t forget to read our Q&A and find the answers to both these and many other interesting QA questions too!

Where can I find the webinar presentation?

Find even more quality content from The Software House on our official Slideshare account.

How about the full recording of the webinar?

We record every one of our webinars and upload them later to our YouTube channel. Watch the QA strategy webinar below:

What about security tests? Do you do it? Do you care about the security of the application from data leakage (GDPR)?

We do security tests and we have security audits in our projects. And yes, we care about GDPR and because of that we never work on the production data. To achieve this, we use Fogger (our very own open-source tool for GDPR-friendly data masking) that obfuscates the data that we use.

What is this “Kakunin” that you’re talking about? Where can I find it?

Kakunin is an open-source framework for end-to-end testing purposes. Underneath, it uses Protractor with Selenium, and so you have access to all their methods. The biggest benefit of Kakunin is that it provides a built-in mechanism and integrations that allow you to speed up testing processes.  For example, you can cover a complex form without thinking about how it’s implemented. Your only real consideration is business logic.

Do you do a performance test? If you do, then what tools do you recommend?

Our clients request us to validate their application performance. Usually, we create and run tests using artillery.io due to the fact that it is easy to learn and with optimal effort, you can create scenarios that are required for successful validation.

In a few cases, we also used JMeter. However, it requires much more effort just to start creating scenarios.

What do you think about Playwright?

Playwright is a Node library to automate the Chromium, WebKit, Firefox browsers. It is an alternative for Puppeteer introduced by Microsoft. There are a few great benefits of using it. First of all, it enables cross-browser web automation.

It is fast. But they also added a great concept, the so-called “BrowserContext” which allows users to run all the tests in the same browser without being forced to clear cookies and localStorage. Unfortunately, we have not had the opportunity to use it in a big project, but we definitely are going to play with it.

How to be 100% that everything works when we don’t test everything?

It’s all about the optimized approach to QA (the right strategy). You need to think about what kind of test layers to implement and also what coverage should be enough for you. In case of execution, we recommend you follow Risk-Based Testing Strategy where you can identify application areas that are critical for the business and have to work without any issues, hence you can prioritize tests and plan their execution based on the highest risk areas. 

How do you feel about contract testing?

Contract tests focus on the messages that flow between a consumer and a provider. They are great if combined with other functional tests. They do not test the side effects, so you cannot rely just on them. So this is one of the testing layers that might speed up your processes as it gives fast feedback.

What KPIs do you use for manual and automation QAs?

Due to the fact we work in Scrum, the most important thing for us is to make all planned tasks to be ‘done’, which means they are tested and accepted by the client during a demo (simply the velocity). We also measure the number of defects raised during a sprint (grouped by severity). In the case of creating manual regression tests, we focus on test coverage. For the test automation. the most commonly used KPIs are test coverage, total test duration, percentage of tests passed or failed.

Important note: do not define KPIs only to show everyone that you have them. Wrongly defined KPIs are dangerous, they may measure the wrong things or even worse, they may reward wrong behaviours.

How do you handle test data management on your automation? i.e. API testing, UI tests.

We wanted to avoid using fixtures as it gives greater chances that the tests will give false results as we had faced dynamic changes. Because of it, we decided to generate the test data once before the test run. In our case, it took up to two minutes. The test data generator included creating users, creating new business items, and making some correlations between them.

In our opinion, this way it’s much easier to maintain the data which reflects the system requirements.

What solution do you use for cross-platform mobile testing of react-native applications?

We actually used Appium for cross-platform mobile testing of native applications. For an application built on react-native, we used Detox. The good part is that you can execute the same tests on different platforms. The only required part is to implement methodWrappers that will use appropriate methods for the given device.

From our experience, it is better to run the tests on real devices as the simulators are slower and overload the computer.

What tools did you use for non-UI integration tests? Who was writing those?

To write these tests, we used “tape” JS library, the very lightweight JS test runner. Unit tests and the integration tests were implemented by developers but we also shared this work to extend the coverage, and this job was made by QAAs. For example, QAA were responsible for covering more complex scenarios where e.g. the same service was restarted a few times. 

How accurate are the results of UI automated tests now, compared to the manual ones?

Manual tests cannot be fully replaced by automated tests. There are some cases when covering them might be almost impossible but usually, it’s too expensive to try. But of course, there is a way to test positioning, colours and other design-related things.

Regarding that, there are two most popular types of layout tests. There are static tests that compare images currently displayed and the base image that the new image is compared to. In our projects, we used protractor-image-comparator. But it did not cover all the cases because it failed on the pages with dynamic content e.g. list of users.

To test dynamic content, we used the Galen framework. As explained on its official website: “Layout testing seemed always a complex task. Galen Framework offers a simple solution: test location of objects relatively to each other on-page. Using a special syntax and comprehensive rules you can describe any layout you can imagine”.

So to sum it up, it’s good to have at least some layout tests because it really does the job especially when there are many people working on the project.

What is this Puppeteer you spoke about? Can you give more details?

This is what the authors say: “Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol. Puppeteer runs headless by default but can be configured to run full (non-headless) Chrome or Chromium”.

Puppeteer is a very fast framework that can also be lightweight which is a huge advantage over other frameworks. To achieve this, you need to install puppeteer-core.

Unfortunately, they support only Chromium, so if you search for cross-browser automation, then you should take a look at Playwright that shares the same concept.

Would you recommend SonarQube for static code analysis?

This is a powerful tool that might help you detect bugs, code smells and will help you to keep the good quality of your code. AT TSH, we used SonarQube in several projects.

When will you make another QA webinar?

The Software House is constantly making new webinars. As for another QA webinar? If you sign up for Techkeeper’s Guide – our tech newsletter, you will be among the very first to learn about any new webinars, fireside chats and other online activities we constantly organise. 

Summary

Have we answered all of your questions regarding our latest online webinar? We sure hope we did, but if you still have some questions, be sure to give us a shout.