QAA DIY: How to introduce automated tests and not lose your mind during the process

In this article, I’m going to introduce The Software House’s very own end-to-end test automation tool – Kakunin. But first, let’s take a step back and find out what’s really up with automated testing today.

It’s the automation, stupid!

Let’s face it: the more complicated a web app is, the more important the role of a Quality Assurance team becomes. In many cases, manual tests are all you need. You can simulate the user’s activity, check all the modules, verify the design… However, with every new feature and every new version of the app, manual tests get more and more inefficient. And automated ones become a necessity.

It’s especially noticeable when speaking of regression testing. But there are even more situations when automating the tests can be of great help. Firstly, such automation goes along well with the good practices of Continuous Integration (CI) and Continuous Delivery (CD). Secondly, while every manual tester has his/her own way of doing things, automated testing scenarios can be highly standardised. Thirdly, well-written automated tests can be executed by virtually any development team member. And at any time.

There’s, however, one thing which we cannot forget about: to write automated tests you need a person who knows at least one programming language. Not a very low entry level, is it? But we’ll come to that later. Let me now tell you a short story…

A short story of a serious problem

The story includes a lot of software developers and testers struggling to get along, making many silly mistakes along the way, so I promise it’ll be fun. In The Software House, we’re focused on developing custom apps – for various clients coming from different markets. No ready, off-the-shelf products here, sir. Such an approach means that we rarely get bored. However, on the other hand… Yeah, you guessed it:

Each and every project requires a special approach, pretty often a different technology and, in the end, it’s almost impossible to standardise all processes.

Imagine such a situation: you have a tester in your company. He’s always performed manual tests, so he’s not very good at programming. However, you decide that you want to introduce automated tests on a larger scale. You convince the tester that it’s a really great opportunity for him, you find a developer who can tutor the tester (and we all know that not every great developer is a good mentor)… Everything’s time-consuming but you believe that it’ll eventually turn out to be rewarding.

Then, after about a year, you want to move your brand-new automated testing engineer to another project. Nope, mister. He’s been working with AngularJS which means that he’s familiar with the Protractor testing framework… And the second project’s based on React. Zonk! It’s not a problem that cannot be overcome but on a larger scale, it becomes quite frustrating.  This is pretty much the situation that we’ve been dealing with in The Software House. And I’m really surprised that none of our board members lost all of his hair during the process.

In effect, after a while, we ended up using a few different tools for testing. So, there was the aforementioned Protractor – easy to install, with a huge community and great support for apps made with Angular but requiring pretty advanced knowledge of JavaScript and not very compatible with frameworks other than Angular.

Then we had Nightwatch – easy to install and with the same testing mechanisms for both the Angular and the non-Angular apps but, again, pretty demanding in terms of JS proficiency and (at least according to our experience) slower than Protractor. Last but not least, there was Cucumber – based on the very simple and natural Gherkin language, which makes it useful for those who are not JS masters (not only manual testers but even your non-technical business clients) and compatible with other frameworks (Protractor, Nightwatch, Behat…), however, unusually hard to install.

I’m sure that this story sounds familiar to many of you. Each framework has its pros and cons, so you end up using more and more of them. In the world of software development, we’re quite used to that, so it’s not a big problem. But is there really no way of making things better?

What if…

Enter Kakunin

What if we create a tool combining all the pros (and, if that’s possible, none of the cons) of the frameworks mentioned above? I know what many of you have just thought of: “Classic IT scenario. There are 10 competing standards, so you try to come up with a new one which will cover all cases. The effect is that now there are… 11 competing standards”. However, in The Software House, we’ve tried to do something different:

Instead of developing a whole new framework, we’ve decided to LITERALLY combine the best existing tools.

Say hello to Kakunin. Firstly, it’s based on Protractor and Cucumber, so there was no need of reinventing the wheel. Secondly, it works with every JS framework, so it doesn’t matter if you’re a fan of React, Angular, jQuery or anything else. Thirdly, the scenarios are written using Gherkin, so the tests can be run by people who aren’t familiar with programming at all. Finally, Kakunin is extremely easy to install, as the whole installation process is based on a question/answer system.

When testing with Kakunin you use so-called steps. Such a step may, in fact, contain a few operations, sometimes pretty complicated. For example, one of the most often used steps fills a form on a web page with the data which we tell it to. We’ve prepared c. 50 predefined steps for you to use (and each of them was tested on a substantial number of web apps, so you may be sure that they’ll always work in the same way). All in all, it means that you can run tests with Kakunin even before you learn to programme.

On the other hand, none of the features of Protractor and Cucumber is blocked in any way. It means that if you’re already familiar with those frameworks you can use your favourite commands. The fans of Protractor will be probably glad to hear that they can work with locators ($, $$, by.css…). And those who used to perform tests with Cucumber will find their beloved mechanisms like hooks, steps and using tags to run the individual scenarios.

The same goes for operating systems – it doesn’t matter if you prefer to work on MacOS, Linux or Windows, as installing Kakunin is equally simple on all of these OSs.

What’s more, Kakunin’s architecture is based on modules. It means that it’s fairly easy to add new features. For example, Kakunin already supports testing MailTrap emails but it’s not a problem to add the support for MailHog or any other similar system. It also works well with WYSIWYG editors – support for CKEditor is already there but if you want to use Redactor (or anything else), you’re welcome. Handlers, transformers, matchers, dictionaries… All of them let you customise Kakunin to work the way you want it to.

What’s next?

The current version of Kakunin is 2.0, so it’s not some new, unreliable framework. However, we’re constantly working on making it even better – all in all, we use Kakunin on a daily basis in The Software House, so we really want it to be a top-notch tool. What can you expect from Kakunin 2.1? We want to improve the quality of the code, increase the code coverage for some of the steps and get rid of those steps which turned out to be of no use. However, probably the most important improvement is the integration with BrowserMob Proxy which will make it possible to test the website’s efficiency.

What’s next? Version 2.2, of course! As more and more people start using Kakunin, the amount of feedback also increases. As for now, we plan to add one pretty important feature, i.e. testing REST endpoints. We want Kakunin to validate the answers according to predefined criteria – e.g. by checking if the answer contains a specific element or if its structure is correct.

Too long; didn’t read

In this article, I’ve tried to get you interested in Kakunin – a framework for automated end-to-end testing of web apps. Is it a solution to all your problems? Probably not. However, it brings you at least three considerable advantages. Firstly, it has a very low entry level – easy installation and Gherkin-based scenarios make running tests possible even for those who are not very familiar with JS development. Secondly, it isn’t limited to basic features – if you’ve already been performing automated tests for quite awhile, you’ll be happy to find your favourite Protractor or Cucumber tools included in Kakunin. And yes – it works well with every popular JS framework, whether it be Angular, React or anything else.

Finally, using Kakunin in a software company changes your way of thinking about the tests – instead of coming up with new mechanisms for each and every project, you rather start wondering if a specific functionality may be reused in testing other apps.

Such a strategy is very time-effective, as you can use the same tests over and over, only with some little changes. All in all, it’s the automation that we’re talking about, aren’t we?

Of course, if your company is focused heavily on BDD, the process of reusing steps won’t suit you very well. It’s also not true that by introducing automated test you can completely stop performing manual tests. Verifying the design, the visual part of web apps, is still a thing which only the manual testers can perform well. At least for now.

However, we’ve been using Kakunin in The Software House on a daily basis for about a year now and I’m confident that developing this framework was a highly profitable investment. It’s an open-source tool, so you can give it a try anytime. If you’re familiar with the problems which I’ve described a few paragraphs back, I’m pretty sure that Kaunin will come to you as a welcome relief.