Test architect to the rescue of automated test mess – improve development efficiency & app performance

You might already know that without a skilled software architect, your app will likely suffer in the long run due to poor software design choices that have an impact on code organization and deployment, choice of technologies, or development best practices.

Is the same true for automated tests and a test architect?

As a Head of Quality Assurance at The Software House, I experienced working with a test architect on a real project. Let me tell you all about it.

In the article I’ll go over:

• what is it like to not have a test architect on board in a project,
• the benefits of working with a test architect,
• the responsibilities of a test architect and how they are different from a software architect,
• the requirements expected of a good test architect,
• and what test architects do at TSH.

Let’s take it from the top!

Life (and software development project) without a test architect

– You know what, all of those manual regression tests take way too long. How about some automated tests to speed things up? 

– Sure, let’s do this!

As a QA engineer with years of experience, I heard this conversation on many occasions. What happens next?

Well, the QA folks are first full of energy and enthusiasm to replace or supplement manual testing with automated testing. They pick the most popular framework and immediately get to writing test scenarios so that they can cover as many use cases as possible as fast as possible. And as the app grows, they continue to add new tests so that each new functionality has E2E tests, even if it’s only in the UI layer. After all, it’s all about making the app reliable for the end user, isn’t it? To top things off, a new development team begins work on a new product. Naturally, it will receive automated tests as well.

For a time, everything went smoothly. But then, all of a sudden, the test repo has become so big that the team barely understands what’s going on there.

What’s more, it turns out that each individual team had a different idea about how to approach writing automated tests in the app. As a result, there is no consistency and common point of reference for new people to adhere to.

If you don’t want to be in this place, you should put some effort into planning for test automation first.

Much like it is in the case of software architecture, in which a proper specialist creates a blueprint for the organization and style of all code, a test architect can do the same for the automated testing strategy.

Modern test automation tools, especially frameworks such as Cypress or Playwright aim to make automation testing as welcoming as possible, making it easier to implement the first test. It’s not a bad thing at all – as long as you won’t let the test tools draw your attention away from the bigger picture. It’s especially true in the case of projects in which this big picture is really huge.

To put it in different words, if you know that the project may end up growing both in terms of scope and team composition, you should consider taking a step back to think about the strategy and avoid all the maintenance and scalability issues revolving around automated tests.

But how can the test architect exactly do that?

Test architect responsibilities

If you do have access to a test architect as well as to proper project knowledge, including its long-term growth strategy, you can set about planning for test architecture.

There are a number of areas that a test architect will look into.

Test tool choice

The test architect will conduct an analysis to determine the best tools given the goals of test automation in a specific project. It’s a very important decision to make as migrating from one framework to another in an existing project is rarely a walk in the park.

Test implementation standards

Describing all the necessary test implementation standards for all application layers is yet another important responsibility of a test architect. When done right, it ensures consistency regardless of how many different people write tests in a project.

Typically, tests are written by QA specialists, but I do see a trend of developers getting interested in it. As they work on new features, they write unit tests, integration tests, or even end-to-end tests in their apps. I think that’s a good trend. It’s in tune with the DevOps culture and well worth pursuing. QAs benefit from that as well.

By working together with developers, on a code review, for example, testers become better and better developers themselves.

The test architect should keep tabs on this cooperation, create test plans, ensure the implementation of standards, and make sure that everyone understands how much easier it makes writing tests in the long run.

Writing and reviewing automated tests

With all the democratization of test automation work, it begs the question – should the test architect themselves write automated tests?

I do believe so. It’s still highly recommended, especially at the beginning of a project. Their tests might serve as a model and reference point for all other project stakeholders interested in test writing. Of course, it should not render official implementation standards and common sense test automation best practices obsolete.

As more and more people write tests well, the test architect should increasingly focus on just code reviews and making sure that everyone follows test implementation standards.

Infrastructure issues

Other than the code, the test architect needs to be wary of the infrastructure. The most important areas of concern include:

• deciding on the number of testing environments,
• proper configuration of testing environments,
• figuring out test plans to be launched at each stage of the deployment process.

The issue of infrastructure in test architecture is a big part of a high-level project overview that a test architect can and should offer.

High-level project overview

I mentioned the modern test automation frameworks that make it easier to write tests. Chances are you’ve already used them quite a lot. You probably have been more than satisfied with how they work on a local machine. But what’s next? How will they behave when deployed? How will the whole system work when the number of tests increases in the future?

The test architect is there to make sure that nobody has to worry about it.

Test architect at The Software House

All the examples above refer to a new project in which we can design an automated testing strategy from scratch. In practice, more often than not, we don’t have the luxury.

Test architecture in an existing project

At The Software House, our testers and developers have got to take over projects from other teams or join existing teams on many occasions.

In both cases, our very own test architect was able to change the project for the better by diagnosing the issues that revolved around automated testing. They were able to tell:

• if and where there are too many tests,
• if the project needs thorough code refactoring,
• or if a modest polishing effort is enough.

Let’s take a look at two real-life test architecture project examples from my own work as a QA engineer.

Test architecture case study #1 – fast & reliable CI/CD process for fintech

Let’s call this one Mercury. It was a fintech app. Our development team worked alongside other teams on a payment system. Some of them were responsible for the end-user or UI layer of the app and the rest for the backend. All teams shared a tailor-made test automation framework that allowed everyone to write consistent tests for each and every of the app’s layers.

The project involved a dedicated QA team headed by a test architect. They were responsible for the implementation and further development of the framework. In addition to that, they provided advice on how to best use it and how to go about test development in fintech specifically.

The final result was a fully automated and highly reliable CI/CD process based on so-called Quality Gates, which involved carefully planned end-to-end test sets.

Test architecture case study #2 – implementation audit for a marketplace app

If it didn’t hit home, let’s move even closer to the Earth. This one is called Venus and concerns an e-commerce app, namely a pet marketplace platform available in multiple countries.

In this project, our QA team led by a test architect joined the client’s team in order to help them polish their whole development process, including the implementation of a test automation strategy. Eventually, the development team was able to maintain a proper test pyramid involving complex end-to-end tests all on their own. They can now maintain the test strategy efficiently while developing new functionalities.

In this particular project, the test architect provided educational support meant to ensure that everyone understands how and what kind of tests should be done. The client gained valuable transferable knowledge that will carry the project on whether we remain on board or not.

Test architect requirements

I hope that you understand by now just how useful a test architect can be. Perhaps you are in hiring one.

In that case, you must be interested in the test architect skillset. Based on my experience, test architect jobs descriptions should include:

• Excellent knowledge of at least one programming language. In our case, it should be JavaScript/TypeScript. It’s an absolute necessity as the test architect shares responsibility for the test code quality.
• The ability to write automated tests for both UI and API and for mobile applications as well. After all, it’s common for a web app to have its mobile counterpart.
• Experience in creating custom automated testing frameworks based on popular solutions such as Cypress or Playwright. It makes it possible for a test architect to tailor-build a solution for each project.
• The ability to create custom CI/CD configurations for a variety of version control management tools (especially Bitbucket, GitLab, and GitHub). While this often tends to be the responsibility of developers or DevOps engineers, the test architect can take it to another level as evident in the e-commerce case study above.
• Experience in maintaining existing tests and frameworks. As the e-commerce case study shows, we don’t always start from scratch. Taking over a project or joining an existing team successfully requires both hard and soft skills and is an expertise in and of itself.
• Knowledge of Agile methodologies commonly used in modern software development projects.

If you can get a test architect that has all the skills, you will be able to tap into all the benefits of test architecture design!

Get yourself a test architect because…

I hope that by now you know that a test architect can really make or break your project, especially in the long run – not unlike a software architect. 

These two share the same responsibility of ensuring that your system is built in a way that ensures continuous smooth performance and development in the future – without needless technological debt.

Working with a test architect:

• you will avoid problems with maintenance, performance, scalability, and readability issues with automated tests now and in the future,
• you will improve and streamline your continuous delivery and deployment process, saving time and bringing the product faster to the market,
• and you will educate both your testers and other developers about how to write and think of tests, to the point that your developers may eventually be able to maintain proper test automation strategy themselves.

So go get yourself a test architect!