Continuous Integration for E2E tests (2/4): CircleCI configuration for test automation

The scenarios I faced during the aforementioned project weren’t too complicated, so I managed to write down the tests swiftly. It was time I decided to check how CI works in practice. I wanted to experience how to deal with it – how difficult it is and if I can do it myself or if I rather need a hand of DevOps. In the previous part of the article, I focused on reporting with Cypress and Mochawesome. Today, we’ll take a closer look at automating E2E process with CircleCI. 

Jenkins? Not this time.

My first thought was to use Jenkins. It’s a pretty popular tool with multiple functions. Unfortunately, Jenkins is as popular as well-known for its coarse UI. I’ve experienced it a few times and that’s why I decided to use CircleCI instead. 

CircleCI – simple setup

Setting up CircleCI is straightforward. I started by creating an account on CircleCI website. You can use one of the available options, such as signing up with GitHub or Bitbucket. 

Using GitHub or Bitbucket is pretty useful because once you are logged in, you can see all the repositories from your GitHub/Bitbucket account in the CircleCI dashboard. After choosing a repository where tests need to be performed, you have to add a configuration file that is used by CircleCI.

To do it, you have to start by creating a new folder “.circleci” in your project. It must contain “config.yml” file. For more information about configuration, you can visit CircleCI official page.

What are orbs? As per official CircleCI website – “orbs define reusable commands, executors, and jobs so that commonly used pieces of configuration can be condensed into a single line of code.”

Description:

• version: which version of CircleCI is used,
• cypress: what orb you want to use,
• cypress/run: it simply runs tests,
• executor: it defines the Node version you use

Once you perform all the changes in a branch – you will notice the build starts running with your tests in CircleCI. If tests fail, you receive an immediate report with all the information about the test failure in your email.

CircleCI nightly run setup

If you need to configure the start of a build at a certain time (for example to verify tests every night) – I recommend using the configuration presented below.

Description:

• nightly: in this place you can put any name which describes the time test need to be performed,
• triggers: it is used to communicate to CircleCI that you need to plan a process with a certain flow,
• schedule: it indicates that you plan/schedule the test,
• cron: it is for defining the crone to set how your build is started (to generate a proper crone, I recommend using Crontab); it’s important to remember that CircleCI doesn’t accept */1 neither commas,
• filters: it can be used to define processes, 
• branches: it communicates CircleCI to use certain branches, 
• master: it narrows the use of CircleCI to the use of branch master (you can define any existing branch).

You can wonder why section jobs is repeated twice. It’s because the first job will be always started after the newest changes. The other job will trigger the run of night tests. With .yml files, you need to remember the proper amount of spaces and tabs, because your CI tool may end up not performing your script.

CircleCI advanced setup with the artifacts

The last example I will present – starts your tests from Docker. Then, it will cache node_modules to save time and then, it will store the artifacts which are relevant for the test (such as reports described in the first part of this article or a folder with a video of the tests). 

You can check the full example on a repository in branch circle.ci

Definitions:

• docker: it defines what image you use when using Cypress (in environment section you can define xterm – it helps you get coloured syntax in the terminal),
• working_directory: it’s a place where your app and tests will be built 
• checkout: it can perform checkout from your branch to a working directory, 
• restore_cache, save_cache: https://circleci.com/docs/2.0/caching/#using-a-lock-file those commands can save your time during caching node_modules if your packages haven’t been changed during the project,
• run: this section runs the command for testing (it was defined in the previous part of the article in the section about mochareporter) 
• store_artifacts: you can provide the paths for a folder which you need to store for further verification.

Let’s take a look at the application itself.

CircleCI – application overview

Below, you can see a screenshot from the CircleCI application. Similarly to the reports – everything is segregated and clear. The information you can find in the section jobs is:

• who released a build,
• a commit name,
• the time duration of a build,
• when a build was released,
• a build status.

Once you open a certain build, you can see the detailed information about it. You can go through all the steps which have been performed. If any failure appeared – you can easily verify what was wrong in your CI. Whether was it caused by wrong configuration or tests finished with an incorrect result.

It’s also worth digging into the artifacts section. You can find there full access to your folders with reports and video recordings. CircleCI advantage is also the fact you don’t have to download files – you can simply take a look through them in a browser.

What’s the deal with CircleCI configuration for test automation?

Thanks to the configuration above – you can be sure that your tests will start every time you need them. They will store all the necessary data and will be set for a certain time. In one of the next parts of this article, I will focus on a similar configuration for Bitbucket Pipelines and plugging into Slack reporter. Thanks to that – you can be notified whether your tests turned red or green as early as your first coffee and Slack scrolling in the morning. Stay tuned!

***

If you want to learn more about CircleCI, I recommend visiting the websites below.

Cypress documentation for CI 

Exemplary project on the branch circle.ci