What is usability?

I prefer to explain things in practical examples. Imagine your favourite restaurant has an online food ordering system. You go to the website, browse the options and make your picks. Were you able to scroll the menu easily? Did you have problems with finding the “order” button? Was anything confusing, hidden or making it difficult to place an order? What were your feelings after completing the order?

If everything went smoothly, the application was very well designed for sure. If you’re not completely satisfied or had some troubles, that means the app has usability problems. Probably, because the usability testing might have been missing from the quality control process.

What is usability testing?

According to usability.gov, “usability testing refers to evaluating a product or service by testing it with representative users”.

Thanks to usability testing you can:

• check whether the product meets the intended purpose and software requirements,
• establish which areas are difficult or confusing for the user,
• analyse the conversion rate,
• detect deficiencies in the app’s operation,
• review end-user satisfaction.

How do you plan usability tests?

Good practice shows that you should thoroughly prepare for each test – usability tests included. In this section, I’ll tell you what needs to be done beforehand.

The Quality Assurance specialist conducting the test must perfectly know:

• product requirements,
• business needs behind the product,
• full range of product functionalities,
• target audience.

Based on the aforementioned points, you should define two things: test scenarios and the ideal test group. Additionally, an important element is setting the main goal of the tests and collecting feedback about the application after the test, e.g. by completing the survey by the participants.  

Wait, what is all that? Please don’t freak out, in a moment you’ll see that the planning phase of usability testing is quite straightforward. So let’s take it step-by-step.

Writing test scenarios in usability testing

The scenario should show a real situation that the end-users are going to face while using your app. It’s important to set a clear path with certain steps to follow. It could be picking a dish from the menu and adding it to a cart, setting up an account, payment procedures, changing personal data and so on. I’ll show you some very practical examples later.

Picking a test group for your usability tests

In a perfect world, we should collect participants that fit into the defined target group of end-users. After all, you want to see how people you’re directing the product to will react to it. If you have a possibility to do that – super, it’s every QAs dream.

If not, it’s better to invite some colleagues (who had nothing to do with the project, obviously) to complete some tasks, instead of dropping usability tests altogether.

How many people do you need for conducting usability testing?

You should designate the time you want to spend on performing the test and analysing the results. Then, choose the number of participants accordingly. If usability tests are a part of regular, basic testing, feedback from five users is enough. If they are more extensive, the size of a group should be larger.

Quoting Nielsen Norman Group:

The test carried out on only 5 users allows you to discover nearly 85% of errors on the page. With at least 15 users you discover all the usability problems.

What’s the main goal of the usability test?

You need to establish what will you check and measure, e.g.:

• What’s the level of users’ satisfaction after using the app?
• Was the app easy to use?
• Are the ways of presenting data clear?
• Is the information presented understandable?
• Is the application readable?
• Which design style or colour version the users like better?

How to conduct usability testing?

When everything is planned, you can start with usability testing. How do you conduct such a study?

• The participant performs the task using the tested application. It should be a specific scenario that will allow users to go through the application’s main functions.
• The leader observes the participant’s progress through the individual steps and their behaviour. It’s very important to take notes for the upcoming analysis of test results.

The usability test can be performed remotely or in a specially prepared place. The participant can also record or take screenshots of their usability test, and later send them to the organizer.

There are many tools that allow you to automatically collect data, such as Loop11, UserZoom or hotjar. After collecting data from particular tasks, these tools facilitate the result analysis and show the final usability test results.

Now, let’s see how the theory works in practice…

Usability testing template

In this usability testing example, I’ll stick to the food ordering app I’ve mentioned at the beginning of this article.

The functionality: ordering a meal in a selected restaurant.

The goal: checking user satisfaction after using the app’s functionality

What’s tested: how long it takes to place an order, what user behaviour and emotions are evoked by using the app, where the user spends the most time, the final user satisfaction

The test group: people aged 18 – 40

The first tasks: order fish & chips from “Crazy Fish” restaurant to your address

The second task: search for dishes or restaurants you fancy and add them to favourites

Why two separate tasks?

Task one may seem simple, but this scenario will require a few steps from our test users: creating an account, searching for the “Crazy Fish” restaurant, searching for a particular meal, adding it to the cart and completing the order.

Task two is a classic open-world scenario. Here, the user decides what they want to order, they will browse multiple screens and discover the app for themselves.

Sometimes A/B tests are included in the usability testing too. It’s a good occasion to present the test group two (or more) variants of the same application. Based on their opinions and (dis)satisfaction, you can decide which version will be implemented and which are going straight to the bin. Tools like Split.io can help you with that task.

Usability testing results

After the test is done, it’s time to analyze the results. On their basis, you define the areas that caused any difficulties to the users, the paths that the participants used to complete the tasks. Then, compare the results and draw conclusions.

During usability tests, you can examine the conversion rate. It is expressed as a percentage of people who performed the desired action. Therefore, the conversion rate expresses the effectiveness of the app or website (also used in the advertisement).

Usually, if 50% of users pass their tests successfully, your conversion rate is acceptable. However, if less than 40% of users correctly perform the task, you need to fix a certain function or a whole platform (depending on which elements you were testing). 

Let’s go back to our food app. It has a very large button that leads the user to search for a restaurant. The search engine can also be accessed in other ways. But the app’s main assumption is that the large design with a button should be the main stimulus for users looking for the search option.

If 10 participants take part in our study and six of them performed the desired activity (in this case, pressing the large button to search), the conversion rate is 60% (6/10*100%). A high ratio means success!

The cherry on top of usability tests – the final survey

As I mentioned before, the completed usability test can be supplemented with a final questionnaire for the test group. The survey is a great chance to obtain additional feedback that participants couldn’t give during the test. The results will allow you to measure the examined indicators.

How to create a usability testing survey? Based on my experience, questions with answers on a scale work quite well, e.g. On a scale of 1-5…

• Is the application understandable?
• How intuitive was it to use the app?
• What rating would you give the app in the store? 

You can also try some open questions:

• What did you like the most about the application?
• What you didn’t like about the app?
• Were there any features missing?

The results of this survey should also be thoroughly analysed. If all users gave a high rating for the intuitive, it’s a great success. If not, you should probably delve into this topic and find the reason why users were unable to work with the application.

Benefits of usability testing

Usability tests provide you with:

• a credible opinion of application users
• good practices for creating applications fulfilling user expectations
• option to be performed remotely
• a fairly low cost for extremely valuable guidance
• much better understanding the needs of the target group
• information on improving the attractiveness, efficiency and flexibility of your app
• experience for creating user-friendly applications in the future

By inviting work colleagues from other projects to test your app, you introduce an element of integration as well. 😀

Usability testing examples show that it doesn’t have to be difficult

 Your test results are a foundation to propose changes to the application that otherwise wouldn’t probably happen. I know I used a pretty universal example of a food ordering app that most of us use on a regular basis. But imagine if you were to design an app for a niche industry.

For a Quality Assurance specialist, any experiences with end-users are a goldmine of great feedback that will allow you to create even better applications in the future.  

Three cheers for satisfied users!