15 September 2022
A perfect process for a11y accessibility testing - according to TSH’s QA engineering queens
Customer expectations, technologies, and trends are always rapidly changing in the world of IT. Any organization that wants to keep up with these changes should adapt quickly in order to stay competitive. These demands are very real, whether you’re a B2B or B2C – let’s talk about the one change you can’t afford to ignore.
Technologies that are available to all users require a great deal of effort to make – we’re talking about accessibility, of course. Rightly so, a11y has become the newest trend (and also a legal necessity in a growing number of places). The implementation of accessibility solutions is a real challenge for technology-oriented companies and for businesses that have websites, apps, and any online presence. Yes – that’s pretty much everyone.
Still, applications don’t meet their full potential to include every user. A lot of them miss the mark at the design stage and set themselves up for failure from the very beginning when these features are easiest to implement.
Why is this happening? Where and when to start? In this article, we’d like to show you a step-by-step guide for implementing accessibility, based on our experience working on projects with international companies and a11y.
Why is there a global need for a11y?
According to the WHO, 15% of the world’s population has some form of disability – that’s over 1.15 billion people!
We often fall victim to poorly designed applications or web pages ourselves, often attempting to decode content with tiny font or zero contrast (think light green letters on a light grey background), or trying hard to click on a microscopic basket icon on a smartphone screen. (Have you ever just gone to get the same product elsewhere?)\
We can undoubtedly say that the need for A11Y exists and we want to meet its standards. As a person who actively participates in software development, have you ever wondered how to cover all your bases to ensure accessibility in your project?
What pushed us over the edge?
The COVID-19 pandemic accelerated many processes in online markets, forced the creation of more useful applications, affected the number of digital products used in both mobile and desktop applications, and increased the number of people in the online marketplace. It also forcibly increased the diversity of users.
Because of this, accessibility should become the “new normal” at every stage of application development. In other words, it should and will become a routine – and updates in the rule of law will see it happen whether you’re ready or not.
But don’t say you’ve been blind-sighted – the first 14 WCAG guidelines were already put in place in 1999. (That’s the year Bill Clinton got impeached.)
Accessibility benefits for driving your business
“Not only do we enjoy doing the right things, but it’s a great market opportunity in itself.”
John Browett, CEO of Tesco
Tesco – profiting from accessibility
The CEO of Tesco has a huge point, and we want to give you an example of the positive impact of a11y when changes within an in-store application resulted in £13 million in annual sales profits. This increase was measurably thanks to accessibility. This is not a small change, and it’s much more than just “the pandemic”:
Tesco’s online sales remain “significantly ahead” of pre-Covid levels in the weeks leading up to Christmas, showing like-for-like growth compared to 2019 of more than 60% giving the retailer the highest share of grocery for 22 consecutive periods.
In a nutshell, Tesco changed their app so that visually impaired customers could shop easily, and made sure that they could purchase 30 items in 15 minutes. This is a great example of inclusion working for the general benefit. They changed the site for everyone by considering WCAG and functionality for people with visual disabilities so that anyone could use it. It was not a separate application for people with disabilities They took a few steps to achieve this:
- They removed unnecessary images from the page,
- added large, clear text,
- eliminated the column layout of the home page,
- introduced intuitive layout with excellent UX (this involves a lot of testing).
As a result, pre-holiday (2001) orders have risen to 700,000 a week for an average sum of £95.
People that were excluded from the app before were:
- even young people (our vision starts declining after 40!),
- people born with mild to severe vision problems,
- people suffering from illnesses like MS that cause blurry vision,
- people who are colorblind.
This is a great example of making the page accessible to a whole new group of clients while making it even easier for existing clients to use a service.
Banking sector accessibility improvement – at Barclays
At Barclays, accessibility is about more than just a disability. It is about helping everyone in work, banking, and life, regardless of age, situation, abilities, or circumstances “.
Paul Smyth, Digital Accessibility Manager, Barclays
Barclays has made its services and bank accessible to people with disabilities. They introduced talking ATMs, high visibility debit cards, and a virtual sign language interpreter service. They started collecting awards for being the first bank that introduced a11y-friendly products as early as 2016.
Officially, Barclays says that taking people with disabilities into account is a powerful and high-quality tool for attracting and keeping loyal customers. They are not doing it “just to be nice”. According to BBC, the purchasing power of people with disabilities and their households (often referred to as the Purple Pound) is £ 265 billion in the UK alone and over $ 1 trillion worldwide.
For more basics on web accessibility and guidelines, see our article on the importance of web accessibility – we’re doing a whole series on a11y!
Implementing A11Y at the beginning of the project
The number of lawsuits regarding accessibility is constantly growing. These lawsuits, because they pertain to human rights, usually involve pricy settlements and other legal fees – not to mention penalties if actual law is being violated.
If there are no laws regarding certain aspects of accessibility in your region, you still should consider making A11Y your priority – because of CSR.
There is a real need for accessibility in the digital world regardless of lawsuits. If mobile applications and websites ignore this issue, they end up having a negative impact on society, potentially completely excluding large groups of users.
If someone isn’t able to operate a mouse, it becomes a real access barrier. If someone cannot use a keyboard or navigate menus, and forms, it is an access barrier. If an application cannot be navigated with a text reader, it is an access barrier. Incorrect magnification, bad contrasts, or colors are all accessibility barriers. This is a long list that goes on and on.
There are a lot of tools to help you test accessibility. They may test colors, contrasts, or headline sizes, but there is no magic button that will make accessibility issues go away.
So how to create applications to facilitate everyone’s access? In order to make a product function at the highest possible level, it needs attention from all people involved in the project, and feedback from their diverse expert points of view.
A11Y execution = all hands on deck
Accessibility testing is not an easy process and requires the involvement of all team members at various skill levels. Each team in the project will have its own separate accessibility-related tasks. This may include:
- Product Owners,
- Higher management/executives,
- UX specialists,
- Program managers,
- Human resources of diversity team members (if you have them),
- UI specialists.
The most important thing is to make the client understand that if their product is accessible, it will have more visitors and thus it will bring more profits. (Not to mention the benefits that CSR brings to the table.) The client is the most important part of this all-hands-on-deck equation.
If we introduce accessibility to an existing project, conduct audits, and fix errors, then it’s true that accessibility is an expensive addition to the project. If this is what you’re working with, your first step is to carefully check which functionalities need to be modified in the first place. This process is much more demanding and much harder to agree to by both the development team and the client. It can still be argued that it’s worth it in the long run, even if it is painful.
Of course, if we introduce availability from the beginning of the project, the costs are quite low and bring many benefits we mentioned above.
I mean, it looks great on paper, doesn’t it? Convince the stakeholders to include a11y from the very beginning may sound so easy, but the more you work on software development projects, the more you start to notice that there’s nothing easy about persuading clients to put more effort and time into a product than it was already needed. And as we know – time’s money and the stakeholders know that very well.
But maybe – when we talk with clients about the need of a11y – we should shift from focusing so much on terrifying lawsuits and paralyzing penalties companies pay for not following the a11y from the start, and turn their focus to what standards we as the development team want to follow and how we want to be perceived.
There’s an initiative we took at TSH: we’ve created a group of designers, developers, and QAs and together we’re trying to figure out the standards we want to set as a company.e also discuss the everyday problems we face when it comes to implementing a11y in the already existing projects.
Nevertheless, it’s still a work in progress and it sure will take a lot of effort and time before it works perfectly. We hope that one day we’ll become the true advocates for a11y.
The accessibility process in a project
The accessibility process in a project should be properly defined, and repeatable. It must be agreed upon with all stakeholders from the very beginning and become a routine in the project. Let’s face it – it’s a commitment.
Manual and automated test plans should be clarified at the beginning of collecting information about the application. An important factor will be the possibility of testing the product with a group of diverse users with disabilities to see how they cope with the functionalities. These companies offer testing services and provide teams of people with certain disabilities to test products. This isn’t always the case though – there are a lot of simulation tools out there as well.
The entire team should have the basic knowledge to create needed functionalities, properly master the process, and communicate with each other about them.
The most important task is to discover problems before users find them. Can it be done?
Automatic testing alone will not help you find everything, which is why manual testing is so important.
Automatic tests have their drawbacks, which is why a manual audit is so important in a11y. Without a human element, it’s difficult to obtain appropriate conclusions regarding visual aspects such as user interface, colors, fonts, contrast, or button sizes. Test automation tools are not foolproof – each has its limitations.
In addition, test scripts themselves are also prone to errors, so they also need to be debugged from time to time.
Guidelines and regulations – they’re actually our side, so let’s use them!
As a tech team, we have to make sure that everything is browsable in different ways:
- That the applications have alternative multimedia, and
- clearly separated content with the
- possibility of different displays (like the option of changing the contrast of font size),
- image descriptions, or
- sign language readers.
We also have to make sure that everything actually works – the keyboard, mouse, alternative navigation, touchpads – any accessibility hardware has to be tested on the app.
We make sure that the content is understandable, that all text is legible, that the page works in a predictable manner, and that it is compatible with any tools that the user might have.
Want to read more about a11y in projects?
Get a little poke every time we publish new content on a11y! (No other pokes.)
The ideal a11y project: step by step
W3 guidelines present implementation at the very beginning of the project. What does their perfect scenario look like?
The perfect plan according to W3:
Introducing A11Y at the very beginning of the project:
- This one is familiar: the cost of ensuring accessibility from the very beginning is much lower than implementing a11y at further stages of the project. It also makes it easier to maintain consistency throughout development.
- At the first glance, implementing a11y may seem too time-consuming, but if our project already has a predetermined action plan, it will not be as labor-intensive as you might think. For example, we can combine accessibility tests with other previously planned test processes, making them a part of one plan.
- Detecting and fixing a11y bugs at the starting stages will turn out to be less expensive than finding them at a later stage of development (and certainly less expensive than paying penalties caused by the lack of accessibility in the product).
- The Project Manager plays a key role when it comes to app availability. They are a person who makes sure that every stakeholder is aware of their role and the requirements that appear throughout the development process.
- At the initiation stage of a project, factors such as legal requirements surrounding a11y and understanding the goals and processes of accessibility by those working for the project and within the organization.
- WCAG requirements will help you set the goal that you want to achieve by introducing a11y to our project. An organization may establish WCAG 2.1 Level A or Level AA as the standard that it wants to follow when developing its applications. This approach will help establish a company’s policy on accessibility.
- The last thing in the W3 guidelines that should be taken into account at this stage is the understanding of accessibility by all players working in the organization, i.e. all stakeholders, developers, designers, and content providers. The biggest problem in successfully incorporating a11y into a design may be the barriers due to a lack of understanding of the whole concept. If all employees are made aware of the a11y standards in good time (e.g. through workshops – before they start working on the project), it will be more certain to avoid misunderstandings and errors.
At this stage, it is worth remembering a few rules that will positively affect the maintenance of a11y during the development stage:
- Plan the technical specifications and criteria that you want to meet in terms of a11y. They must be properly documented.
- Include work on a11y in the project budget – the sooner we ensure that WCAG standards are met in our project, the less expensive it will be at a later stage. Additionally, W3 draws attention to the separation of the fund for individual activities that can significantly improve the work on accessibility.
- Budget for and invite people with disabilities who will act as collaborators in the project.
- Budget for additional QA specialists.
- Budget for workshops with availability or additional employment.
- A11y tests were performed by QA as part of the test plan.
- It is primarily setting the requirements for a11y in the project, as well as establishing a common understanding among people involved in development. Team members should also organize a joint review during the design and development process
Elements for which designers are responsible, i.e.
- graphic interface,
- navigation elements,
- contextual changes, etc.,
should be analyzed in terms of WCAG standards and good practices.
The introduction of a11y at the same time as establishing the technical specifications ensures that elements such as contrast, font size, and interface design will be available.
Thanks to the fact that at the aforementioned stages we took care of a proper understanding and introduced A11y to the organization, programmers will be able to effectively integrate accessibility in programming at the very beginning of code writing.
Documenting solutions that improve the product in terms of a11y makes them reusable in the future. Programming for accessibility may become a standard rather than an additional task for developers. If you haven’t done this before, this is a great time to develop a process that works for you and make your own template for it.
In the end, it is worth documenting everything we learned during the implementation of a11y so that the collected information can become a source of inspiration for us in the future.
The fact that the project has come to an end does not mean the end of work on the availability of our product. Now you have to set up a plan to continually improve your product. In the maintenance phase, keep the following in mind:
- Regular check-ups of the product.
- Test for new problems during a redesign, or when new features or updates are introduced.
- Make sure that past solutions are still valid and current (sometimes, new solutions and technologies become available).
- User feedback is something that has to be collected on a regular basis.
- Maintenance teams are educated about accessibility and have a good knowledge of it.
The Facebook standard for a11y – a good role model
Of course, this is all theoretical, and the above plan might seem too perfect. We know from everyday experience how time can run out while working on the project, deadlines chase deadlines and introducing a11y as a standard might seem like too much in light of the whole rest of the project. Especially if you’re just learning the process.
The counterargument to this approach, however, may be the case described by Caterina Falleni, who works as the lead accessibility designer for Facebook.
Caterina Falleni’s case study
Caterina’s case study focused on the implementation of a11y in Facebook – an application that is used by close to 3 billion people worldwide (and according to this study, 74% of disabled people use Facebook.), confirming the theory presented by W3.
Below, you will find the designer’s recommendations, which she followed when implementing accessibility in her projects. These are processes that have been proven to work in practice for Facebook – and can be seen as universal:
- Preparing to implement accessibility at the very beginning of the development process helps to save time and resources and helps to improve the overall usefulness of the product.
- Anyone, not just specialists in this field, can contribute to introducing a11y in the project: “It takes a village to launch an accessible product. Accessibility is a team effort. “
- Establishing a common understanding of a11y standards by the entire team is one of the most important aspects of a project’s success.
- Engage people with disabilities to work on the product.
- Fear of making a mistake should not stop us from implementing a11y. Even if we do not start out as specialists, we will gain experience over time.
The Facebook Case Study shows that you do not need to be a specialist to start introducing WCAG to your project, nor do you have to stress that our product does not equal the WCAG AAA level from the beginning. You can make more and more corrections with time to make the product it was becoming more and more accessible. As time progresses, your more experienced a11y specialists can take over the training and onboarding of less experienced people.
Do not be discouraged by the number of standards that need to be introduced. Everyone who learns about a11y sees how many things have to be taken into account. Remember that everything is manageable if executed with a good plan.
Consider that too much pressure and stress at the beginning will not help anyone, and simple activities implemented gradually can contribute to achieving an increasingly higher level of a11y.
The availability controller guards the quality of the application – i.e. the role of QA in the a11y process
QA for a11y should start at the project initiation level. We do design analysis, but also documentation/specification analysis. At this stage, you can already get to know the initial ideas and functionalities and think about how we want to approach the a11y.
A11y tools – Storyboook
For accessibility tests at an early stage of development, you can also use the a11y plug-in Storybook. Storybook is an open-source tool that allows you to create the UI of components, document them, and test them.
In ‘everyday’ testing, Storybook can be used as a source for performing UI tests. We can perform initial tests of given components before they even go to the testing environment. We can also check how a given element behaves in isolation and catch inaccuracies and smaller errors early.
Storybook also offers a plugin that analyzes each component and points out a11y bugs and suggests fixing them. Of course, keep in mind that plugins are only an addition to the manual and automated a11y tests – they can never be trusted 100% and should be treated only as support, not a substitute for a11y tests. What Storybook catches still needs to be verified.
Setting in iOS and Windows reduced motions should make animations on our website (e.g. spinners while loading) static.
In general, when it comes to spinners, skeletons are a good alternative in programming – not only are they good in terms of the UX itself (they reduce the feeling of frustration when the page loads), but they also reduce the cognitive load. They show an easy-to-understand shadow layout. Here are some examples of how that looks: https://mui.com/material-ui/react-skeleton/
A11y process overview – what did we learn?
Accessibility is a group responsibility. It may seem that we can still skip this process and not think about it “yet”. We’ll fix it later when time and budget allow. We’ll worry about it when the law changes.
But designers, testers, and programmers have the responsibility and authority to show clients the need to implement accessibility. Not to mention that the laws are already changing, and lawsuits on behalf of excluded users are rolling in.
If at least one major player in the project understands how important a11y really is, they can make it possible! We hope we only reinforced how you feel about accessibility!
Our last parting words:
- Remember to look for materials and extensions for the tools you use. If your product has not introduced any accessibility improvements before, be the first to suggest it and defend this idea. Everything is in your hands!
- Working with accessibility might be an additional process, but we’re problem solvers, and this might mean the difference between using or not using an app or website for a lot of people.
- Accessibility applies to everyone sooner or later!
Now, go out there, and change the world for the better!
Is your project accessible?
We can help you be inclusive to all users!