Make microservices more efficient & scalable with Backend For Frontend: Qiwa tech study

We have recently worked on a project that provided a perfect opportunity to use the popular Backend For Frontend pattern. But no matter how fancy the technology, it’s nothing without business context. Let’s start with some basic information about the client.

Background

Let’s start with some business context.

The client

Back in 2018, we embarked on a new project with Takamol Holding, a Saudi government-owned company in charge of developing an innovative IT system meant to play a pivotal role in the digitalization of business in Saudi Arabia, the labor market in particular. The goal was to serve a variety of different users, both employers, and employees. As such, it required a number of separate functionalities. 

If you want to learn even more about the platform and business context, we highly recommend you read our previous article on the Qiwa digital transformation software.

Starting from now, we’re going to go more technical and focus on how we made it happen as developers.

What is it all about?

Since the platform was to serve multiple purposes, a decision was made on the client’s end to divide it into a group of smaller sub-platforms. They would be represented as tiles on the homepage of the service.

Each sub-platform represents a single business operation, including (but not limited to):

• starting a business,
• signing a contract,
• transferring employees from one company to another.

The objective here was to move all of the services to the internet. The client had its own API service, which communicated with the government’s database to complete various business operations.

As The Software House team, our goal was to develop the visual layer of a number of such individual sub-platforms as well as integrate them with various services of the client’s API.

Challenge & solution

The complex nature of the platform posed a number of challenges.

The challenge – be ready for today & tomorrow 

From the start, we wanted to make sure that the platform would be well-equipped to deal with present and possible future issues. We needed to plan it really thoroughly: 

• Not only does the system consists of various sub-platforms, but the number of platforms is expected to grow in the future. Hence, scalability is of utmost importance.
• The number of users will grow too. In fact, the plan is to serve all employers and employees in Saudi Arabia eventually.
• Each sub-platform provides a separate service. A particular service may be relevant just for some of the whole platform’s users (e.g. just for employers or just for office workers.). Considering this, it’s easy to see why this division makes a lot of sense from the business perspective.
• Each sub-platform will communicate with only selected services of the client’s API – only the ones it needs to do its job.
• The client cooperates with other external contractors. The architecture makes it possible for them to focus on one sub-platform assigned to them. In fact, they don’t even have to be aware of other sub-platforms’ existence most of the time.

Our answer – Backend For Frontend

We agreed the best way for the frontend to work with such an architecture is to use the Backend For Frontend design pattern, a popular architectural pattern, an alternative to the API Gateway pattern. Its defining feature is that each separate frontend app has its own dedicated API, which communicates with external APIs. That way, different frontend apps of varying purpose and structure, including those for mobile applications / mobile interfaces or desktop web UI, can have their own dedicated backend services.

As you can see in the diagram above, each “backend for frontend” communicates with various API services under the hood. Each backend for frontend has three primary functions:

• receiving responses in various formats.
• managing and resolving errors it encounters.
• delivering the response to the frontend layer in the desired form (typically in the JSON format).

Backend For Frontend in action

Before we get to the implementation, let’s take a closer look at the BFF pattern. It is best used in very specific cases:

• projects based on the microservices architecture,
• projects that communicate with external services that require security authorizations,
• projects in which it is desirable for various separate teams to work independently on different parts of the system,
• and projects, in which client apps call for separate APIs (e.g. separate APIs for mobile apps, web apps, desktop apps, etc.).

It looks like our project fits the bill just about perfectly. Nothing’s stopping us from implementing Backend For Frontend!

Backend For Frontend implementation

We’re going to divide this section into the actual implementation and our thoughts on the pros and cons of BFF in projects of this kind.

BFF in our project

This diagram below depicts the implementation of the BFF pattern in our project for two separate sub-platforms/services (employee contract management service and employee transfer service).

There are multiple BFFs or sub-platforms. The employee contract management service is one of the biggest sub-platforms. It consists primarily of three views:

• a list of contracts table,
• a contact creation form,
• and a contract preview.

Each of these views communicates with one or more endpoints of a dedicated API (optimized backend for frontend). The latter on the other hand communicates with one or more external services in order to pass responses in a given format.

The other sub-platform also has a dedicated API. It communicates with the same external services as well as with an additional one – an employee transfer data service.

There are two basic conclusions to draw from it:

• The BFF API communicates only with the services it needs to work.
• The frontend apps do not communicate directly with external services. The latter is hidden from them and protected with proper security keys.

The fundamental part of the BFF pattern here is the contract made between the frontend app and its dedicated backend app. It includes a specific structure and format of frontend requests as well as backend responses. The contract is independent of the external services. When the latter change, all that is required is to modify the communication between the dedicated backend (for frontend) and the external services.

What did BFF give us? Backend For Frontend benefits

The Backend For Frontend pattern provides a number of benefits from the frontend perspective.

• No need to go through the hassle of integrating with various external services. The dedicated backend handles all the work.
• No need to modify anything on the frontend side when external services change.
• All of that means that you can focus more on polishing your user interface.

There are also clear BFF benefits for the backend:

• All of the data handling logic is in one place.
• Since the dedicated frontend app is the only receiver of data, you know exactly what type of data to prepare. 
• Security keys for external services are stored entirely in the dedicated backend. They are not available in any way whatsoever to the frontend app.

In addition, going for the Backend For Frontend design provides various general benefits that welcome in just about any project:

• The ability to start development quickly. After all, each team only needs to know the codebase of its own separate service. Very useful when a new team takes over a project.
• The very same modularity means that you can use different technologies for each separate service. You can pick the best technology for the job, or simply the one each team likes best. 
• Each service is a greenfield project. There is no legacy code. You are free to use the latest version of frameworks or libraries for each service.
• Simplified refactoring – when there is an error, you know where to look for it. And once you find it and remove it, you know that it will not affect other services.
• Each service has a separate repo – it makes for an orderly and easy-to-browse-through project.
• Independent deployment of each frontend-backend pair, which reduces the number of things you need to worry about when shipping.

The best way for you to experience the benefits of Backend For Frontend is by taking a look at the code.

BFF pattern benefits – a practical example

Let’s see how you can use a dedicated backend to make sure that your app integrates easily with external services.

The backend’s API sends a request payload to an external service. The payload includes the ID and SECRET fields in the header. For security reasons, this data can only be stored in the backend. When the request is processed successfully, the API receives the following HTTP 200 response:

When there is only one CompanyActivity, the external API might send a single object instead of an object table. However, thanks to the dedicated backend, the response the frontend eventually receives always has the same formatting:

In the browser, the response looks like this:

BFF shortcomings to account for

What about BFF cons? When it comes to frontend:

• The development is extended as the process of integrating the backend app with external services can be time-consuming. 

As far as the backend goes, the biggest problem with BFF is:

• High risk of repeating pieces of code, especially when the dedicated backend apps often community with the exact same external services.

General BFF cons come down in particular to:

• Modifying an existing working feature may entail the need of changing the contract between the frontend app and its dedicated backend.
• Since its dedicated backend is independent and can use different technologies, there is a lack of cohesion and consistency in terms of technology choices. It may pose maintenance issues.

All the cons of Backend For Frontend have to do with the size of the project. The larger it is, the less of a problem they are. When you work on a large project like this one, you can (and should) afford the extra setup work in order to reap the scalability benefits in the future. When you do, the BFF pattern will hardly have any cons for you.

Deliverables

Below, you can see the project in numbers.

Our team worked mostly in Node and React, having switched from Vue.js. The backend used Laravel. You can learn more about a wider technological context of the project in the previously mentioned general Qiwa case study.

Let’s now put our effect into tangible business benefits.

Business Implications

The project brought improvements to the client’s bottom line of both quantitive and qualitative nature.

Quantitative impact

• By introducing so many services, we made it possible for separate teams to continue their work on them independently from one another.
• The project contributed to the digital transformation of the public sector in Saudi Arabia – one of the stated missions of our client.

Qualitative impact

The project will bring a number of other benefits along the way.

• It’s fully scalable, making it easy to add new features/services in the future.
• It’s very stable. Since all the services are independent of one another, even if one malfunctions, it will not affect other services.
• It solves real problems of Saudi employers and employees, making it possible for them to solve many everyday problems way faster.

Backend For Frontend is surely interesting, but the story is not really about the BFF pattern itself.

Conclusions – is BFF right for you?

The Qiwa challenge has proven very successful for both the client and us. We always love it when we get a new major challenge from a company we have already been working with for a long time – it’s proof of just how healthy and fruitful our cooperation is. And that’s how this backend service challenge started.

In the duration of that project we managed to:

1. diagnose the technological needs of the client based on its business requirements,
2. find the right patterns and technologies to turn the theory into an actual working application with multiple services,
3. deliver a high-performing scalable backend services ready for painless future expansions.

While Backend For Frontend isn’t a cure-all, chances are that it might be just what you’re looking for in your microservice architecture.