How to merge multiple marketplaces into one and not go crazy? Migrating websites case study

If you’ve ever tried merging completely separate websites into one, you know how troublesome it might be from a technical standpoint.

Add the need for multiple languages based on location, to preserve data, and to add new features and you have a Challenge with a capital “C”.

From this case study, you’re going to find out how Pet Media Group achieved the following:

• A uniform web platform for all of their online marketplaces in different countries.
• All of the data moved to the new platform, including user data.
• No organic traffic lost – the combined user traffic of all the separate platforms retained in a new full SEO-ready platform.

Sounds interesting?

Background

Let’s start by taking a closer look at Pet Media Group.

The client

Pet Media Group operates online marketplaces for animals. Millions of pet owners trust them to find a new and safe home for their beloved pets. The client owns multiple brands in Europe, all of which are among the most recognizable ones in their respective countries. Those include:

• Pets4Homes, the single largest pet marketplace in Europe,
• Netherlands-based PuppyPlaats,
• Italy-based Annunci Animali, 
• Spain-focused MundoAnimalia,
• and the Swedish Hastnet,

… as well as many others! For more information on PMG’s activity and mission, check out its official website.

Back to the project!

The project

In each pet marketplace, the seller can offer their pets for sale or adoption to the buyers. They can also offer related services (e.g. daycare or dog walking) and accessories (e.g. beds, blankets, cages, harnesses, and toys). The buyer can find pets in their location and communicate with the seller using the built-in chat feature. Every market contains also a lot of static content for sellers and buyers, including pet advice or big breed directory.

Pet Media Group grew by acquiring more marketplaces, but every single acquisition brings new challenges. Each one was written with a different technology and in a different period of time. It was difficult to maintain or upgrade them individually.

The client came up with an idea to build their own custom solution that will provide long-term scalability and reduce maintenance costs.

That’s when they contacted The Software House.

Challenge – scalability through uniformity

The new platform goal was to solve all of the client’s problems with separate marketplaces and add new features while keeping the general focus of the company on the idea of pet rehoming.

The most important features to consider:

• advert upload flow for sellers,
• advert search flow for buyers,
• chat that connects sellers with buyers,
• mobile apps,
• support for multiple marketplaces, locales, and themes.

Once the new platform challenge is built, the next goal is to move the existing marketplaces into the new platform. In the following sections, I’m going to focus on the latter goal.

In order to understand the migration challenge better, let’s go over the requirements.

Business requirements

The businesses insisted on the following:

• Users from existing marketplaces should be able to log in to their accounts and see their adverts, conversations in the chat, etc.
• SEO traffic from existing marketplaces needs to be maintained.
• It follows that static content (e.g. news, guides) from existing marketplaces should be available on the new platform as well.

To make these happen, some specific technical requirements need to be considered as well.

Technical requirements

In particular:

• For each regional version of the one platform, you should be able to configure things such as currency, logo, categories, account types, etc.
• The platform should support multiple languages and color palettes based on marketplace settings.
• It should have one data structure so we have to somehow map all of the data structures from different marketplaces into one new structure.
• It should integrate existing third-party tools such as Google Tag Manager, Google Analytics, or Sendbird to keep historical data from legacy marketplaces

That’s a lot to take in! Where to begin?

Solutions & Implementations

I’m going to focus on the seven most important areas of the migration challenge. I’ll present the theoretical solution to each of them and immediately follow up with an implementation. 

I’m going to go over:

• Marketplace configuration
• SEO traffic
• Static content
• Internationalisation
• Theme
• Data migration

Let’s start from the top!

Marketplace configuration

The configuration makes it easier for our one platform to account for the differences between regional marketplaces and the apps they include.

Solution

In this case, our one platform contains apps such as:

• a web app,
• a mobile app.

Additionally, we’ve got a couple of separate microservices for sitemaps or the entire backend infrastructure.

My team and I decided to create one shared marketplace configuration module that could cover all of these apps (and side services) for each regional marketplace. It should include elements such as currency settings, categories, account types, the name, locale, etc. 

The marketplace configuration will not only cover the similarities, but also the differences in the behavior of features between apps of distinct regional marketplaces. During the refinement of new upcoming features, we discuss how specific features should behave in a given marketplace and what their configuration should look like.

Implementation – basic marketplace configuration

At the beginning, we need to define the enum with a list of supported marketplaces.

Then, we have to define marketplaces settings. Every marketplace is a separate object in a separate file:

Then, it’s time to implement the selector method for the marketplace configuration:

As you can see, the logic is really straightforward. The selector receives the marketplace name and uses it to return the correct marketplace configuration. Additionally, it throws an error when it’s passed an unsupported marketplace name into the function.

This is how it works for a single specific application:

We pass the environment variable into the getMarketplace method, and the output is saved into the marketplace variable that is then used across the entire application. Every app  – a web, mobile, or server – needs to implement its own marketplace.ts file since the NEXT_PUBLIC_TARGET_MARKETPLACE comes from app configuration (env, react-native-config, etc).

SEO traffic

As I mentioned before, our job was to maintain the existing organic traffic from legacy marketplaces. To that end, properly implemented redirects were essential.

Solution

Every legacy marketplace has a different URL structure that is additionally localized. We decided to implement those redirects separately for every migration based on URL mapping that was to be prepared before the migration took place.

Additionally, we decided to implement the redirects for the most visited pages (that covered 95 percent of the organic traffic) only in order to maximize the efficiency of our work.

Implementation – redirects

Our new platform has one strict URL structure that is localized on a per-marketplace basis. It differs significantly from the structures of legacy marketplaces. To keep the traffic, we had to implement redirects from the old structure to the new one. Our web app relies on Next.js so we chose to use Custom Server to implement the redirects.

It’s worth it to mention that all redirects will use the 301 status code for best SEO results. A 301 redirect indicates the permanent moving of a web page from one location to another.

First of all, we have created a small helper function makeRedirectHandler. Then, we made a list of redirects for every marketplace separately since the legacy applications had completely different routings:

Now, for the per-marketplace configuration, we created a redirect list selector based on the marketplace name:

And onto the most important part – a custom server for Next.js:

Here, we created our server with request handlers by using the Express framework. Before the Express server is created, we read our redirect lists from the selector method based on our environment variable. Then, we iterated through the list and created the request handlers within the Express server.

Static content

The legacy marketplaces had a lot of valuable static content for us to migrate. Doing so was crucial for both SEO and UX.

Solution

Since our platform integrates with a CMS for static content management, we decided to build our own service called cms-data-migrator. It uses the API exposed by the CMS in order to make bulk uploads of articles, breed directories, and other types of static content.

The client delivered the legacy static content to us in the form of CSV files. Then, we transformed the CSV data into a format supported by the CMS so that we could process it using our cms-data-migrator service.

Implementation – cms-data-migrator

Let’s see the cms-data-migrator in action.

We start by creating API helper functions. One method is for uploading the content, and the second one is for uploading the assets from the URL. Both methods need CMS_API_URL and AUTH_TOKEN from environment variables.

We can now start the actual implementation of the articles upload:

As you can see, we have to define the GraphQL mutation for creating an article. Then, we iterate through the JSON file that contains our article data. Then, we use methods from the cms-api.ts file. The first one is for uploading an image. If the article does have an image, we upload the content of the article with the image id returned by the CMS to connect the asset with the article.

Since we have multiple different content types to upload, every content type has its own command to run the migration in the package.json file, like so:

migrate:articles”: “rm -rf ./out && tsc && node out/src/articles-upload.js

To run the migration for articles, we have to add a JSON file that contains the article data. Then, we upload it in the data directory and type in the console:

npm run migrate:articles

Internationalization

Solution

The requirement is easy – one locale per marketplace. But since we have one platform now, it needs to support multiple languages.

Initially, we kept the translations in a simple spreadsheet format. However, the development and subsequent migrations showed us that it’s far from optimal. It was very hard to maintain, the list of keys was long, etc. In the end, we decided to get rid of spreadsheets with translations and we integrated our platform with Phrase. 

Each app of our ecosystem was integrated one by one into Phrase without any changes in the code (except for the configuration). We chose to keep the same translation structure of keys.

Implementation – Phrase

Since there are multiple apps such as a web app, mobile app, etc., we decided to have separate projects per each app plus one common project. Let’s take a look at how we integrated it.

To be able to fetch translations, we defined the phrase.json file with the phrase token and projects IDs to be fetched:

Then, we created an API helper function for fetching translations from a single project:

We also needed a function that will write translations into the JSON file:

Now, we can fetch translations from all projects needed for a given app.

Theme

The needs of the project led us to develop themes for specific marketplaces. Why and how did we do it?

Solution

Early in the platform development, we worked on its design. We agreed to use a strict number of colors in the app. This allowed us to come up with a color palette that could be shared across all of our apps.

In addition, we had a requirement to come up with different color themes per marketplace. To that end, we provided separate color schemes for the same interface to our React- and React Native-based components.

Implementation – theme & assets

The implementation concerned both the development of themes and the use of assets unique to each regional marketplace.

Theme

Currently, we use the same theme for each and every marketplace. However, the initial requirement was to develop a different theme for each marketplace. Each theme would have a different color palette. 

That needs to work for both the web app and the mobile app. Still, we wanted to keep the code in one place. We settled for the same mechanism that we used for marketplace configuration.

First, let’s define our themes:

Then, we need to implement a theme selector:

As you can see, we copied the pattern from the marketplace selector. As such, we return the correct theme for the given marketplace name!

The final result:

Honorable mention: We employ the SCSS modules in the web app. In order to use a single source for themes, we have an additional script that parses a JavaScript object into SCSS files using a variable declaration.

Assets

Every marketplace has a different set of assets such as a logo, favicon, or some images. And then, there are some assets that are shared across all marketplaces. Given these circumstances, we agreed on the following public directory structure:

As you can see:

• We have the fonts directory where we keep all of the font files. We use the same font on every market.
• We have the images directory where we keep the common images.
• We have the pets4Homes and puppyplaats directories that contain marketplace-specific assets such as logos.

When we want to load a marketplace-related asset, we just use the familiar environment variables: 

NEXT_PUBLIC_TARGET_MARKETPLACE

<img src={`./${process.env.NEXT_PUBLIC_TARGET_MARKETPLACE}/images/logo.svg`} />

Data migration

The main goal of the data migration was to allow existing users to log in to their accounts and access all their data from the legacy platforms.

Solution

Since each legacy marketplace was the work of a different team, at a different time, their database structure was bound to be different, possibly even localized (to a specific language such as Spanish, or Dutch).

On the other hand, our new platform will have exactly the same data structure.

Due to these two factors, our backend colleagues decided to create a dedicated service for data migration. That service will map the legacy database structure into a new structure supported by the unified platform in a number of steps. The only thing required during the marketplace migration is implementing a set of mappers from the old structure to the new one.

Implementation – read more

Our team created a dedicated service for data migration called data-migrator. I will not go into details about it here as the backend part of the migration is covered by my colleagues in another article.

Deliverables

We successfully met all the business and technical requirements.

Business

The business department is satisfied with our work due to a number of reasons:

• We solved problems arising from having multiple apps delivered in different technologies. The client can now implement a feature once and deploy it on multiple markets.
• We maintained SEO traffic from legacy marketplaces.
• Users can still log in to their accounts and enjoy a lot of new features at the same time.

Of course, none of these would be possible without proper technological choices and their implementations.

Technology

In particular:

• We delivered a fully configurable and scalable solution.
• We implemented two custom services dedicated to data migration.
• We provided e2e test suites that simplify the deployment process for marketplaces.

Let’s sum things up and see what this experience taught me and my team!

Merging websites into one multilingual marketplace solution – site migrations lessons learned

If you made it all the way here, you must be considering site migration of some kind. I’ll leave you with a couple of final thoughts on website migrations that can help you.

• We learned how to implement and deliver configurable features and solutions within multiple various apps with shared configuration. We achieved this by creating a marketplace configuration module that is easy to understand, use and extend regardless of whether we run frontend, backend, or a React Native app. The site’s performance is great regardless.
• We managed to keep the organic traffic from an entire site of each legacy marketplace in a new platform by implementing redirects using a custom server for Next.js.
• We taught ourselves how to do releases and survive them during the marketplace migration by implementing a couple of software release management best practices, including a migration checklist. These included a “Go Live Plan” with a list of steps to do during a release day, test releases on beta environments, a rollback strategy, and a maintenance page for users to get the latest information.

Are you also thinking of migrating your web portals? Whatever technologies, goals, and assumptions you have in mind, you’re going to need developers experienced in delicate projects such as this.