How a startup cut cloud costs from $30K to $2K per month by nesting 1K websites in 1 Next.js app

In this article, we’re going to show an incredible story of cloud cost optimization that dispels some myths about how to reduce cloud costs. But that’s not all.

What you’ll learn

Stick with us and by the end you’ll know:

• How an innovative company was able to quickly pivot their business model under the tough and unexpected circumstances of the COVID pandemic.
• How we discovered the flaw of the system that caused massive cost bleeding of the cloud infrastructure.
• How we drastically improved the system’s scalability by changing its architecture.
• How we run the old system and the new one at the same time while working to complete the hybrid migration.
• How we customized Next.js to be run with multiple domains.
• What it all meant for the future of the system.

So… what was it about an incredible story?

Reduce cloud cost – a time-consuming grind?

The story you’re about to read challenges a notion that many capable developers and managers seem to have – that cloud cost optimization is bound to be a slow process that rewards you only with small gains over a long period of time. And that cost cutting is the center of it.

If you approach it with these assumptions, you’re not going to be very passionate about it. What’s more, these assumptions simple aren’t true. Our experience shows that the best results are achieved by companies that first and foremost want to improve the efficiency of their system and architecture. The cost decrease is merely an afterthought. That kind of optimization becomes a transformational process that heals your architecture and the entire business centered around it.

And yet, that approach still doesn’t tell the whole story. Because the truth is that it is possible to decrease cloud costs quickly. Not every app has those hidden optimization opportunities, but when they do exist, discovering them might result in a multi-fold cloud cost decrease. There are no magical tricks involves.

It’s not too late

Another myth is that you can never really optimize a legacy system. You have to begin from scratch. Well, the truth is that:

it’s never too late to begin cloud cost optimization efforts

Speaking from our experience, we know that most of troubles with excessive cloud infrastructure costs stem from rushed decisions regarding application architecture – the State of Cloud 2023 report from Pluralsight shows that 44% of businesses adopt the latest cloud products as soon as they hit the shelves, all while nearly 50% of executives can’t get cloud costs under control – what a combo!

As a result of such actions, your company uses up a lot more resources than it actually needs. And since the majority of cloud customers use the pay-as-you-go model, the costs keep piling up.

But we have some good news for you:

• A lion’s share of this cost is most definitely avoidable.
• Even if you made non-optimal decisions regarding your infrastructure and architecture at the beginning, you can still turn the situation around. It’s not too late!
• In order to achieve it, all you need is a team of experts ready to go the extra mile to create a highly customized solution tailored to your needs.

The project that we’re going to show you in detail confirms all of these assumptions.

It all started with a successful implementation of a web app that allows its users to upload short videos to the system. The traffic to the application increased exponentially. So did the cost of cloud infrastructure – to around $30,000 a month! We looked into that and realized that the architecture consists of 25 servers running over 1,000 WordPress websites!

You can certainly reduce costs with the cloud just like you can reduce cloud costs. But this is true only as long as you really take care of the architectural efficiency of your system.

OK – no more waiting. Let’s go right into the story!

Background

About the client

It’s a United Kingdom-based business that works with students and universities to turn graduation ceremonies into one-of-a-kind experiences for all alumni.

The outbreak of COVID caused the company to speed up a project of moving some of its business to the internet.

Timely and quality development? Call The Software House!

The Software House helped make the conversion possible, delivering a new app at a record pace, ahead of a new graduation ceremony season. The app:

• Used React for the frontend and Node.js for the backend,
• Opened the door for many new clients to come in, including +800 universities and thousands of students,
• Featured stress tests from our QA specialists to make sure it withstands all the new traffic.

Both the client and our team were satisfied with the speed of development and the quality of deliverables.

The client reached high infrastructure costs that weren’t caused by their cloud services or providers but by the inefficient use of cloud storage.

It turned out that the increase in traffic affected parts of the system that were not created by us.

These parts included the previously mentioned multiple servers that powered over a thousand WordPress websites, each of which required a separate database.

The increased traffic directed to these sites caused the monthly cloud bill to skyrocket.

If nothing was done about it, the cloud storage costs would just keep on growing.

Cloud optimization challenges

First, we needed to understand:

• Why our client needed all this,
• How it is used by users,
• And how it is connected to the rest of the system. 

Business challenges

The entire system itself was quite huge at this point. It consisted of 4 different frontend apps and one monolithic core.

Why did it end up being so unscalable? 

The WordPress sites were introduced as a simple way of generating a new page for each new incoming client.

The system worked reasonably well when it only served a small group of clients.

But the customer base started growing rapidly at the beginning of the COVID-19 pandemic when the product became fully virtual.

As evidenced by the chart below, from May 1st, 2020 till March 28th, 2021, the app was visited by 1,300,000 unique users, who generated 3,100,000 views.

Technical challenges

Before we started implementing any changes, the simplified structure of the system looked like that:

There were a lot of services (the WordPress instances) and some additional panels running on the old application. That’s what we had to work with.

The plan was to transform it into something like this:

All WordPress instances should be replaced with one instance of a Next.js application. We couldn’t do it all at once.

We needed a transitional step, which was to involve something like this:

Why was this transitional step necessary?

The transition – coexistence of two systems

One of the challenges we faced was running two systems at once.

As is often the case when you take over an existing project, we couldn’t just turn off the old one and start using the new one.

The migration process involved business activities so we had to maintain both systems.

Next.js customization

Our friends at TSH have vouched for the power and versatility of Next.js multiple times before.

We agree with these assessments, but since we wanted to use Next.js in a special way, we still run into trouble.

We needed a Next.js mechanism for rendering a page with the same URL’s path structure (which means that it is exactly the same page for the application) but fed with different data, depending on the domain.

And that’s quite an ambitious goal because Next.js was not designed to be used like that.

A typical Next.js app is a Single Page Application, so it is visible under just one domain rather than under multiple completely different ones.

Next.js app implementation

As you can see, in order to succeed, we needed to modify the standard behavior of Next.js quite a bit. First, we tackled the transitional phase.

Incorporating a new system into the existing one

As mentioned earlier, we built a new system besides the existing one.

The new features we introduced did not match the original features one-to-one. We strived to map the models and relations of the original system to the new one. We used this mapping as a basis for developing new features.

This approach allowed us to continue working with the existing system quite smoothly.

Building a Next.js application

The new application was rather simple. It had a home page and a few additional pages.

From the developer’s perspective, the Next.js app was a SPA. However, in reality, it was just the frontend component of a sophisticated mechanism hidden underneath.

The frontend communicated with the core of the old system through rendered views.

Here is a general overview of the new home page:

We used getServerSideProps to prepare data for a page meant to be pre-rendered by the Next.js engine. The function works like this:

Houston! We have a problem!

So far so good. We created a SPA and an API to go with it. We connected them and they work well together.

Thanks to Server-Side Rendering, the performance is more than satisfactory. But the job is far from over.

Why? In order to understand this, we need to take a step back and analyze the way Next.js works from the perspective of the goals of our project.

SSR and caching in Next.js

To put it simply, Next.js-powered Server-Side Rendering takes some data from a URL path as well as some query string parameters and then applies it all to the React component.

The result of that action is an HTML page stored in the cache.

The next time a user visits the same page, the server will not need to render it for them again. Instead, they will be served the cached version. 

Everything seems fine so far.

But here’s the catch – using Next.js you can find the URL path quite easily but it is not going to include the domain!

It means that from the Next.js app’s point of view, there is no difference between calling foo.example.com and bar.example.com.

As far as the app is concerned, it is the exact same home page!

One instance, many domains, one solution

Our idea was to make our app visible under the *.example.com address and have it handle all of the subdomains.

To make it happen, we needed to find a way to somehow force the Next.js framework to take into consideration the full address of the domain attached to the application.

In our pursuit of a way to overwrite the normal behavior of Next.js, we decided to dive deeper under its hood.

When we did that, we realized Next.js was using Express.js internally for all kinds of processes.

That was it! Express.js definitely had a method to access the domain name!

Still, we wanted to implement the change in a way that does not interfere with the framework or app too much. That’s because the more you interfere, the more likely you are to cause unexpected behaviors within the application.

We started our implementation from a very simple Express.js app (the code below is even further simplified to focus purely on the main issue at hand).

As you can see above, we encapsulated the Next.js application inside the server and used it in expressHandler. Here is a simplified version of the handler:

The application needs to check if a given page should be cached.

If it should, we call renderAndCache, which in turn calls nextJsApp.renderToHTML under the hood calls if necessary. If it should not, we simply call nextJsHandler.

In brief, here’s what happened during our cloud optimization process:

1. We replaced multiple WordPress instances with one Next.js application.

2. The new app handled all the traffic previously received by the WordPress pages.

3. We wrapped the Next.js app with customized Express.js code to pass domain names as parameters.

4. In doing so, we lost the out-of-the-box Next.js caching mechanism, so we built a new one.

Deliverables – costs down to 2k a month & scalability

Are you looking for a simple list of deliverables we produced during the project? Don’t worry, there will be one!

As far as cost optimization is concerned, our project exceeded all expectations.

The goal was to simply reduce costs – there was no fixed number we were supposed to achieve.

And yet, we reduced the cost 15-fold, down to a mere $2,000 a month.

In the field of scalability, we delivered plenty as well.

Not only is it much easier now to build new features due to the unification of all the content under a single application, but the deployment of the features to the clients is going to be way more straightforward. 

Oh, and the list of deliverables? Here it is:

• A brand new application powered by Next.js to replace over 1,000 WordPress sites.
• A redesign of the app for all clients – since the new app replaced all the numerous WordPress services simultaneously.
• A whole new process for the implementation of new features, based on Next.js and Express.js.
• A new client panel for all of the customers.

What does it all mean for business?

No more punishment for business growth

We really love projects in which it is easy to demonstrate how the developer’s blood, sweat, and tears translated into tangible, countable benefits for the client.

The main benefits are divided into two areas:

• The decrease in the time required to create new clients sites.Before, it required a manual setup of a new page complete with a database. In the new Next.js-powered up, all it takes is to fill out a single form. It is a big thing in terms of efficiency and usability for both clients and admins.
• The reduction of cloud infrastructure costs.Before, acquiring each new client was causing the cloud bill to become bigger and bigger. As a result of replacing the inefficient WordPress sites with a Next.js app, the number of servers droppedfrom 25 to just 1 Azure App Service.

The client no longer worries that each new customer and feature causes the whole application to become even more costly and unmanageable.

They can focus on growing their business instead. The software they own has suddenly become their ally in that pursuit.

And that’s how it should be.

How to reduce cloud costs – conclusions

So what do you think about our project?

Does it feel like it is something that your cloud environment needs or might need in the future? Do you need a cloud cost optimization strategy too?

Given the current trends in the cloud usage and in the offerings of cloud providers, chances are that the answer is: “yes”.

One way or the other, you need to know your options and what you’re capable of in terms of optimizing cloud costs.

That’s why we want to reassure you that:

Yes, you can:

• greatly decrease your current cloud infrastructure bill and save money through skilled cloud cost management and a variety of tactics such as reserved instances, regardless of what cloud provider you use,
• replace outdated and efficient architecture in cloud environments with a new one and keep the application running at all times,
• replace multiple inefficient servers with just one,
• simplify the management of your architecture,
• future-proof your application so that further development of new features and apps does not significantly increase your cloud bills.

Of course, ideally, you would want to do all that before your application is released. But there’s always a chance and place to optimize cloud costs.