From bottlenecks to seamless communication. Mastering message handling and optimizing resource utilization with real-time observability

real-time-observability
Przemysław Gacka

Implementing real-time observability tools allowed our partner to streamline operations, prompt notifications, and adapt infrastructure. The result? Enhanced performance, optimized resource utilization, cost efficiency, and agility.

Business challenge

The application’s success depended on prompt user notification delivery, especially of new offers via email. However, performance analysis identified a bottleneck: the consumers tasked with managing events from RabbitMQ.

Like many businesses, our Saudi Arabian client faced a common dilemma: efficiently handling a deluge of messages while ensuring timely delivery to customers.

With notifications piling up in RabbitMQ queues, the pressure was on to streamline operations and keep the flow of communication moving.

But that was just the tip of the iceberg. Our client’s infrastructure, powered by Kubernetes, needed to adapt on the fly to fluctuating demands. The client sought application and infrastructure observability in response to the following:

  • increased website traffic (incl. sudden surges),
  • need to generate real-time reports or other business events at specific times,
  • unpredictable flow of daily operations.

Plus, the client needed a quick heads-up about any anomalies in their system, so we made sure those notifications were sent ASAP to the relevant teams.

The observability process

A data-gathering RabbitMQ-Exporter

To tackle these challenges head-on, we chose RabbitMQ-Exporter software. This nifty tool allowed us to gather real-time data on message volume, feeding it into Prometheus for analysis with lightning speed. 

Armed with this wealth of information, our teams were able to proactively manage workload distribution and ensure that critical notifications reached their destinations.

Kubernetes and Prometheus

Kubernetes and Prometheus formed a dynamic ecosystem where resources could scale up or down quickly. When additional Kubernetes cluster resources were necessary, the system seamlessly included them. No more graveyard shifts for 9/10ths. Now, they operate on autonomous 24/7 system management.

Turning Insights into Results

With RabbitMQ-Exporter seamlessly integrated into the active RabbitMQ cluster, we didn’t just accumulate data – we turned it into actionable insights. By relaying important information to development teams via email and Slack, we kept everyone in the loop and ready to act immediately.

Scaling Smarter, Not Harder

Leveraging Prometheus data, Kubernetes autonomously generated custom metrics that informed the application’s scaling behavior. 

When message volume surged, our system seamlessly included additional Kubernetes cluster resources, optimizing cost efficiency with spot instances. As messages slowed, Kubernetes axed extra instances, optimizing resource use.

Check out another successful observability project:

Project outcome

The impact of our solution was transformative. Gone are the days of frantic firefighting and manual interventions.

  1. With improved message handling and autoscaling capabilities, our client’s system operates smoothly, effortlessly handling a surge in concurrent events without disruption.
  2. The implemented solution seamlessly manages a significantly increased volume of concurrent events without requiring changes to the application code or manual intervention
  3. Scalability operations run autonomously 24/7/365. Designated personnel receive timely notifications in cases of anomalies or issues, enabling swift remedial actions through dedicated Grafana dashboards.
  4. Anomalies are detected and addressed in real-time through timely notifications via email and Slack. 
  5. Custom metrics guide decision-making and resource utilization for cost efficiency.

In the fast-paced landscape of modern business, agility isn’t just an advantage – it’s a necessity. Yet, many organizations find themselves shackled by the constraints of outdated systems, struggling to keep pace with the demands of today’s dynamic marketplace.

The Sensible Observability Score

Our custom test estimates your ability to measure and analyze your system’s data. In a few minutes, you’ll get the first measurable review of your company’s data capabilities. 

For you?
Acceleration Sprints™

You CAN have time for refinement. Run a 1-2 week sprint to improve product metrics soon

How?

The Software House is promoting EU projects and driving innovation with the support of EU funds

What would you like to do?

    Your personal data will be processed in order to handle your question, and their administrator will be The Software House sp. z o.o. with its registered office in Gliwice. Other information regarding the processing of personal data, including information on your rights, can be found in our Privacy Policy.

    This site is protected by reCAPTCHA and the Google
    Privacy Policy and Terms of Service apply.

    We regard the TSH team as co-founders in our business. The entire team from The Software House has invested an incredible amount of time to truly understand our business, our users and their needs.

    Eyass Shakrah

    Co-Founder of Pet Media Group

    Thanks

    Thank you for your inquiry!

    We'll be back to you shortly to discuss your needs in more detail.