Case Study: Scaling an E-Commerce Backend to 10,000 Requests/Second
When a rapidly growing e-commerce client approached me, their infrastructure was crumbling. Their Ruby on Rails monolith, running on a single Heroku dyno with a standard Postgres database, was crashing during flash sales. At 500 requests per second (RPS), the database locked up, and customers saw 502 Bad Gateway errors.
Here is how I re-architected their system to handle 10,000 RPS using FastAPI, AWS, and advanced database replication.
1. De-coupling with FastAPI
The first step was identifying the bottleneck: the product catalog and checkout flows. We extracted the product catalog into a high-performance FastAPI microservice.
Because FastAPI natively supports asynchronous I/O (async def), a single Uvicorn worker could handle thousands of concurrent requests waiting for database responses, whereas the legacy Rails server was tying up blocking threads.
2. PostgreSQL Read Replicas and PgBouncer
The database was hitting 100% CPU because 95% of the traffic was read-heavy (users browsing products).
We implemented PostgreSQL Read Replicas on AWS RDS. The FastAPI application was configured with two database URLs:
writer_db: For processing orders and inventory updates.reader_db: For fetching product details.
To prevent connection exhaustion, we placed PgBouncer in front of the databases to pool connections efficiently.
3. Asynchronous Order Processing with Celery
The checkout process previously blocked the HTTP response while charging the credit card, updating inventory, and sending a confirmation email.
We integrated Celery and RabbitMQ. Now, when a user clicks "Checkout", the FastAPI endpoint simply validates the request, drops an order_processing task into RabbitMQ, and returns a 202 Accepted response instantly. Background Celery workers consume the queue at their own pace, virtually eliminating checkout timeouts.
Results
After a 6-week migration period, the new architecture was put to the test during a holiday flash sale. The system peaked at 11,200 RPS with an average response time of 45ms and zero downtime.
Need help scaling your backend? Let's talk.