I Tried to Beat Peter Norvig and Accidentally Became Ryan Gosling: Scaling a Meme to 10K RPS on 8GB RAM

The internet moves fast. One moment you are a nobody with a cheese bread recipe, the next, Ryan Gosling’s Twitter fingers have turned your side project into a distributed systems stress test. This is how we survived 10,000 requests per second on 8GB RAM with bounded queues, race condition free SQLite, and a healthy fear of thread explosion.

The Gosling Effect: When Your Side Project Goes Supernova

The initial setup was simple: static files on Netlify, a Flask endpoint on Heroku for analytics. Traffic was a trickle. Then Gosling tweeted. The tweet linked to a simple Flask analytics endpoint that logged requests. The expectation was minimal traffic, perhaps a few hundred requests. The reality, following a viral tweet by Ryan Gosling, was an onslaught of 50,000 concurrent users hitting the /track endpoint within minutes. This sudden, massive surge transformed a trivial side project into an immediate, high-stakes distributed systems challenge.

Diagram illustrating the initial simple architecture of Netlify static files and a Heroku Flask endpoint.

Failure Walkthrough: The Threaded Flask Bottleneck

The initial architecture relied on Flask’s default Threaded mode. This mode spawns a new thread for each incoming request. Under normal, low-traffic conditions, this is a perfectly acceptable and common pattern. However, when 50,000 concurrent users simultaneously hit the /track endpoint, the Heroku dyno, which had a mere 512MB of RAM, was overwhelmed. Each new request consumed precious memory and CPU cycles for thread creation and management. The system rapidly exhausted its available memory, leading to cascading failures and an inability to process any requests. This classic threading bottleneck, often hidden under low load, was brutally exposed by the unexpected viral spike. The system wasn't just slow; it was actively crashing under the load.

From Threaded Flask to Bounded Queues: Architectural Overhaul

The immediate priority was to stop the bleeding. Simply throwing more resources at the problem wasn't feasible with the existing architecture due to the threading model. The core issue was that each request tried to consume independent resources without restraint. The solution involved re-architecting the request handling to prevent resource exhaustion. This meant moving away from a naive thread-per-request model to a more controlled, asynchronous approach using bounded queues.

The team adopted a pattern where incoming requests were placed into a bounded queue. This queue acts as a buffer, limiting the number of requests that can be processed concurrently. Instead of creating a new thread for every request, a fixed pool of worker threads would pull requests from the queue and process them. This approach has several critical advantages:

  • Resource Control: The bounded queue prevents the system from being flooded with requests, ensuring that the number of active processing threads remains manageable and within the system’s memory and CPU limits.
  • Decoupling: The queue decouples the request ingestion rate from the processing rate. Even if traffic spikes, new requests are queued rather than immediately crashing the server.
  • Graceful Degradation: When the queue is full, new requests can be rejected or handled with a specific error response (e.g., HTTP 429 Too Many Requests), signaling to the client that the service is temporarily overloaded. This is far preferable to a complete system crash.

Race Condition-Free SQLite: Leveraging WAL Mode

The analytics endpoint needed to write data. A common challenge with SQLite in concurrent environments is managing write operations. Traditional SQLite locking mechanisms can become a significant bottleneck under heavy load, as only one writer can access the database at a time. To overcome this, the team switched SQLite to Write-Ahead Logging (WAL) mode.

WAL mode allows readers and writers to operate concurrently. While there is still a single writer process, it doesn't block readers. Readers access a consistent snapshot of the database from a previous checkpoint, while the writer appends changes to a separate journal file. Periodically, these changes are “checkpointed” into the main database file. This significantly improves concurrency for read-heavy workloads, which is typical for analytics logging, and avoids the lock contention that would cripple a high-RPS system. The surprising detail here is not the performance gain itself, but how readily SQLite, a file-based database, can be tuned for such high throughput under the right configurations.

Achieving 10K RPS on 8GB RAM

The combination of a robust, asynchronous request handling system with bounded queues and a concurrency-friendly SQLite configuration (WAL mode) allowed the system to scale dramatically. The target of 10,000 requests per second (RPS) was achieved on a surprisingly modest 8GB RAM server. This was made possible by:

  • Efficient Thread Management: Limiting the number of active threads to a manageable pool, preventing memory exhaustion.
  • Asynchronous Processing: Non-blocking I/O operations and efficient queue management meant that the server could accept new connections while processing existing ones without getting bogged down.
  • Optimized Database Writes: WAL mode for SQLite ensured that database writes did not become a system-wide bottleneck.

The system effectively handled the surge, transforming from a potential disaster into a testament to thoughtful architectural choices under pressure. The ability to scale to such high RPS on limited hardware highlights the power of understanding and correctly applying fundamental distributed systems principles, even for seemingly trivial applications.

Lessons Learned: Beyond the Meme

This experience, while triggered by a viral meme, offers critical lessons for any developer or team operating in a high-traffic environment. The immediate takeaway is the danger of underestimating traffic spikes and the importance of building resilient systems from the outset. Relying on default configurations for web frameworks or databases can lead to catastrophic failures when load increases unexpectedly. The project team learned that even simple applications can become distributed systems nightmares. The key was understanding the underlying bottlenecks—thread exhaustion and database contention—and applying proven solutions like bounded queues and WAL mode. It’s a stark reminder that the principles of scalability, concurrency, and resource management are paramount, regardless of the application's perceived complexity or origin.

What nobody has addressed yet is what happens to the thousands of developers who built on the old API, assuming it would remain stable. This incident, while a success for the meme project, could have been a disaster for any dependent services. The rapid, unexpected scaling of a single endpoint often implies a lack of backward compatibility planning for downstream consumers.