Understanding Neki Router in Sharded Databases

Operating a sharded database presents a fundamental challenge: efficiently directing queries to the correct shard or shards, and then aggregating the results. This process is typically handled in two distinct planning stages. The first stage, managed by the Neki router, determines which shards will process the query and how data from multiple shards will be unified. The second stage, handled by the database system itself (like Postgres), creates the execution plan for each selected shard, dictating how that specific shard performs its work.

The Neki router's primary role is to interpret the database's data topology. This topology describes the arrangement and distribution of data across the various shards. By understanding this layout, the Neki router can make informed decisions about query distribution. It identifies the relevant shards for a given query and orchestrates the retrieval and merging of data from these distributed sources. This initial planning is critical for performance, as it minimizes unnecessary data movement and ensures that queries are processed by the most appropriate subsets of the database.

The Problem of Connection Bottlenecks

Traditional relational databases, particularly those based on Postgres, often employ a process-per-connection architecture. This means that for every active client connection to the database, a dedicated backend process is spawned on the database instance. While straightforward, this model introduces significant scaling limitations. Each backend process consumes system resources, primarily memory. As the number of concurrent connections grows into the thousands, the sheer volume of these processes can overwhelm the database server.

This resource consumption leads to two major issues. Firstly, the memory overhead becomes substantial, potentially limiting the amount of memory available for caching data, which is crucial for query performance. Secondly, a high number of processes increases the operating system's scheduling overhead. The system must constantly manage and switch between these numerous processes, leading to increased latency and reduced throughput. For applications that require a massive number of concurrent users or services to interact with the database, this process-per-connection model quickly becomes a critical bottleneck, hindering scalability and responsiveness.

Neki Router as a Solution to Connection Scaling

The Neki router addresses the connection bottleneck by acting as a layer that decouples client connections from database processes. Instead of each client establishing a direct, resource-intensive connection to a Postgres backend process, clients connect to the Neki router. The router then manages a pool of connections to the actual database instances. This approach significantly reduces the number of backend processes required on the database servers.

Think of it like a busy restaurant maître d'. Instead of every diner directly bothering the chefs in the kitchen, they speak to the maître d' at the front. The maître d' knows how many tables are available, who is waiting, and can efficiently direct diners to tables and manage the flow. Similarly, the Neki router acts as the intermediary. It accepts incoming client requests and intelligently forwards them to the appropriate database shards using a limited number of persistent connections. This drastically reduces the load on the database instances, allowing them to handle many more logical clients than they could with a direct connection model. The result is a database system that can scale to support a far greater number of concurrent users and applications without succumbing to connection-related resource exhaustion.

The Two-Stage Planning Process

The Neki router's involvement in query execution is best understood within the context of the complete query plan. When a query arrives, the Neki router constructs the first plan. This plan is essentially a routing directive. It uses the data topology to determine:

  • Which specific shards contain the data relevant to the query.
  • How to combine or aggregate results if the query spans multiple shards.
  • The most efficient path to send the query components to the identified shards.

Once the Neki router has formulated this first plan and dispatched the appropriate query segments to the necessary shards, the database system (e.g., Postgres) takes over to build the second plan. This second plan is what database administrators and developers are more familiar with. It details the execution strategy within each shard, such as the specific indexes to use, the join orders, and the filtering mechanisms employed by Postgres itself for that shard's data subset. This separation of concerns—routing and aggregation by Neki, and shard-local execution by Postgres—allows for more granular control and optimization at each stage of the query lifecycle.

What This Means for Sharded Databases

The Neki router represents a critical architectural component for modern, horizontally scaled databases. By abstracting the complexity of data distribution and connection management, it enables databases to achieve higher levels of concurrency and throughput. For developers and operators, this means the ability to build and scale applications that demand robust, high-availability data stores without being fundamentally limited by the underlying database's connection handling capacity. The two-stage planning process, with Neki handling the initial routing and aggregation strategy, is key to unlocking this scalability. It allows the database to focus on efficient data processing within shards while Neki manages the complex task of coordinating work across the distributed system.