The 40-Second Answer
A simple question from a founder – "Where are our holders located? Korea or the US?" – can trigger a 40-second delay. This isn't due to complex calculations, but rather a cascade of approximately 110 API calls, each taking about two seconds to complete. This latency issue was the catalyst for building Holder Atlas, a tool designed to visualize this data as a world map. Holder Atlas queries Nansen's exchange entity labels and a custom exchange-to-country mapping to pinpoint holder locations, displaying them country by country on a map.
The project's focus shifted from pure visualization to performance optimization when the sheer number of calls became apparent. The initial architecture, while functional, was a bottleneck. The goal was to reduce the time it took to aggregate data from these disparate sources, turning a slow, multi-step process into something near real-time.
The Concurrency Experiment
The author decided to tackle the latency by increasing the concurrency of the API calls. The hypothesis was straightforward: if more requests could be processed simultaneously, the overall time to gather the data would decrease. The initial setup involved making calls sequentially or in small batches. Doubling the concurrency was the next logical step to test the limits of the system and the underlying infrastructure.
The experiment involved modifying the code to allow for a higher degree of parallelism. Instead of waiting for one set of calls to complete before initiating another, the system was configured to manage a larger pool of active requests. This approach is common in high-throughput systems, aiming to maximize the utilization of network and processing resources.
The Results: A Mixed Bag
The outcome of doubling the concurrency was a 14% improvement in speed. While any performance gain is valuable, this figure was lower than anticipated, suggesting that the system was not purely I/O bound or that other bottlenecks were emerging. More concerning were the side effects: three API calls failed outright after the change. This indicates that simply increasing concurrency without addressing other potential issues can lead to instability and errors.
The failures point to several potential causes. Firstly, the downstream APIs might have rate limits that were exceeded by the increased number of concurrent requests. Services often impose limits to prevent abuse and ensure fair usage, and a sudden surge in traffic could trigger these protections. Secondly, the author's own server might have struggled to manage the increased load, leading to dropped connections or timeouts. Resource exhaustion – such as running out of available sockets or exceeding memory limits – is a common consequence of aggressive concurrency tuning.
Deconstructing the Bottlenecks
The 110 API calls originate from the need to aggregate data from various sources. The Holder Atlas application makes five top-level calls to Nansen endpoints. Two of these are relatively quick. However, the subsequent calls require further data enrichment. For instance, one endpoint might return a list of exchanges associated with a token, but to determine the country of those exchanges, additional lookups are necessary. This often involves querying a separate database or API that maps exchanges to countries. Each of these sub-queries adds to the total latency.
The problem is compounded by the fact that not all exchanges are neatly categorized. Some might be missing from databases, or their country of origin might be ambiguous. This necessitates fallback mechanisms or manual curation, further increasing the complexity and potential for error. The process of building the map is akin to assembling a jigsaw puzzle where each piece requires a separate trip to a different store, and some pieces might be out of stock or have unclear labels.
Lessons Learned and Next Steps
The experiment with Holder Atlas provided critical insights into API performance and the challenges of data aggregation. The 14% speedup, while modest, confirms that concurrency has an impact. However, the three broken calls serve as a stark warning: increasing throughput without careful consideration of downstream systems and resource limits can degrade reliability. This is a common pitfall in distributed systems, where the behavior of individual components can have cascading effects on the whole.
For developers working with similar data aggregation tasks, several strategies emerge. Firstly, meticulous monitoring of individual API call durations and error rates is crucial before and after changes. Understanding which specific calls are the slowest and most error-prone is key to targeted optimization. Secondly, implementing robust error handling and retry mechanisms for transient failures is essential. This might involve exponential backoff strategies to avoid overwhelming downstream services.
Furthermore, exploring caching strategies can significantly reduce the number of API calls needed. If holder data for a specific token doesn't change frequently, serving it from a cache can provide near-instantaneous responses. This requires careful management of cache invalidation to ensure data freshness.
The author's experience highlights that optimizing distributed systems is an iterative process. It's not just about raw speed, but about finding the right balance between performance, stability, and resource utilization. The next steps for Holder Atlas might involve a more nuanced approach to concurrency, perhaps using adaptive concurrency algorithms or implementing more sophisticated rate-limiting controls on outgoing requests. Understanding the exact failure modes of the broken calls—whether they were timeouts, connection errors, or explicit rejections—is paramount for devising effective solutions.
What remains unaddressed is the fundamental trade-off between data richness and real-time performance. For many applications, a 40-second delay for comprehensive data might be acceptable, especially if the alternative is a less accurate or incomplete picture. However, for interactive dashboards or time-sensitive analytics, such latency is untenable. The challenge lies in determining the acceptable latency threshold for different use cases and engineering systems that meet those demands without sacrificing reliability.