Kokoro-82M TTS is 9x slower than Piper TTS, Obole reports

Performance Discrepancy in Text-to-Speech Models

In a direct comparison of text-to-speech (TTS) technologies, the Kokoro-82M model has been found to compute slower than the speech it generates. Obole, an AI operating on a small server, conducted tests using its own operational tools to measure performance. The findings indicate that Kokoro-82M operates at x0.91 to x0.93 of real-time speed, meaning it is slower than the rate at which it produces audio. This benchmark was measured on a specific server and with a particular text sample.

To contextualize this performance, Obole then ran the same text through Piper TTS, a popular open-source TTS system. Using the fr_FR-siwis-medium voice and the identical methodology on the same machine, Piper TTS achieved speeds ranging from x8.11 to x8.47 of real-time speed. Across twelve runs, Piper TTS showed a median speed of x8.32. This translates to Piper TTS being approximately 8.7 to 9.3 times faster than Kokoro-82M on this particular hardware and text input.

These measurements, first published in French and then translated to English, are presented as raw numbers by Obole, emphasizing transparency in AI tool performance. The AI explicitly states it is not claiming Piper TTS is universally superior, but rather highlighting a significant performance difference observed under specific conditions. The implications of such disparities are critical for developers and users relying on TTS for real-time applications, voice assistants, or content generation, where speed and efficiency directly impact user experience and operational costs.

Understanding the Benchmarking Methodology

The methodology employed by Obole is designed for practical, real-world application performance measurement. The AI runs on its own server, and the benchmarks reflect the actual tools it uses to function. This approach provides a grounded perspective, distinct from theoretical benchmarks that may not translate to typical deployment scenarios. The tests involve processing a standard piece of text and measuring how the TTS model's computation time compares to the duration of the generated audio output. A score above x1.00 indicates the model is faster than real-time, while a score below x1.00 signifies it is slower.

For Kokoro-82M, the results of x0.91 to x0.93 are concerning. This implies that for every second of audio produced, the model requires slightly more than one second of computation time. In a live application, this could lead to noticeable delays, stuttering, or an inability to keep up with demand, especially if multiple speech instances are required concurrently. The performance is so slow that it essentially lags behind its own output.

Piper TTS, on the other hand, achieved significantly higher scores, with a median of x8.32. This means that for every second of audio generated, Piper TTS completes its computation in approximately 0.12 seconds (1 / 8.32). This substantial difference suggests a more optimized architecture or implementation within Piper TTS, enabling it to process text and synthesize speech at a much greater speed. The use of a specific voice model (fr_FR-siwis-medium) indicates that performance can vary based on the chosen voice and its complexity, but the magnitude of the difference observed remains notable.

Broader Implications for TTS Development and Deployment

The disparity in performance between Kokoro-82M and Piper TTS raises important questions about the efficiency and optimization of different TTS models. While feature sets, voice quality, and ethical considerations are paramount, raw computational speed is a critical factor for many applications. For developers building voice-enabled interfaces, real-time translation tools, or interactive AI agents, a faster TTS system translates directly to a more responsive and fluid user experience.

The findings from Obole’s tests suggest that developers should carefully benchmark TTS solutions against their specific use cases and hardware constraints. Relying solely on reported benchmarks or theoretical capabilities might overlook significant real-world performance bottlenecks. The fact that Piper TTS, an open-source project, can achieve such high speeds compared to a model like Kokoro-82M highlights the potential for efficient, performant TTS solutions that are accessible to a wide range of developers.

Furthermore, the comparison underscores the ongoing advancements in AI model optimization. Techniques such as model quantization, efficient inference engines, and optimized C++ implementations (as often seen with Piper TTS) can yield dramatic improvements in speed and reduce computational overhead. This is particularly relevant in resource-constrained environments, such as edge devices or small servers, where every watt of power and every CPU cycle counts.

The AI’s commitment to sharing raw, unvarnished data is a valuable contribution to the AI community. It provides a concrete data point that challenges assumptions and encourages a deeper look at the practical performance of AI tools. As the demand for natural-sounding, real-time speech synthesis continues to grow across various industries, understanding these performance metrics will be key to selecting and deploying the most effective TTS solutions.

What remains to be explored is how these performance differences scale with different languages, text complexities, and hardware architectures. While Obole's test provides a clear snapshot for a specific French text on a particular server, a comprehensive understanding would require broader testing across diverse parameters. This will help paint a complete picture of the strengths and weaknesses of each TTS model in varied operational contexts.