Understanding the Core Difference: Current vs. Historical Performance

When evaluating AI agents within a Laravel application, the landscape has evolved. Two packages, Vizra Evals and Pest's Evals Plugin, now offer capabilities that might seem overlapping at first glance. The critical question for developers is not whether they need both, but rather which one best suits their immediate testing requirements. The answer hinges on a single, fundamental query: Are you concerned with your agent's performance right now, or do you need to track its performance relative to a previous state, like last month?

This article aims to provide a clear, unbiased comparison. As the author of one of these packages, I want to ensure you have the information to make an informed decision. We will explore what they share, their individual strengths, and crucially, when one is sufficient on its own.

Shared Foundations: Built on Pest

A key point of convergence is their foundational architecture. Vizra Evals is not an independent entity; it is built directly upon Pest. This means your evaluation tests are, in essence, Pest tests. They integrate seamlessly into your existing test suite and are executed via the standard Pest binary. This tight integration offers a familiar workflow for developers already using Pest for their PHP testing needs.

Both packages also share a common activation mechanism. To run your evaluation tests, you append the --evals flag to the Pest command. If this flag is omitted, the evaluation tests are skipped, allowing for a clean separation between standard unit/feature tests and your AI agent evaluations. The command structure is straightforward:

./vendor/bin/pest             # evals skipped
./vendor/bin/pest --evals    # evals run

Vizra Evals: Your Current State Snapshot

Vizra Evals is designed for the immediate assessment of your AI agent's capabilities. Its primary function is to provide a clear, up-to-the-minute report on how well your agent is performing against a defined set of criteria. Think of it less like a historical performance tracker and more like a high-fidelity snapshot of your agent's current state. It answers the question: "Is my agent meeting the desired standards today?"

This package excels in scenarios where you need to:

  • Validate recent changes: Did the latest code commit break the agent's core functionality?
  • Perform ad-hoc testing: Quickly check if the agent is behaving as expected before deploying.
  • Establish a baseline: Document the agent's performance at a specific point in time for future reference.

Vizra Evals focuses on the output and behavior of the agent in the present moment. It doesn't inherently store or compare results against past runs by default, making it efficient for current-state validation.

Pest's Evals Plugin: Historical Performance Tracking

Conversely, Pest's Evals Plugin is built with regression testing and historical performance analysis in mind. Its core strength lies in its ability to compare the current performance of your AI agent against a previously recorded baseline. It answers the question: "Is my agent performing worse than it did last week, last month, or at our last stable release?"

This is invaluable for maintaining the integrity of AI agents over time, especially in dynamic environments where subtle changes can lead to performance degradation. The plugin facilitates:

  • Regression Detection: Identify if new code or data shifts have inadvertently made the agent less effective.
  • Performance Trend Analysis: Track improvements or declines in agent performance across multiple runs and over extended periods.
  • Auditing and Compliance: Maintain a verifiable record of agent performance against established benchmarks.

The plugin achieves this by storing baseline results and providing tools to compare new evaluation runs against these stored benchmarks. This historical perspective is crucial for long-term AI agent management and ensures that performance does not degrade silently.

When to Use Which (and When You Might Not Need Both)

The decision boils down to your immediate objective:

Use Vizra Evals if:

  • You need to know if your AI agent is working correctly right now.
  • You are performing quick checks after code changes.
  • You are building out your initial set of evaluation tests and want a straightforward way to execute them.
  • You do not need to compare current results against historical data on a routine basis.

Use Pest's Evals Plugin if:

  • You need to detect if your AI agent's performance has degraded compared to a previous state.
  • You are concerned about regressions introduced by new code, data, or model updates.
  • You require a system to track and visualize performance trends over time.
  • Your workflow demands auditing of AI agent performance against established baselines.

Do you need both?

For many teams, the answer is likely no. If your primary concern is validating the current state of your AI agent, Vizra Evals provides a streamlined, efficient solution. It integrates directly into your Pest workflow without the overhead of historical data management. If, however, your AI agents are critical components of your application and subject to continuous updates and potential degradation, the regression testing and historical analysis capabilities of Pest's Evals Plugin become essential. In such cases, you might use Vizra Evals for initial development and ad-hoc checks, and then rely on Pest's Evals Plugin for ongoing, automated regression detection as part of your CI/CD pipeline.

The choice is not about which package is 'better,' but which tool is the right fit for your specific stage of development and ongoing maintenance strategy for AI agents in Laravel.