Data validation is fundamental to robust software. Most discussions, however, stop at the point of failure. The common narrative is: define a schema, validate incoming data against it, and if it fails, raise a ValidationError. This approach, while seemingly complete, often offers a dishonest end to the job. It correctly identifies that a data payload is incorrect, but it fails to address the critical next step: who is responsible for fixing it, and how should the system respond?

The Limits of ValidationError

A ValidationError, by its nature, is a statement of fact. It signals that a piece of data does not conform to an expected structure or set of rules. Think of it as a red flag on a report. It tells you something is wrong, but it doesn't tell you what to do about it. The typical tutorial or blog post on data validation often assumes a simple client-server interaction. In this model, a client sends bad JSON, the server responds with a 400 Bad Request error, and the client is expected to correct the payload and resend. This is only one type of boundary, and crucially, it's the only one where the entity responsible for fixing the error is external and capable of reading and acting upon explicit error messages.

However, real-world systems operate with multiple internal and external boundaries, each with different actors and different capabilities for remediation. The simplistic client-server model breaks down when validation errors occur deep within a system, far from the original source of the data, or when the entity receiving the error is not a human developer but another service, a database, or an automated process.

Internal vs. External Boundaries: Who Fixes What?

Consider a science and technology digest that processes incoming submissions. This system might have three distinct boundaries within a single process, each requiring a different error-handling strategy:

  • Boundary 1: Ingestion API. This is the external boundary, similar to the client-server model. A user submits an article draft via a web form or API. If the JSON payload is malformed (e.g., missing required fields, incorrect data types), the system should respond with a 400 Bad Request. The error message should be detailed enough for the user (or their client application) to understand and correct the submission. Here, ValidationError is appropriate, and the client is the designated fixer.
  • Boundary 2: Internal Data Processing Pipeline. Once ingested, the article data might enter an internal pipeline for enrichment, analysis, or categorization. This pipeline might involve multiple microservices or modules. If, for instance, a service responsible for extracting keywords encounters data that doesn't match its expected format (e.g., a malformed date string that passed initial validation but is unusable for chronological sorting), raising a ValidationError here is problematic. The 'client' is another internal service or module. This internal component cannot 'read' an error message and correct itself in the same way an external user can. The responsibility for fixing this might fall on the data ingestion service to ensure cleaner data, or a dedicated data quality service might need to step in.
  • Boundary 3: Archival and Search Indexing. After processing, the article data is archived and indexed for search. If the data fails to index correctly due to an unexpected format (e.g., a field that should be a simple string is unexpectedly a complex object, preventing serialization into the search index), a ValidationError is again the wrong tool. The 'client' is the search indexing service. It cannot fix the data itself. The problem might stem from a bug in the processing pipeline, or a misunderstanding of the search index's requirements. The response must be different – perhaps logging the failure, alerting a data engineering team, or attempting a transformation before retrying.

The core question remains: Who is supposed to fix this, and can they? The answer dictates the policy, not the validation error itself.

Beyond Punctuation: Defining Remediation Policies

Raising a ValidationError is merely punctuation. It marks the end of valid data according to a schema. A policy, however, defines the system's behavior in response to that punctuation. This policy must consider the context of the validation failure:

  • Is the error from an external, human-readable source? If so, a 4xx client error response with a clear message is appropriate.
  • Is the error from an internal service or component? The response should not be a generic error that halts processing without a clear path to resolution. Instead, it might trigger a retry mechanism, quarantine the data for manual review, initiate a data repair process, or log a critical alert for a specific team.
  • Is the data unrecoverable or potentially corrupting? In such cases, the policy might dictate discarding the data, but this decision should be explicit and logged, not an implicit consequence of an unhandled exception.

This distinction is crucial for building resilient systems. Treating every validation failure as an external client error leads to brittle internal logic and missed opportunities for automated data correction or graceful degradation.

The Unanswered Question: Systemic Data Quality

What nobody has addressed yet is how to build systems that proactively manage data quality across multiple internal boundaries. If a ValidationError is raised deep within a system, it's often a symptom of a deeper issue – perhaps an upstream service failed to validate correctly, or a transformation process introduced an error. The current paradigm focuses on the 'what' (invalid data) but not sufficiently on the 'why' or the 'how to prevent recurrence' across interconnected services.

A truly robust system doesn't just validate; it orchestrates data quality. This involves defining clear ownership for data integrity at each stage, implementing appropriate error-handling strategies tailored to the context of the boundary, and establishing mechanisms for monitoring and rectifying data issues before they cascade. ValidationError is a tool, but it's only one small part of a comprehensive data quality strategy. The policy around its use, and the systems that react to it, are what truly define the system's robustness.