Schema Validation in API Integrations: What to Validate and When

Validation is the moment a message is compared with a schema. The schema can be careful and the comparison can still happen in the wrong place, or not happen at all. This note says what Draft 2020-12 checks, where an integration can run that check, and what a pass does not prove. It does not claim a percentage of production failures that validation would have caught. This desk does not have that number.

What the validator is allowed to reject

What the schema is asserting defines the keywords. The validation behavior that matters in an integration is narrower than a careful reading of the file.

required fails an object when a listed name is missing. It does not fail a number, a string, or null just because those values are the wrong shape. JSON Schema is constraint-based: a keyword that applies to objects does not by itself reject other types. If a message must be an object, the schema needs type.

additionalProperties: false rejects names that were not covered by properties or patternProperties. Without that keyword, extra names are allowed. A consumer that wants a closed object has to write the keyword. A producer that wants room to add response fields should not be surprised when a consumer copied an old schema that closed the object.

The format keyword is the sharp edge. Draft 2020-12 split format handling into an annotation vocabulary and an assertion vocabulary. The release notes for that draft record the split. By default, format names a semantic hint. It does not reject a value. OpenAPI 3.1 defines additional format names and says that, unless stated otherwise, Schema Object properties follow JSON Schema. It does not require every server to reject a string that fails a format. If a date must be rejected in production, use a constraint the implementation actually enforces, such as pattern, or name the dialect that turns format assertions on and test that dialect. Hoping that “the validator checks format” is how two libraries disagree about the same file.

Producer, consumer, and CI

These are three different comparisons. The table is an editorial map. It is not a result from a test suite.

The cells are review questions, not measured failure rates.

Where to validate

Validate on the producer when you publish a schema and can stop a bad release before a client sees it. That check belongs next to the build that emits the message, using the same schema artifact the document is generated from. A wiki copy of the schema will not do this job.

Validate on the consumer when a bad message would cause a worse outcome than a rejection. Payment instructions, inventory adjustments, and any write that is hard to undo belong in this group. The consumer should reject with a response the producer understands. For events, validation specific to inbound events includes the status code the sender will retry. A consumer that returns 200 and then drops a message it dislikes has told the sender the delivery worked.

Validate in CI when the schema or the consumer expectations change. Tests that compare a message with a consumer’s expectations are the stronger form of that check, because they start from what the consumer reads rather than from the entire schema. A CI job that only validates the producer’s own examples stays green when those examples no longer match the code.

Do not validate the same closed schema on both sides if your compatibility rule says responses may grow. The producer can validate the fields it promises to send. The consumer should validate the fields it reads, and ignore the rest, unless the contract says the object is closed. When a tighter schema becomes a breaking change is the policy choice. Validation will enforce whichever schema you hand it. It will not notice that you handed the two sides different policies.

Schema drift

Drift, in this note, means the messages on the wire and the schema you point people to are no longer the same constraint. The word is a label for that gap. It is not a metric, and this site does not estimate how often it happens.

Drift shows up in a few concrete ways:

• A field is required in the published schema and missing in traffic, or the reverse, and no version change was published.
• An enum gains a value in code and in logs, while the schema clients were given still lists the old set.
• format or pattern in the document is stricter than the parser the server runs.
• Two files both named as the schema differ, and neither points at the other. OpenAPI’s jsonSchemaDialect and a Schema Object’s $schema exist so a tool can tell which dialect to apply. If those values are absent, OpenAPI 3.1 says the OAS dialect schema is the default for Schema Objects in the document. A bare JSON Schema file does not inherit that default. Tooling that assumes the wrong draft will “fail” a document the other tool accepts.

When you find a gap, either change the sender, change the published schema, or version the contract. Silently updating the examples in a document so they match last week’s bugfix, without saying the contract moved, is how the next client trusts the wrong file.

A validation checklist

1. The schema artifact in CI is the artifact clients can fetch, or the difference is explained.
2. Objects that must be objects set type. Required names are a separate list.
3. format is either treated as a hint, or the asserting dialect is named and tested.
4. Unknown properties are rejected only where the compatibility rule says the object is closed.
5. Producer checks run before a release. Consumer checks run where a bad write is expensive.
6. A failure names the keyword and the path. A boolean “invalid” with no path is not an operable result.
7. Event receivers persist the event id before they depend on the body, so a validation failure after a side effect is not the first time the id is stored.

What validation is not

A valid message can still be the wrong effect. The schema does not know that quantity should have been reserved in a stock system, or that this event id was already applied. Those checks are behavior. Put them next to validation, and do not describe a green schema job as proof the integration works. The testing note draws that line in more detail.