Can't I just write contract tests?

For APIs you control, contract tests are excellent and you should have them. For third-party APIs they test your assumptions on your schedule: they catch a change only when they run, only for the paths they cover, and they go stale exactly as fast as the provider moves. They complement traffic-based detection; they do not replace it.

What about watching the provider's changelog?

Do it, it is cheap. But changelogs document what the provider decides to announce, which excludes most of what breaks integrations in practice: undocumented fields, nullability changes, response restructures considered internal. Changelog watching is necessary and radically insufficient.

Why do drift monitors get muted?

False alarms. Live responses vary on every call, and a naive detector that diffs raw responses fires on variance: counters, timestamps, list contents. After the third false page at 2am, the channel gets muted and the one real alert dies in it. Distinguishing sustained structural displacement from value churn is the entire engineering problem.

What data does traffic-based monitoring need to see?

Only structure: field names, types, and shape. A well-designed collector discards values at the edge and never stores request or response bodies, headers, or tokens. If a vendor cannot demonstrate that property against their own database, treat the claim as marketing.

API drift monitoring: an honest guide

The approaches to catching third-party API contract changes, compared honestly: spec diffing, per-customer monitors, contract tests, status pages, and neutral cross-company observation. Where each one is right.

The problem being solved

Third-party APIs change without telling you. When the change is structural, a field's type, presence, or shape, integrations break while every availability signal stays green. API drift monitoring is the discipline of catching those contract changes early, with evidence, before they become production incidents. The definition and failure modes are covered in what is API drift; this page compares the ways to actually catch it.

The five approaches, honestly compared

1. Changelog and deprecation watching

Subscribe to provider changelogs, deprecation feeds, and mailing lists; some tools automate the watching and route matches to Slack. Right when: always, as a baseline, because it is nearly free. Fails when: the change is unannounced, which is precisely the class that hurts. Changelogs are the provider's account of the provider's behavior.

2. Spec-to-spec diffing

Diff OpenAPI documents between versions, in CI or on a schedule; oasdiff is the well-known open tool. Right when: the provider publishes a spec, keeps it current, and you want review-time warnings. Fails when: the spec and reality diverge, which is common enough to have its own name (documentation drift). A diff of two documents cannot see what the API actually returns.

3. Contract tests and synthetic probes

Assert expected response shapes in scheduled tests against live endpoints. Right when: a handful of critical paths deserve explicit, versioned assertions you control. Fails when: coverage is the question: probes test the requests you wrote, on your schedule, with your test account's data shapes, and every assertion is maintenance. Real traffic exercises orders of magnitude more surface than any probe suite.

4. Per-customer traffic profiling

Observe the API responses your own application actually receives, reduce them to structure, baseline the shape, and alert on sustained displacement. This is the strongest single-team approach: it watches the real contract, under real traffic, including the undocumented behavior your code quietly depends on. The two hard requirements: the engine must separate structural drift from value variance or it will cry wolf, and the collector must discard values at the edge or it becomes a data-custody liability larger than the problem it solves.

5. Cross-company observation

The structural limitation of everything above is that each team watches alone, and learns about a change only when its own traffic hits it. A neutral network watching the same providers across many companies can see the first tremor anywhere and warn everyone exposed, before most of them have encountered the change at all. Status pages cannot do this, single-tenant monitors cannot do this, and the provider will not. This is the layer ShiftGraph exists to build; its public seed is the Observatory, and its honest current limitation is youth: network effects compound with participants.

What a trustworthy alert must carry

The structural evidence: which field, what kind of change, old shape versus new shape.
Temporal honesty: first seen when, sustained since when, and the explicit distinction between a first-look profile and confirmed drift.
Blast radius: which of your operations and environments touch the changed surface.
Severity that means something: a removed required field and an added optional one are not the same page-at-2am event.

The practical build-versus-buy walkthrough, including the parts you can assemble yourself, is in how to detect breaking changes in third-party APIs.

Common questions

Can't I just write contract tests?: For APIs you control, contract tests are excellent and you should have them. For third-party APIs they test your assumptions on your schedule: they catch a change only when they run, only for the paths they cover, and they go stale exactly as fast as the provider moves. They complement traffic-based detection; they do not replace it.
What about watching the provider's changelog?: Do it, it is cheap. But changelogs document what the provider decides to announce, which excludes most of what breaks integrations in practice: undocumented fields, nullability changes, response restructures considered internal. Changelog watching is necessary and radically insufficient.
Why do drift monitors get muted?: False alarms. Live responses vary on every call, and a naive detector that diffs raw responses fires on variance: counters, timestamps, list contents. After the third false page at 2am, the channel gets muted and the one real alert dies in it. Distinguishing sustained structural displacement from value churn is the entire engineering problem.
What data does traffic-based monitoring need to see?: Only structure: field names, types, and shape. A well-designed collector discards values at the edge and never stores request or response bodies, headers, or tokens. If a vendor cannot demonstrate that property against their own database, treat the claim as marketing.