---
title: "We checked 13 public APIs against their own OpenAPI specs. 9 had hard mismatches."
description: "A read-only survey of public APIs that publish a spec: what their live responses actually return, classified strictly, with every request reproducible."
date: 2026-11-02
author: API Drifter
canonical: https://apidrifter.com/research/public-api-spec-drift-2026-11/
license: Apache-2.0 (text and data)
---

# We checked 13 public APIs against their own OpenAPI specs. 9 had hard mismatches.

By API Drifter, 2026-11-02. Canonical: https://apidrifter.com/research/public-api-spec-drift-2026-11/

Every API that publishes an OpenAPI document is making a promise: send this, get that back. Generated clients, typed SDKs, mocks and contract tests all depend on it. We wanted to know how often the promise holds, so we pointed [API Drifter](/) at thirteen public APIs that publish their own spec, sent one read-only request per operation, and compared what came back to what the document said.

The short answer: nine of the thirteen returned at least one successful response that a client generated from their own spec would choke on.

## What we counted, and what we did not

Numbers like this are easy to inflate, so the strict rules first.

A finding only counts as **hard** when a typed client generated from the spec would break on it: a field with the wrong type, a required field missing, a field present where the schema forbids unknown fields, a null where the schema does not allow null, or a success response in a media type the spec does not list. Nothing else is in the headline number.

Three categories are reported separately and deliberately kept out of it:

- **Undocumented fields.** OpenAPI allows extra properties unless a schema says `additionalProperties: false`. An extra field is a documentation gap, not a violation.
- **Undocumented error statuses.** We sent no credentials, so we received 401, 403 and 429 responses that the specs do not mention. Almost no spec documents those, and nobody treats that as drift.
- **Our own mistakes.** A handful of requests used values that violated the spec's constraints. Those are our fault and are labeled as such.

We also found one false positive in our own validator during review, in how OpenAPI 3.0's `nullable: true` combines with `oneOf`. We fixed it, added a test, and re-verified every API from the recorded responses before writing this.

## Results

The table below is generated from the survey data. Each row links to the full findings, where every finding carries the exact request we sent, the status we got, the location in the response body, and two suggested fixes: one if the spec is the source of truth, one if the implementation is.

| API | Spec | Ops in spec | GET sent | 2xx | Hard | Body | Media | Errors | Input | Reviewer note |
|-----|------|-------------|----------|-----|------|------|-------|--------|-------|---------------|
| [Swagger Petstore (reference demo)](https://apidrifter.com/research/public-api-spec-drift-2026-11/petstore/) | [OpenAPI 3](https://petstore3.swagger.io/api/v3/openapi.json) | 19 | 8 | 6 | 1 | 1 | 0 | 0 | 0 | Confirmed: GET /user/login returns a plain-text session string under application/json. |
| [APIs.guru directory API](https://apidrifter.com/research/public-api-spec-drift-2026-11/apis-guru/) | [OpenAPI 3](https://api.apis.guru/v2/openapi.yaml) | 7 | 6 | 6 | 13 | 13 | 0 | 0 | 0 | Confirmed: GET /specs/{provider}/{api}.json returns the OpenAPI document itself; the spec documents the API entry object (preferred, versions). |
| [PokeAPI](https://apidrifter.com/research/public-api-spec-drift-2026-11/pokeapi/) | [OpenAPI 3](https://raw.githubusercontent.com/PokeAPI/pokeapi/master/openapi.yml) | 102 | 30 | 30 | 1 | 1 | 0 | 0 | 0 | Confirmed: evolution-chain baby_trigger_item is null; the spec does not allow null. |
| [Wikimedia REST API (en.wikipedia.org)](https://apidrifter.com/research/public-api-spec-drift-2026-11/wikipedia-rest/) | [OpenAPI 3](https://en.wikipedia.org/api/rest_v1/?spec) | 49 | 19 | 13 | 8 | 20 | 0 | 9 | 0 | Confirmed: media-list items carry undocumented leadImage and srcset; /page/lint returns an array where the spec says object; 400 errors come back as application/json, documented as application/problem+json. |
| [Discourse (meta.discourse.org)](https://apidrifter.com/research/public-api-spec-drift-2026-11/discourse-meta/) | [OpenAPI 3](https://docs.discourse.org/openapi.json) | 95 | 21 | 13 | 106 | 126 | 0 | 8 | 0 | Confirmed: category custom_fields holds plugin keys the spec forbids; topic tags are objects where the spec says strings; permission is null. The spec is generated from test fixtures and lags the live site. |
| [Open Library](https://apidrifter.com/research/public-api-spec-drift-2026-11/openlibrary/) | [OpenAPI 3](https://openlibrary.org/openapi.json) | 58 | 20 | 9 | 1 | 0 | 1 | 9 | 4 | Confirmed: /health returns text/plain, documented as application/json; the spec's own example for search.json spellcheck_count violates its schema. Account endpoints 303-redirect when logged out. |
| [GitHub REST API (unauthenticated)](https://apidrifter.com/research/public-api-spec-drift-2026-11/github/) | [OpenAPI 3](https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json) | 1254 | 40 | 9 | 0 | 0 | 0 | 28 | 0 | First run: gist file language is null but not nullable; event payloads match two oneOf branches. The rerun hit the anonymous rate limit (403) after nine successes, so this row understates GitHub; rerun with a token pending. |
| [Gitea (gitea.com, self-served Swagger 2.0)](https://apidrifter.com/research/public-api-spec-drift-2026-11/gitea/) | [Swagger 2.0](https://gitea.com/swagger.v1.json) | 537 | 40 | 19 | 15 | 30 | 0 | 14 | 0 | Confirmed: activity feed comment and user objects are null; User carries an undocumented username field. Swagger 2.0 cannot express null, so the null findings are partly a format limitation. |
| [Forgejo (codeberg.org, self-served Swagger 2.0)](https://apidrifter.com/research/public-api-spec-drift-2026-11/codeberg/) | [Swagger 2.0](https://codeberg.org/swagger.v1.json) | 506 | 40 | 16 | 23 | 35 | 0 | 18 | 1 | Same drift as Gitea (Forgejo fork), plus repo.parent null on feeds. |
| [httpbin.org (self-served Swagger 2.0)](https://apidrifter.com/research/public-api-spec-drift-2026-11/httpbin/) | [Swagger 2.0](https://httpbin.org/spec.json) | 78 | 20 | 18 | 0 | 0 | 0 | 2 | 0 | Confirmed: GET /cookies/set and /cookies/delete respond 302, documented as 200 only. Earlier input mistakes fixed. |
| [Nager.Date public holidays](https://apidrifter.com/research/public-api-spec-drift-2026-11/nager-date/) | [OpenAPI 3](https://date.nager.at/openapi/community-v3.json) | 8 | 8 | 7 | 0 | 0 | 0 | 0 | 0 | One 502 on LongWeekend during the run; transient, not drift. |
| [Open-Meteo forecast API](https://apidrifter.com/research/public-api-spec-drift-2026-11/open-meteo/) | [OpenAPI 3](https://raw.githubusercontent.com/open-meteo/open-meteo/main/openapi/forecast.yml) | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | Minor: current_units.interval is undocumented. |
| [GitLab REST API (gitlab.com, unauthenticated)](https://apidrifter.com/research/public-api-spec-drift-2026-11/gitlab/) | [OpenAPI 3](https://gitlab.com/gitlab-org/gitlab/-/raw/master/doc/api/openapi/openapi_v3.yaml) | 1864 | 15 | 8 | 3 | 3 | 0 | 6 | 0 | Confirmed: GET /projects/{id}/issues and /repository/contributors return arrays where the generated spec says object; namespace.parent_id is null but not nullable. gitlab.com rate-limits anonymous clients after a dozen calls, so coverage is thin. |

**9 of 13 APIs** had at least one hard violation on a successful response, across 171 distinct schema locations. Data generated 2026-11-02, source: https://github.com/apidrifter/apidrifter/tree/main/tools/survey/results

## The ones worth reading

**Discourse** publishes a spec generated from its own test fixtures, and it has drifted furthest from the live site. Topic tags come back as objects where the spec says strings. Category `custom_fields` carries plugin keys that the schema forbids with `additionalProperties: false`. Several fields the spec marks required only appear for logged-in users. None of this is subtle, and all of it would break a generated client on the first page it fetched.

**Wikimedia's REST API** returns an array from its lint endpoint where the spec says object, its media lists carry `leadImage` and `srcset` fields the spec has never heard of, and its 400 errors come back as plain `application/json` rather than the documented `application/problem+json`.

**GitLab's** generated OpenAPI document declares several list endpoints as returning an object. They return arrays. The namespace `parent_id` is null for top-level groups, and the spec does not allow null.

**APIs.guru**, the directory most of us use to find specs, documents its per-API endpoint as returning an index entry with `preferred` and `versions`. It returns the full OpenAPI document instead.

**Gitea and Forgejo** serve their own Swagger 2.0 documents, and Swagger 2.0 has no way to say a field can be null. The live API returns null for nested objects constantly. That is less an API that drifted than a spec format that cannot describe the API, which is its own kind of problem for anyone generating a client from it.

**Swagger Petstore**, the reference demo for the whole format, returns a plain-text session string under an `application/json` content type on login.

**GitHub** is the one we cannot yet report properly. The first pass found gist files with a null `language` the spec forbids, and event payloads matching two `oneOf` branches at once. The second pass hit the anonymous rate limit after nine successful responses, so GitHub's row understates it until we rerun with a token.

## What this does and does not show

It shows that published specs and live behavior disagree often enough that checking is worth automating, and that the disagreements are the kind that break real clients, not pedantry about descriptions.

It does not show a rate. Thirteen APIs chosen because they publish a spec, one request per operation, read-only, no credentials, is a sample, not a measurement. Treat the nine of thirteen as "common," not as a percentage.

## Method

- One `GET` per operation, values from the spec's own examples or a small hand-written set per API, listed in the open-source target file.
- A delay between requests, a User-Agent naming the survey, one polite retry on 429, and a hard request budget per API.
- Swagger 2.0 documents were converted to OpenAPI 3 before checking.
- Every request, status and finding is in the repository. The tool, the runner and the targets are open source, so anyone can rerun it.

If you maintain one of these APIs and a finding is wrong, we want to know. If it is right, we are opening fixing pull requests on the smaller specs first.

## Cite this

API Drifter (2026). We checked 13 public APIs against their own OpenAPI specs. 9 had hard mismatches.. https://apidrifter.com/research/public-api-spec-drift-2026-11/