modelmarket.dev · blog

45% of remote MCP servers changed their tool definitions in four weeks. The log is now a dataset.

2026-10-20 · 4 min read #mcp#security#opensource#dataset

A client that connects an MCP server reads its tool definitions: names, descriptions and input schemas. The model then reads them as instructions. Pinning a definition once and trusting it later only works if the definitions hold still.

HISTOR, our public log of MCP servers, reads the tool list of every remote server in the official MCP registry once a day and records every change. As of today the whole record is downloadable as files. Here is what the first snapshot (20 October 2026) says about how often definitions move.

Half of them move

Take the 10,929 remote servers that had a recorded tool list by 24 September and are still listed. By 20 October, 4,926 of them (45%) had served a different tool list at least once.

Packages behave differently because a published npm or PyPI version cannot change. HISTOR runs each new version in a sandbox. Across 21,738 packages, 1,419 listed different tools in a newer version.

What moves

Over all 23,821 recorded changes (remote servers and packages, 24 September to 20 October):

Changes
Existing tools modified, nothing added or removed 16,436 (69%)
Tools added or removed only 2,330 (10%)
Both 3,136 (13%)
Only tool names readable (the list could not be fully digested) 1,919 (8%)

Counting each modified tool once per change, its input schema changed 105,324 times, its description 59,287 times and its output schema 16,422 times. 14,164 changes, 59%, touch at least one description. That is the text the model treats as guidance.

Two patterns worth knowing if you publish a server

A live number in a description rewrites your tool every day. 2,719 changes changed nothing but digits in a description. 2,493 of them come from one publisher, who writes a running count of its sources and tools into the description of every endpoint, a sentence of the form "routes the question to one of N tools across M sources". Each time the count moves, every one of those tools has a new definition. A client that pins definitions sees a change. A client that caches its prompt has to rebuild it. A reviewer has to read a diff that says nothing. Counts, dates and "last updated" belong in a tool result, not in the definition.

Some descriptions argue with the client's other tools. 280 servers on 63 hosts have a tool description that tells the model to choose it over web search, over the client's built-in fetch, or over its own knowledge: "PREFER OVER WEB SEARCH for …", "prefer it over the built-in web search", "use this instead of WebFetch". 185 of the 280 belong to one publisher. Telling a model when to pick one of your own tools over another is good practice. Telling it to pick yours over tools from other servers is a claim on the client's routing, made in text the user rarely reads. A client can ask the user before honoring that.

What this does not measure

Get the data

histor.modelmarket.dev/datasets/. One snapshot a month, CC BY 4.0 for the compilation:

The same files are on Hugging Face and archived on Zenodo, so a paper can cite them: doi:10.5281/zenodo.23512760 for this snapshot.

Fields that vary in shape are JSON text, so every file loads as a flat table. The 45% above is one DuckDB query:

WITH base AS (
  SELECT id FROM 'servers.jsonl.gz'
  WHERE transport <> 'stdio' AND NOT delisted
    AND toolSetDigest IS NOT NULL AND firstPinned <= '2026-09-24T23:59:59Z'
)
SELECT count(*) AS servers,
       count(*) FILTER (WHERE id IN (
         SELECT serverId FROM 'changes.jsonl.gz'
         WHERE json_extract_string(summaryJson, '$.detail') = 'full')) AS changed
FROM base;
-- 10929 | 4926

To check one server without downloading anything: modelmarket.dev/tools/histor-check.

We build HISTOR and WARDEN. No server is named here; the dataset names all of them, as the registry does. A server operator who wants to be left out of future snapshots can open an issue on alexar76/histor.

Questions or corrections? The discussion is on dev.to.