Ask the Spec in Claude: same answers, new door
An MCP server that lets Claude, or any assistant that speaks the Model Context Protocol, search, read and question the same published specification, with the same citations and the same refusals.
The story in three parts
A web page is one door
Ask the Spec answered questions on its own page. An assistant someone already works in had no way to use it.
Four read-only tools
List the documents, read one excerpt, search without an answer, or ask for a cited answer. Two of the four cost nothing to call.
Live, 5 of 5 checks held
11 automated tests pass, and all 5 live checks from Claude came back right, including a refusal and a withheld value.
What the work turned up
Claude picked the cheap tool
Asked what TC‑38 tested, Claude read the one excerpt instead of running the paid answer pipeline.
No new answer logic
The tools reuse the retrieval, citation check and name check the web page already runs, so the two can't drift apart.
A server can't police its client
The server refused a company question. Claude then answered it anyway from its own memory of me.
The problem
Ask the Spec worked, but only for someone who opened its page.
Ask the Spec answers questions about the Anomaly Upload Validator's five published specification documents, cites the exact section, and refuses when the documents don't say. It did that through one web page.
The Model Context Protocol (MCP) is a standard way for an AI assistant to call outside tools. I wanted to see what it takes to turn a working retrieval app into tools an assistant can call, and whether the guardrails survive the move.
What I built
One new endpoint on the existing app, ask.kileysullivan.com/mcp, with four tools. Every tool only reads; none can change anything.
| Tool | What it returns | Cost per call |
|---|---|---|
| list_documents | The five documents, how many excerpts each has, and their sections | Nothing; reads the index |
| get_section | One excerpt in full, by its ID (TC‑38, FR‑02.3, v2.5) or its excerpt number | Nothing; reads the index |
| search_spec | The best-matching excerpts with links, and no written answer, so the assistant reads the source itself | One search embedding |
| ask_spec | The same cited answer or refusal the web page gives | One search embedding and one Claude call |
The server also sends the assistant short instructions when it connects: cite the excerpt link, never guess a withheld value, and never name a company, client or vendor.
Decisions and trade-offs
Each choice bought something specific and cost something specific.
| Decision | Why | What it cost |
|---|---|---|
| Wrap the existing pipeline, not a copy | The web page and the tools give the same answer, and a fix to one is a fix to both | The tools inherit every limit the web page has |
| A search tool with no written answer | An assistant can read the source and reason over it, for one search embedding instead of an embedding plus a Claude call | The assistant, not my pipeline, decides what to say about what it reads |
| No sign-in | The documents are already public, and anyone can try it in a minute | Anyone can spend against my monthly caps, up to a per-visitor hourly limit |
| Stateless: every request stands alone | Fits a serverless function, with no session store to run | No live updates pushed to the assistant; it has to ask |
| Every tool marked read-only | Tells the assistant these tools are safe to call; how it uses that is up to the assistant | Nothing; there is nothing for them to change |
How I tested it
Two layers on October 22, 2026: 11 automated tests before it shipped, then 5 live checks from Claude after.
The automated tests start a real server and call it with the official MCP client, over the 314 real excerpts. They cover each tool, a lookup by ID, a search that should match nothing, input that is too long, and a request the server should turn away. The search scores and the answers are stand-ins in the tests, so they cost nothing and give the same result every time.
| Asked from Claude | Tool Claude used | Result |
|---|---|---|
| What did TC‑38 test? | get_section | Right test case, with a link to its table |
| Can I export while Manual Reviews are still open? | ask_spec | Right answer, cited to a user story, two test cases and a requirement |
| Which company built this tool? | ask_spec | The server refused |
| A withheld conversion value | ask_spec | Said the value is withheld and why, and cited the redaction note |
| An overview of the documents | list_documents | All five documents, 314 excerpts |
I didn't rerun the 30-question evaluation from Ask the Spec through these tools. ask_spec runs the same code that evaluation measured, so its results carry over; the other three tools are new and covered only by the checks above.
Where it doesn't hold
- The server controls what it returns, not what the assistant says. In the company check, the server refused, then Claude supplied the answer from what it already knew about me. A stranger's assistant wouldn't know it, but the lesson stands: guardrails in a tool stop at the tool.
- search_spec hands the reasoning to the assistant. Its excerpts carry no written answer, so the citation and name checks that guard ask_spec don't run on whatever the assistant writes from them.
- Five live checks is a small sample. They show the tools work end to end, not how well they hold up across many questions.
- The hourly limit is best effort. It counts per server instance, so it resets when the host starts a new one. The monthly spend caps are the real ceiling.
Connect it yourself
In Claude, add a custom connector with this address, and leave sign-in blank.
https://ask.kileysullivan.com/mcp
Then ask "Using Ask the Spec, what did TC‑38 test?" To see a refusal, ask for a conversion table value. Any other assistant that supports remote MCP servers over Streamable HTTP can connect the same way.