Check an MCP server against the spec
plune mcp conformance checks an MCP server against the text of the
MCP specification, check by check. It
starts the server you name after --, or connects to the --url you give, sends its requests, and
reports for each check whether the server did what the specification says. Run it when you build or
change a server, in CI on every pull request, and before you tell anyone your server “follows the
spec”. It needs no Plune account and sends nothing to the Plune platform: the only traffic is between
the command and the server you point it at.
Quick start
Section titled “Quick start”For a server that speaks stdio, name its start command after --:
plune mcp conformance -- node dist/server.jsFor a server at a Streamable HTTP address, give the URL (--header "Name: value" adds a header, such as
a login, to every request to that address and to no other):
plune mcp conformance --url http://127.0.0.1:3000/mcpThe whole output (88 lines)
$ plune mcp conformance -- node server.mjs goodplune mcp conformance · conformance-good 1.0.0 · stdio · protocol 2025-11-25, legacy era (server/discover answered error -32601, not a modern one: legacy, the initialize handshake)Profile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST life-initialize-result passed MUST life-version-negotiation passed MUST life-capabilities-declared-served passed MUST life-capabilities-served-declared passed MUST life-ping-empty-result not-applicable MUST mod-discover-result (era) · not in the profile, not scored not-applicable SHOULD mod-discover-server-info (era) not-applicable MUST mod-meta-required-fields (era) · not in the profile, not scored not-applicable MUST mod-unsupported-version (era) · not in the profile, not scored not-applicable MUST base-jsonrpc-parse-error (transport) passed advice stdio-parse-error-advice passed MUST base-unknown-method-error passed MUST base-response-id-matches passed SHOULD base-invalid-params-error passed MUST tools-list-shape passed MUST tools-input-schema-valid passed MUST tools-output-schema-valid could-not-run MUST tools-structured-conformant (needs-input) could-not-run SHOULD tools-structured-has-text (needs-input) passed SHOULD tools-name-format passed SHOULD tools-name-unique passed SHOULD tools-call-unknown-error not-applicable SHOULD tools-list-order-stable (era) passed MUST resources-list-shape passed MUST resources-read-listed passed SHOULD resources-read-not-found passed MUST resources-templates-shape passed MUST prompts-list-shape passed SHOULD prompts-get-unknown-error passed SHOULD pagination-walk-complete passed SHOULD pagination-invalid-cursor passed SHOULD logging-set-level passed SHOULD completion-complete not-applicable MUST mod-removed-methods (era) · not in the profile, not scored not-applicable MUST mod-stateless-no-prior-context (era) · not in the profile, not scored not-applicable MUST mod-list-stable-across-connections (era) · not in the profile, not scored not-applicable MUST mod-resource-not-found (era) · not in the profile, not scored not-applicable MUST mod-resource-no-empty-contents (era) · not in the profile, not scored not-applicable MUST mod-cache-hints-coverage (era) · not in the profile, not scored not-applicable MUST mod-cache-ttl-valid (era) · not in the profile, not scored not-applicable MUST mod-cache-scope-valid (era) · not in the profile, not scored not-applicable MUST mod-cache-scope-stable-pages (era) · not in the profile, not scored not-applicable MUST mod-undeclared-capability (era) · not in the profile, not scored not-applicable MUST mod-sub-ack-first (era) · not in the profile, not scored not-applicable MUST mod-sub-filter-and-tag (era) · not in the profile, not scored not-applicable SHOULD mod-sub-graceful-close (era) not-applicable MUST http-notification-202 (transport) · not in the profile, not scored not-applicable MUST http-request-content-type (transport) · not in the profile, not scored not-applicable MUST http-response-utf8 (transport) · not in the profile, not scored not-applicable MUST http-single-endpoint (transport) · not in the profile, not scored not-applicable MUST http-get-stream-or-405 (transport) · not in the profile, not scored not-applicable SHOULD http-modern-get-delete-405 (transport) not-applicable MUST http-session-id-visible-ascii (transport) · not in the profile, not scored not-applicable SHOULD http-session-required-400 (transport) not-applicable MUST http-protocol-version-header-invalid-400 (transport) · not in the profile, not scored not-applicable MUST http-concurrent-posts-matched (transport) · not in the profile, not scored not-applicable MUST http-sse-event-ids-unique (transport) · not in the profile, not scored not-applicable SHOULD http-sse-stream-closes (transport) not-applicable MUST http-sse-no-server-requests (transport) · not in the profile, not scored not-applicable SHOULD http-accel-buffering-header (transport) not-applicable MUST http-close-cancels (transport) · not in the profile, not scored not-applicable MUST mod-unimplemented-method (transport) · not in the profile, not scored not-applicable MUST mod-header-protocol-version-mismatch (transport) · not in the profile, not scored not-applicable MUST mod-header-method-mismatch (transport) · not in the profile, not scored not-applicable MUST mod-header-name-mismatch (transport) · not in the profile, not scored not-applicable MUST mod-header-missing-rejected (transport) · not in the profile, not scored not-applicable MUST mod-header-names-case-insensitive (transport) · not in the profile, not scored not-applicable MUST mod-header-name-base64-decoded (transport) · not in the profile, not scored not-applicable MUST mod-param-header-mismatch (transport) · not in the profile, not scored not-applicable MUST mod-x-mcp-header-declarations (transport) · not in the profile, not scored not-applicable SHOULD mod-no-session-id (transport) not-applicable MUST http-session-terminated-404 (transport) · not in the profile, not scored not-applicable MUST http-sse-no-broadcast (transport) · not in the profile, not scored not-applicable MUST http-origin-invalid-403 (transport) · not in the profile, not scored not-applicable control http-origin-localhost-accepted (transport) not-applicable control http-origin-absent-accepted (transport) not-applicable SHOULD http-bind-loopback (transport) not-applicable advice http-host-rebinding-advice (transport) not-applicable MUST mod-result-type-present (era) · not in the profile, not scored not-applicable MUST mod-error-code-range (era) · not in the profile, not scored not-applicable MUST mod-logs-need-log-level (era) · not in the profile, not scored not-applicable MUST mod-wire-schema-valid (era) · not in the profile, not scored passed MUST stdio-stdout-clean passed SHOULD stdio-exit-on-eof
protocol: Score 100.0 = 15 of 15 | outside the score: not-applicable 1, pending 42, needs-input 1, advisories 23, controls 2Verdict: passed (exit 0): every scored check that applies passed.Exit code: 0.
Read it from the bottom. The last two lines are the result: a score for the checks that count, and a
verdict that decides the exit code. Above them is every check with its state, its level (MUST,
SHOULD, control, advice) and its id. The score and the states
below say what each part means.
Useful flags: --check <id> runs one check (repeatable), --list-checks prints the catalog without a
server, --explain <id> prints one entry, --format json and --out [json=|junit-xml=]<file> write
machine-readable results, --timeout bounds how long the command waits for the server, --fixture <file> names the only tools the run may call (see Tools are never called on their own),
and --profile <version> picks the profile the run is scored with (see The profile).
The score
Section titled “The score”Only MUST checks count, and only those of a frozen profile (see The profile).
The score is
score = floor(1000 × passed / applicable) / 10applicable = passed + failed + could-not-runso it is rounded down to a tenth: 1999 of 2000 is 99.9, never 100. not-applicable checks,
checks outside the profile, and checks that could not run only for want of an input you did not give
(needs-input) stay out of applicable. Any other could-not-run stays in: a check that could not
run lowers the score as a failure would, because “unknown” is not “fine”. With nothing measured
(applicable is 0) the score is none.
SHOULD and MAY checks, controls and advice are printed but never change the score or the exit code.
The summary line of the example above is:
protocol: Score 100.0 = 15 of 15 | outside the score: not-applicable 1, pending 42, needs-input 1, advisories 23, controls 2Score 100.0 = 15 of 15: 15 scored checks passed out of 15 applicable.not-applicable 1: a scored check that does not apply here;base-jsonrpc-parse-erroris for HTTP, and on stdio the parse error is read by an advice,stdio-parse-error-advice.pending 42:MUSTchecks that are not in the profile. Their verdict is on the line above, but the score does not read it, whatever it is (most of these arenot-applicablehere: they are for the other revision, or for HTTP).needs-input 1: a scored check that could not run for want of an input only you can give; heretools-structured-conformant, which needs--fixture.advisories 23:SHOULDandMAYchecks and advice.controls 2: probes other checks rest on (see Streamable HTTP and a local server).
The 84 checks of the protocol suite add up exactly: 15 + 1 + 1 + 42 + 23 + 2. Every check lands in one bucket.
That run was captured under profile 2026-11.2. Under 2026-11.3 the same stdio server gets the same score,
Score 100.0 = 15 of 15, and 14 checks move from pending to not-applicable: the http-* checks that are
now in the profile, and are for HTTP (not-applicable 15, pending 28).
A server that breaks requirements shows it. This one writes a log line on stdout and answers badly, so three scored checks fail:
$ plune mcp conformance --check life-ping-empty-result --check tools-input-schema-valid --check stdio-stdout-clean --check tools-name-format -- node server.mjs badplune mcp conformance · conformance-bad 1.0.0 · stdio · protocol 2025-11-25, legacy era (server/discover answered neither a DiscoverResult nor a modern error: legacy, the initialize handshake)Profile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
failed MUST life-ping-empty-result failed MUST tools-input-schema-valid failed SHOULD tools-name-format failed MUST stdio-stdout-clean
protocol: Score 0.0 = 0 of 3 | failed: life-ping-empty-result, tools-input-schema-valid, stdio-stdout-clean | outside the score: advisories 1Verdict: failed (exit 1): 3 scored checks failed.Exit code: 1.
tools-name-format also failed, but it is a SHOULD: it is in the advisories count and does not
touch the score.
What passed proves, and what it does not
Section titled “What passed proves, and what it does not”passed means the server did what the cited text of the specification asks, in the way this check
asks it: one request, one answer, judged as the server sent it. It does not mean the server is correct
beyond that, secure, or fast. A check passes on the requests the suite makes; a bug that needs another
request goes unseen. could-not-run and not-applicable are not passes: the first says the check
could not be carried out, the second that it does not apply to this server. A 100.0 over 15 checks is
a statement about those 15 checks.
States, verdicts and exit codes
Section titled “States, verdicts and exit codes”Each check ends in one of five states:
| State | Meaning |
|---|---|
passed | The server did what the check asks. |
failed | The server’s answer shows it did not. Never given for a timeout, a refused connection or a server error: those are could-not-run. |
not-applicable | The check does not apply, with a reason: era (the other revision), capability-not-declared, transport (for example an HTTP check on stdio), no-auth (the HTTP server needs no login), not-offered (the server offers nothing the check reads), not-local (a local-server check on another host). |
could-not-run | The check could not be carried out, with a reason: timeout, prerequisite-failed, needs-input, transport-error, unsupported-dialect. It is never a pass. |
pending | No result yet, or not in the profile. |
The verdict of a run is failed when a scored check failed; otherwise incomplete when a scored
check could not run (other than for needs-input) or nothing was measured; otherwise passed. The
exit code follows the verdict:
| Exit | Meaning |
|---|---|
0 | Every scored check that applies passed. |
1 | A scored check failed. A SHOULD, a MAY, advice, or a check outside the profile never gives 1. |
2 | The server did not start or could not be reached, or a flag is wrong. |
3 | Nothing failed, but conformance is not established: a scored check could not run, or nothing was measured. |
Exit 3 is not “all good”. Here the only check asked for needs a tool call that --fixture would
name, so nothing was measured and nothing is proven:
$ plune mcp conformance --check tools-structured-conformant -- node server.mjs goodplune mcp conformance · conformance-good 1.0.0 · stdio · protocol 2025-11-25, legacy era (server/discover answered error -32601, not a modern one: legacy, the initialize handshake)Profile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
could-not-run MUST tools-structured-conformant (needs-input)
protocol: Score none = 0 of 0 | outside the score: needs-input 1Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.Exit code: 3.
A wrong flag, such as an id the catalog does not have, is exit 2:
$ plune mcp conformance --check nope -- node server.mjs goodplune mcp conformance: --check nope: no such check (plune mcp conformance --list-checks lists them).Exit code: 2.
The profile
Section titled “The profile”The score reads a list of check ids written out one by one in the CLI’s source, named in every result:
profileId, profileVersion and manifestDigest (the SHA-256 of the sorted ids, one per line). A check
added to the catalog later runs and shows its verdict, but stays out of the score until a new profile
version lists it, so a new check never changes the score or the exit code of a server you measured
before. Two results with the same profileVersion and manifestDigest were scored by the same rules.
The current profile is plune-mcp-conformance version 2026-11.3, digest
fd68f7f5dc2e…, and it lists 31 checks, all of them in the protocol
suite: the handshake, the capabilities against the methods served, ping, the JSON-RPC answers to invalid
JSON (over HTTP; on stdio an advice reads it) and an unknown method, the lists of tools, resources, templates and prompts, the schemas of tools,
on stdio what the server writes to stdout, and over HTTP the MUST checks of Streamable HTTP on the wire and
of the local-server protection (http-*). Everything else shows its verdict and counts for nothing:
- every check of the oauth, tasks and apps suites, so a run of any of them alone always ends with exit
code
3, even when a check failed: read the list, not the exit code; - every
mod-*check (the 2026-07-28 revision); - every
SHOULD,MAY, control and advice, thehttp-*ones included.
A run is scored with the current profile unless --profile names another version:
| Version | What it scores |
|---|---|
2026-11.3 (default) | The checks above. |
2026-11.2 | The same, without the http-* checks: the score of a run before they counted. |
Over stdio every http-* check is not-applicable (transport), so a stdio server gets the same score under
both. Over HTTP a server that breaks an http-* MUST gets a lower score under 2026-11.3 and can get exit 1
where 2026-11.2 gave it 3. On a host that is not a loopback name, http-origin-invalid-403 is
not-applicable (not-local) and stays out of the score. Any other value of --profile is exit 2;
2026-11.1 is not offered, because a new run applies today’s rules to its ids, and those are the rules of
2026-11.2.
Protocol revisions
Section titled “Protocol revisions”--protocol-version takes auto (the default), a revision such as 2025-11-25 or 2026-07-28, or
both. The checks are split into two eras: legacy, the revisions up to 2025-11-25, which open with
an initialize handshake, and modern, 2026-07-28, which has no handshake: every request carries the
version and the client’s capabilities, and server/discover tells the server’s versions. A check for the
other era is not-applicable (era).
With auto, the command first sends server/discover for 2026-07-28; an answer that is a discovery
result or a modern error means a modern server, anything else means a legacy one and the run makes the
initialize handshake on a new connection. The first line of the output says which era and why, as in
the examples above (legacy era (server/discover answered error -32601, not a modern one …)). both
runs a server that serves both eras twice, one report per era, each with its own score; the exit code is the
worse of the two. This one is a modern server:
$ plune mcp conformance --protocol-version 2026-07-28 --check mod-discover-result --check mod-resource-not-found --check base-unknown-method-error -- node modern-server.mjs goodplune mcp conformance · modern-good 1.0.0 · stdio · protocol 2026-07-28, modern era (from --protocol-version)Profile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST mod-discover-result · not in the profile, not scored passed MUST base-unknown-method-error passed MUST mod-resource-not-found · not in the profile, not scored
protocol: Score 100.0 = 1 of 1 | outside the score: pending 2Verdict: passed (exit 0): every scored check that applies passed.Exit code: 0.
As of 19 November 2026 the versioning page
of the specification names 2026-07-28 as the current revision and lists 2025-11-25 among the
earlier ones. Check that page for the state of the day you read this.
The checks
Section titled “The checks”The table is built from the catalog the command prints (plune mcp conformance --list-checks --format json),
not written by hand: 132 entries across the protocol, oauth, tasks and apps suites.
MUST is used only where the cited section says so; a level read from neighbouring provisions is marked
(derived), and --explain <id> says which reading the check takes. The specification column links
to the section each check rests on. The table scrolls sideways on a narrow screen.
| Id | What it checks | Level | Specification | Revisions | Transport | Scored |
|---|---|---|---|---|---|---|
life-initialize-result | initialize answers with protocolVersion, capabilities and serverInfo | MUST | basic/lifecycle · Initialization | 2025-11-25 and earlier | stdio, http | yes |
life-version-negotiation | The protocol version is negotiated | MUST | basic/lifecycle · Version Negotiation | 2025-11-25 and earlier | stdio, http | yes |
life-capabilities-declared-served | Every declared capability is served | MUST (derived) | basic/lifecycle · Operation | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
life-capabilities-served-declared | Every served list is declared | MUST | server/tools · Capabilities server/resources · Capabilities server/prompts · Capabilities | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
life-ping-empty-result | ping answers with an empty result | MUST | basic/utilities/ping · Behavior Requirements | 2025-11-25 and earlier | stdio, http | yes |
mod-discover-result | server/discover answers a complete DiscoverResult | MUST | server/discover · Discovery basic/versioning · Protocol Version Negotiation | 2026-07-28 | stdio, http | no |
mod-discover-server-info | server/discover names the server in _meta | SHOULD | server/discover · DiscoverResult basic/index · _meta | 2026-07-28 | stdio, http | no |
mod-meta-required-fields | A request without a required _meta field is Invalid params | MUST | basic/index · _meta basic/transports/streamable-http · Server Validation | 2026-07-28 | stdio, http | no |
mod-unsupported-version | An unsupported protocol version is UnsupportedProtocolVersion | MUST | basic/versioning · Protocol Version Negotiation basic/transports/streamable-http · Protocol Version Header | 2026-07-28 | stdio, http | no |
base-jsonrpc-parse-error | Invalid JSON is a parse error | MUST (derived) | basic/index · Messages | 2025-11-25 and earlier, 2026-07-28 | http | yes |
stdio-parse-error-advice | Invalid JSON on stdio is answered with a parse error (advice) | advice | basic/transports · stdio | 2025-11-25 and earlier, 2026-07-28 | stdio | no |
base-unknown-method-error | An unknown method is Method not found | MUST (derived) | basic/index · Messages | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
base-response-id-matches | A response carries the id of its request | MUST | basic/index · Responses basic/index · Responses | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
base-invalid-params-error | Plainly wrong parameters are Invalid params | SHOULD | server/tools · Error Handling | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
tools-list-shape | Every listed tool has a name and an inputSchema object | MUST | server/tools · Tool | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
tools-input-schema-valid | Every inputSchema is a valid object schema | MUST | basic/index · JSON Schema Usage | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
tools-output-schema-valid | Every outputSchema is a valid schema | MUST | server/tools · Output Schema changelog · Minor changes | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
tools-structured-conformant | structuredContent conforms to the outputSchema | MUST | server/tools · Output Schema | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
tools-structured-has-text | structuredContent is also given as text | SHOULD | server/tools · Structured Content | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
tools-name-format | Tool names use the allowed characters | SHOULD | server/tools · Tool Names | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
tools-name-unique | Tool names are unique | SHOULD | server/tools · Tool Names | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
tools-call-unknown-error | Calling an unknown tool is a protocol error | SHOULD | server/tools · Error Handling | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
tools-list-order-stable | tools/list keeps its order | SHOULD | server/tools · Capabilities | 2026-07-28 | stdio, http | no |
resources-list-shape | Every listed resource has a uri and a name | MUST (derived) | server/resources · Resource | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
resources-read-listed | A listed resource can be read | MUST (derived) | server/resources · Resource Contents | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
resources-read-not-found | Reading an unknown resource is an error | SHOULD | server/resources · Error Handling | 2025-11-25 and earlier | stdio, http | no |
resources-templates-shape | Every resource template has a uriTemplate and a name | MUST (derived) | server/resources · Resource Templates | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
prompts-list-shape | Every listed prompt has a name | MUST (derived) | server/prompts · Prompt | 2025-11-25 and earlier, 2026-07-28 | stdio, http | yes |
prompts-get-unknown-error | Getting an unknown prompt is an error | SHOULD | server/prompts · Error Handling | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
pagination-walk-complete | Paging through a list ends, without repeats | SHOULD | server/utilities/pagination · Implementation Guidelines | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
pagination-invalid-cursor | An invalid cursor is Invalid params | SHOULD | server/utilities/pagination · Error Handling | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
logging-set-level | logging/setLevel is accepted when logging is declared | SHOULD (derived) | server/utilities/logging · Capabilities | 2025-11-25 and earlier | stdio, http | no |
completion-complete | completion/complete answers when completions is declared | SHOULD (derived) | server/utilities/completion · Capabilities | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
mod-removed-methods | The methods the revision removed are Method not found | MUST (derived) | changelog · Major changes basic/transports/streamable-http · Protocol Version Header | 2026-07-28 | stdio, http | no |
mod-stateless-no-prior-context | A list answers on a fresh connection with no request before it | MUST | basic/index · Statelessness | 2026-07-28 | stdio, http | no |
mod-list-stable-across-connections | The lists do not vary per connection | MUST | server/tools · Capabilities server/prompts · Capabilities server/resources · Capabilities | 2026-07-28 | stdio, http | no |
mod-resource-not-found | Reading an unknown resource is Invalid params | MUST | server/resources · Error Handling basic/index · Error Codes | 2026-07-28 | stdio, http | no |
mod-resource-no-empty-contents | An unknown resource is not empty contents | MUST | server/resources · Error Handling | 2026-07-28 | stdio, http | no |
mod-cache-hints-coverage | Every cacheable result has ttlMs and cacheScope | MUST | server/utilities/caching · Cacheable Results | 2026-07-28 | stdio, http | no |
mod-cache-ttl-valid | ttlMs is an integer of at least 0 | MUST | server/utilities/caching · Time-to-Live (TTL) Field | 2026-07-28 | stdio, http | no |
mod-cache-scope-valid | cacheScope is "public" or "private" | MUST (derived) | server/utilities/caching · Cache Scope Field | 2026-07-28 | stdio, http | no |
mod-cache-scope-stable-pages | All pages of a list have the same cacheScope | MUST | server/utilities/caching · Interaction with Pagination | 2026-07-28 | stdio, http | no |
mod-undeclared-capability | A capability the client did not declare is asked for with -32021 | MUST | basic/index · _meta | 2026-07-28 | stdio, http | no |
mod-sub-ack-first | A subscription is acknowledged first | MUST | basic/patterns/subscriptions · Acknowledgment | 2026-07-28 | stdio, http | no |
mod-sub-filter-and-tag | A subscription gets only what it asked for, each tagged | MUST | basic/patterns/subscriptions · Opening a Stream basic/index · _meta | 2026-07-28 | stdio, http | no |
mod-sub-graceful-close | A subscription the server ends gets a successful response | SHOULD | basic/patterns/subscriptions · Graceful Closure | 2026-07-28 | stdio, http | no |
http-notification-202 | A notification POST is 202 with no body, or an error status | MUST | basic/transports · Sending Messages to the Server basic/transports/streamable-http · Sending Messages | 2025-11-25 and earlier, 2026-07-28 | http | yes |
http-request-content-type | A request is answered as application/json or text/event-stream | MUST | basic/transports · Sending Messages to the Server basic/transports/streamable-http · Sending Messages | 2025-11-25 and earlier, 2026-07-28 | http | yes |
http-response-utf8 | The answer is UTF-8 JSON | MUST | basic/transports · Transports | 2025-11-25 and earlier, 2026-07-28 | http | yes |
http-single-endpoint | The MCP endpoint takes POST | MUST | basic/transports · Streamable HTTP basic/transports/streamable-http · Streamable HTTP | 2025-11-25 and earlier, 2026-07-28 | http | yes |
http-get-stream-or-405 | GET opens an SSE stream or is 405 | MUST | basic/transports · Listening for Messages from the Server | 2025-11-25 and earlier | http | yes |
http-modern-get-delete-405 | A modern-only server answers GET and DELETE with 405 | SHOULD | basic/transports/streamable-http · Earlier Streamable HTTP Revisions | 2026-07-28 | http | no |
http-session-id-visible-ascii | A session id is visible ASCII | MUST | basic/transports · Session Management | 2025-11-25 and earlier | http | yes |
http-session-required-400 | A request without the session id is 400 | SHOULD | basic/transports · Session Management | 2025-11-25 and earlier | http | no |
http-protocol-version-header-invalid-400 | An invalid MCP-Protocol-Version is 400 | MUST | basic/transports · Protocol Version Header | 2025-11-25 and earlier | http | yes |
http-concurrent-posts-matched | Concurrent POSTs each get their own answer | MUST (derived) | basic/transports · Sending Messages to the Server basic/index · Responses | 2025-11-25 and earlier, 2026-07-28 | http | yes |
http-sse-event-ids-unique | SSE event ids are unique in the session | MUST | basic/transports · Resumability and Redelivery | 2025-11-25 and earlier | http | yes |
http-sse-stream-closes | The SSE stream of a request ends after its response | SHOULD | basic/transports · Sending Messages to the Server basic/transports/streamable-http · Receiving Messages | 2025-11-25 and earlier, 2026-07-28 | http | no |
http-sse-no-server-requests | The SSE stream of a request holds no request of the server | MUST | basic/transports/streamable-http · Receiving Messages | 2026-07-28 | http | yes |
http-accel-buffering-header | An SSE answer sends X-Accel-Buffering: no | SHOULD | basic/transports/streamable-http · Receiving Messages | 2026-07-28 | http | no |
http-close-cancels | Closing the SSE stream cancels the request | MUST | basic/transports/streamable-http · Cancellation | 2026-07-28 | http | yes |
mod-unimplemented-method | An unknown method over HTTP is 404 with -32601 | MUST | basic/transports/streamable-http · Protocol Version Header | 2026-07-28 | http | no |
mod-header-protocol-version-mismatch | An MCP-Protocol-Version unlike the body is HeaderMismatch | MUST | basic/transports/streamable-http · Protocol Version Header basic/transports/streamable-http · Server Validation | 2026-07-28 | http | no |
mod-header-method-mismatch | An Mcp-Method unlike the body is HeaderMismatch | MUST | basic/transports/streamable-http · Server Validation | 2026-07-28 | http | no |
mod-header-name-mismatch | An Mcp-Name unlike the body is HeaderMismatch | MUST | basic/transports/streamable-http · Standard Request Headers basic/transports/streamable-http · Server Validation basic/transports/streamable-http · Case Sensitivity | 2026-07-28 | http | no |
mod-header-missing-rejected | A request without a required header is HeaderMismatch | MUST | basic/transports/streamable-http · Server Validation basic/transports/streamable-http · Standard Request Headers basic/transports/streamable-http · Protocol Version Header | 2026-07-28 | http | no |
mod-header-names-case-insensitive | Header names are read in any case | MUST | basic/transports/streamable-http · Case Sensitivity | 2026-07-28 | http | no |
mod-header-name-base64-decoded | A Base64 Mcp-Name is decoded before it is compared | MUST | basic/transports/streamable-http · Value Encoding | 2026-07-28 | http | no |
mod-param-header-mismatch | An Mcp-Param header unlike the body, or missing, is HeaderMismatch | MUST | basic/transports/streamable-http · Server Behavior for Custom Headers | 2026-07-28 | http | no |
mod-x-mcp-header-declarations | Every x-mcp-header declaration is valid | MUST | server/tools · x-mcp-header basic/transports/streamable-http · Schema Extension | 2026-07-28 | http | no |
mod-no-session-id | A modern-only server neither mints nor echoes a session id | SHOULD | basic/transports/streamable-http · Earlier Streamable HTTP Revisions | 2026-07-28 | http | no |
http-session-terminated-404 | A terminated session is 404 | MUST | basic/transports · Session Management | 2025-11-25 and earlier | http | yes |
http-sse-no-broadcast | A message goes to one SSE stream, not to each | MUST | basic/transports · Multiple Connections | 2025-11-25 and earlier | http | yes |
http-origin-invalid-403 | A foreign Origin is refused with 403 | MUST | basic/transports · Security Warning basic/transports/streamable-http · Security & Endpoint | 2025-11-25 and earlier, 2026-07-28 | http | yes |
http-origin-localhost-accepted | An Origin of localhost is accepted | control | basic/transports · Security Warning basic/transports/streamable-http · Security & Endpoint | 2025-11-25 and earlier, 2026-07-28 | http | no |
http-origin-absent-accepted | A request without Origin is accepted | control | basic/transports · Security Warning basic/transports/streamable-http · Security & Endpoint | 2025-11-25 and earlier, 2026-07-28 | http | no |
http-bind-loopback | A local server listens on loopback only | SHOULD | basic/transports · Security Warning basic/transports/streamable-http · Security & Endpoint | 2025-11-25 and earlier, 2026-07-28 | http | no |
http-host-rebinding-advice | A foreign Host is refused (advice outside the specification) | advice | basic/transports · Security Warning basic/transports/streamable-http · Security & Endpoint | 2025-11-25 and earlier, 2026-07-28 | http | no |
mod-result-type-present | Every result has a known resultType | MUST | basic/index · ResultType | 2026-07-28 | stdio, http | no |
mod-error-code-range | No error code of the reserved range is invented | MUST | basic/index · Error Codes | 2026-07-28 | stdio, http | no |
mod-logs-need-log-level | No log message for a request without logLevel | MUST | server/utilities/logging · Per-request log level | 2026-07-28 | stdio, http | no |
mod-wire-schema-valid | Every message conforms to the schema of the revision | MUST (derived) | basic/index · Schema | 2026-07-28 | stdio, http | no |
stdio-stdout-clean | stdout holds only MCP messages | MUST | basic/transports · stdio basic/transports/stdio · stdio | 2025-11-25 and earlier, 2026-07-28 | stdio | yes |
stdio-exit-on-eof | The server exits when stdin closes | SHOULD | basic/transports/stdio · Shutdown | 2025-11-25 and earlier, 2026-07-28 | stdio | no |
oauth-requires-auth | A request without a token gets 401 | MUST | basic/authorization · Error Handling basic/authorization · Error Handling | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-challenge-bearer | The 401 carries a Bearer challenge | MUST (derived) | basic/authorization · Protected Resource Metadata Discovery Requirements basic/authorization/authorization-server-discovery · Protected Resource Metadata Discovery Requirements | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-prm-reachable | The protected resource metadata can be found | MUST | basic/authorization · Protected Resource Metadata Discovery Requirements basic/authorization/authorization-server-discovery · Protected Resource Metadata Discovery Requirements | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-challenge-metadata-url | resource_metadata names a URL that serves the document | MUST (derived) | basic/authorization · Protected Resource Metadata Discovery Requirements basic/authorization/authorization-server-discovery · Protected Resource Metadata Discovery Requirements | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-prm-valid | The protected resource metadata names an authorization server | MUST (derived) | basic/authorization · Authorization Server Location basic/authorization/authorization-server-discovery · Authorization Server Location | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-challenge-scope | The 401 names the scope it needs | SHOULD | basic/authorization · Protected Resource Metadata Discovery Requirements basic/authorization · Scope Selection Strategy | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-as-metadata-discoverable | The authorization server publishes metadata | MUST | basic/authorization · Authorization Server Metadata Discovery basic/authorization/authorization-server-discovery · Authorization Server Metadata Discovery | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-as-issuer-matches | The issuer of the metadata is the issuer it was found by | MUST (derived) | basic/authorization/authorization-server-discovery · Authorization Server Metadata Discovery | 2026-07-28 | http | no |
oauth-as-https | The authorization endpoints are HTTPS | MUST | basic/authorization · Communication Security basic/authorization/security-considerations · Communication Security | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-as-pkce-advertised | The authorization server advertises PKCE | MUST (derived) | basic/authorization · Authorization Code Protection basic/authorization/security-considerations · Authorization Code Protection | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-as-pkce-s256 | PKCE with S256 is advertised | SHOULD (derived) | basic/authorization · Authorization Code Protection basic/authorization/security-considerations · Authorization Code Protection | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-as-iss-advertised | The authorization server advertises the iss parameter | SHOULD | basic/authorization · Authorization Response Validation | 2026-07-28 | http | no |
oauth-as-cimd-advertised | The authorization server advertises Client ID Metadata Documents | SHOULD | basic/authorization · Client ID Metadata Documents basic/authorization/client-registration · Client ID Metadata Documents | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-neg-invalid-token-401 | A token that is not valid gets 401 | MUST | basic/authorization · Token Handling basic/authorization · Token Handling | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-neg-wrong-audience | A token issued for another resource is refused | MUST | basic/authorization · Token Handling basic/authorization · Access Token Privilege Restriction basic/authorization · Token Handling basic/authorization/security-considerations · Token Audience Binding and Validation | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-e2e-token-works | A token issued for the server is accepted | MUST (derived) | basic/authorization · Token Handling basic/authorization · Token Handling | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-neg-insufficient-scope | A token with too narrow a scope gets 403 with insufficient_scope | SHOULD (derived) | basic/authorization · Runtime Insufficient Scope Errors basic/authorization · Runtime Insufficient Scope Errors | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-authz-response-iss | iss in the authorization response is the issuer, and is advertised | MUST (derived) | basic/authorization · Authorization Response Validation | 2026-07-28 | http | no |
oauth-neg-pkce-wrong-verifier | A code exchanged with a wrong code_verifier is refused | MUST (derived) | basic/authorization · Authorization Code Protection basic/authorization/security-considerations · Authorization Code Protection | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-neg-token-redirect-mismatch | A code exchanged with another redirect_uri is refused as such | MUST (derived) | basic/authorization · Open Redirection basic/authorization/security-considerations · Open Redirection | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-reg-prereg | A pre-registered client is accepted | SHOULD (derived) | basic/authorization · Preregistration basic/authorization/client-registration · Pre-registration | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-reg-dcr | Dynamic client registration works | MAY | basic/authorization · Dynamic Client Registration basic/authorization/client-registration · Dynamic Client Registration | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-reg-dcr-http-redirect-rejected | A redirect over http outside loopback is not registered | MUST (derived) | basic/authorization · Communication Security basic/authorization · Open Redirection basic/authorization/security-considerations · Communication Security basic/authorization/security-considerations · Open Redirection | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-reg-cimd | A client_id that is a URL is read as a document | MUST (derived) | basic/authorization · Client ID Metadata Documents basic/authorization/client-registration · Client ID Metadata Documents | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-token-format | The token response has access_token and token_type Bearer | MUST (derived) | basic/authorization · Token Requirements basic/authorization · Token Requirements | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-neg-invalid-client | A client that does not exist is not accepted | MUST (derived) | basic/authorization · Open Redirection basic/authorization/security-considerations · Open Redirection | 2025-11-25 and earlier, 2026-07-28 | http | no |
oauth-neg-authorize-redirect-mismatch | A redirect URI that is not registered is not redirected to | MUST | basic/authorization · Open Redirection basic/authorization/security-considerations · Open Redirection | 2025-11-25 and earlier, 2026-07-28 | http | no |
apps-ui-resource-uri-scheme | The address of a UI resource starts with ui:// | MUST (derived) | apps · UI Resource Format apps · Resource Discovery | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
apps-ui-resource-exists | The resource a tool names can be read | MUST (derived) | apps · Resource Discovery | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
apps-ui-mime-type | The content of a UI resource is text/html;profile=mcp-app | MUST | apps · UI Resource Format | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
apps-ui-content-payload | The content is one of text and blob, and is an HTML5 document | MUST (derived) | apps · UI Resource Format | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
apps-ui-listed-mime | A listed UI resource has the mimeType text/html;profile=mcp-app | SHOULD | apps · UI Resource Format | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
apps-ui-visibility-values | visibility holds only model and app | SHOULD (derived) | apps · Resource Discovery | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
apps-ui-tool-text-fallback | A tool with a UI also returns content | MUST | apps · Client<>Server Capability Negotiation | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
apps-ui-meta-shapes | csp, permissions, domain and prefersBorder have the shapes of the text | SHOULD (derived) | apps · UI Resource Format | 2025-11-25 and earlier, 2026-07-28 | stdio, http | no |
tasks-result-type | A task handle has resultType "task", and no other result has it | MUST | tasks.md · Polymorphic Results tasks.md · Task Creation | 2026-07-28 | stdio, http | no |
tasks-undeclared-no-task | A client that did not declare the extension is given no task | MUST | tasks.md · Capability Negotiation | 2026-07-28 | stdio, http | no |
tasks-undeclared-capability-error | tasks/get, tasks/update and tasks/cancel without the declaration are -32021 | MUST | tasks.md · Protocol Errors tasks.md · Capability Negotiation | 2026-07-28 | stdio, http | no |
tasks-get-status-shape | tasks/get answers a task with resultType "complete" and the fields of a task | MUST (derived) | tasks.md · Task Polling tasks.md · Tasks | 2026-07-28 | stdio, http | no |
tasks-status-payload | completed has a result, failed an error, input_required inputRequests | MUST | tasks.md · Task Polling | 2026-07-28 | stdio, http | no |
tasks-ttl-poll-integers | ttlMs is an integer or null, pollIntervalMs an integer | SHOULD (derived) | tasks.md · Tasks | 2026-07-28 | stdio, http | no |
tasks-unknown-id-get | tasks/get for an unknown taskId is -32602 | MUST | tasks.md · Protocol Errors | 2026-07-28 | stdio, http | no |
tasks-unknown-id-update-cancel | tasks/update and tasks/cancel for an unknown taskId are -32602 | SHOULD | tasks.md · Protocol Errors | 2026-07-28 | stdio, http | no |
tasks-cancel-ack | tasks/cancel is acknowledged with an empty result | MUST | tasks.md · Task Cancellation | 2026-07-28 | stdio, http | no |
tasks-update-ack | tasks/update is acknowledged with an empty result | MUST | tasks.md · Task Update Requests | 2026-07-28 | stdio, http | no |
tasks-failed-status-message | A failed task has a statusMessage | SHOULD | tasks.md · Task Execution Errors | 2026-07-28 | stdio, http | no |
tasks-http-name-header | tasks/get with Mcp-Name set to the taskId is accepted | SHOULD (derived) | tasks.md · Streamable HTTP: Routing Headers | 2026-07-28 | http | no |
tasks-input-request-keys-unique | An inputRequests key never names two different requests | MUST (derived) | tasks.md · Task Update Requests | 2026-07-28 | stdio, http | no |
One entry in full:
$ plune mcp conformance --explain tools-list-shapetools-list-shape: Every listed tool has a name and an inputSchema objectEach entry of tools/list has a string name and an inputSchema that is an object, not null.suite: protocolkind: checklevel: MUSTeras: legacy, moderntransports: stdio, httpneeds a fixture: noin profile plune-mcp-conformance 2026-11.2: yes, scoredspec: https://modelcontextprotocol.io/specification/2025-11-25/server/tools (Tool)Exit code: 0.
Tools are never called on their own
Section titled “Tools are never called on their own”No tool is called unless you name it. --fixture takes a JSON list, and those are the only tools the
run calls:
[{ "tool": "get_time", "arguments": {} }]Name only tools that are safe to call: readOnlyHint is what a server says about a tool, not a
guarantee. Without --fixture, the checks that read a tool’s result (for example
tools-structured-conformant) are could-not-run (needs-input). Besides the fixture, the only
tools/call requests are two that cannot run a tool: one without a name and one for a name no server has.
Streamable HTTP and a local server
Section titled “Streamable HTTP and a local server”Over --url, the http-* checks send raw HTTP requests, with no MCP client library in between, because
a library corrects a status, drops a header or accepts an empty body, which is what these checks look for.
Each keeps the requests it sent and the answers in details of the JSON result, with Authorization,
cookies, --header values and the session id masked. Every request goes to the URL under test only; no
redirect is followed. A check opens a session of its own, closes it with DELETE when the run ends, and
the report says how many sessions it could not close (ownSessions). An SSE stream is read for at most 5
seconds, 100 events, 1 MiB, and a line of 64 KiB; a check that needed more of the stream than that is
could-not-run, never judged from a part. A redirect, 429 or a server error is could-not-run, and 401
is could-not-run with the hint to give the login with --header "Authorization: Bearer <token>".
On stdio these checks are not-applicable (transport): there is no HTTP to look at.
A refusal is only a pass when the same request in its valid form was accepted first (a control). The
local-server protection works that way. For a server on localhost, 127.0.0.1, [::1] or a
*.localhost name, http-origin-invalid-403 sends a foreign Origin and expects 403; two controls
without a level (http-origin-localhost-accepted, http-origin-absent-accepted) show that the server
takes its own origin and takes none. For any other host these checks are not-applicable (not-local):
which origins a remote server should take is not known. http-host-rebinding-advice sends a foreign
Host and is advice outside the specification, which never moves the score or the exit code.
The three outputs below were captured with CLI 0.39.1, under profile 2026-11.2, before the http-*
checks counted: each MUST there is marked “not in the profile, not scored”. Under 2026-11.3 the same
runs score them.
$ plune mcp conformance --url http://127.0.0.1:58465/mcp --check http-notification-202 --check http-single-endpoint --check http-session-id-visible-ascii --check http-session-required-400 --check http-origin-invalid-403 --check http-origin-localhost-accepted --check http-host-rebinding-adviceplune mcp conformance · raw 1 · http · protocol 2025-11-25, legacy era (server/discover got HTTP 400 without a modern error: legacy, the initialize handshake)Profile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST http-notification-202 · not in the profile, not scored passed MUST http-single-endpoint · not in the profile, not scored passed MUST http-session-id-visible-ascii · not in the profile, not scored passed SHOULD http-session-required-400 passed MUST http-origin-invalid-403 · not in the profile, not scored passed control http-origin-localhost-accepted passed advice http-host-rebinding-advice
protocol: Score none = 0 of 0 | outside the score: pending 4, advisories 2, controls 1Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.Exit code: 3.
A server that does not check Origin or Host fails them. Note the exit code: http-origin-invalid-403
is a MUST that failed, yet the exit is 3, because under 2026-11.2 no http-* check is scored. Under
2026-11.3 the same run is exit 1: http-origin-invalid-403 is a scored failure.
$ plune mcp conformance --url http://127.0.0.1:58471/mcp --check http-notification-202 --check http-single-endpoint --check http-session-id-visible-ascii --check http-session-required-400 --check http-origin-invalid-403 --check http-origin-localhost-accepted --check http-host-rebinding-adviceplune mcp conformance · raw 1 · http · protocol 2025-11-25, legacy era (server/discover got HTTP 400 without a modern error: legacy, the initialize handshake)Profile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST http-notification-202 · not in the profile, not scored passed MUST http-single-endpoint · not in the profile, not scored passed MUST http-session-id-visible-ascii · not in the profile, not scored passed SHOULD http-session-required-400 failed MUST http-origin-invalid-403 · not in the profile, not scored passed control http-origin-localhost-accepted failed advice http-host-rebinding-advice
protocol: Score none = 0 of 0 | outside the score: pending 4, advisories 2, controls 1Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.Exit code: 3.
A server that refuses everything is not protected, only closed. The control is refused too, so the check says it proved nothing:
$ plune mcp conformance --url http://127.0.0.1:58477/mcp --check http-origin-invalid-403 --check http-origin-localhost-acceptedplune mcp conformance · raw 1 · http · protocol 2025-11-25, legacy era (server/discover got HTTP 400 without a modern error: legacy, the initialize handshake)Profile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
could-not-run MUST http-origin-invalid-403 (prerequisite-failed) · not in the profile, not scored failed control http-origin-localhost-accepted
protocol: Score none = 0 of 0 | outside the score: pending 1, controls 1Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.Exit code: 3.
http-bind-loopback checks that a local server does not answer on an outside address of the machine, and
says in its line how each try went; the addresses themselves are not printed.
The OAuth suite
Section titled “The OAuth suite”--suite oauth --url <endpoint> asks an HTTP server the way a client with no credentials would, and
checks the 401 and its WWW-Authenticate challenge, the protected resource metadata, the authorization
server’s metadata, PKCE, HTTPS, iss, client registration, and the tokens the server accepts and refuses.
It runs once for each revision it covers (2025-11-25 and 2026-07-28, or the one --protocol-version
names) and cannot be combined with another suite in one run. A server that answers a request without a
token with 2xx is public: every check is not-applicable (no-auth). No oauth-* check is in the
profile, so a run of this suite always ends with exit code 3; read the list of checks.
Without a config the checks that need a client are could-not-run (needs-input). --config <file> gives
the run its client, as JSON or as YAML (a file whose name ends in .yaml or .yml): the same keys and
rules for both, at most 64 KiB, no anchors, aliases, custom tags or merge keys. A secret is never a value
in the file: registration.clientSecretEnv names the environment variable that holds it, and a file with a
key that names a secret is refused. This is the file the runs below used:
auth: headlessredirectUri: http://127.0.0.1:8765/callbackscopes: [mcp:tools]registration: method: preregistered clientId: plune-test-client clientSecretEnv: MCP_CLIENT_SECRET| Key | Meaning |
|---|---|
auth | How the run gets a code or a token: headless, client-credentials (see below). interactive is accepted by the file but not supported: no browser is opened, and the checks that need a signed-in person are could-not-run (needs-input). |
registration | One object, or a list of objects, each with a method: preregistered (clientId, optional clientSecretEnv), dcr (dynamic registration, off unless allowClientCreation: true), or cimd (clientIdMetadataUrl, clientIdMetadataMismatchUrl). |
redirectUri, scopes, narrowScopes, resource, protocolVersions | The redirect URI the client registered; the scopes asked for; a subset of scopes for the insufficient-scope check, which must differ from scopes; the canonical URI of the server; the revisions to run. |
Sign-in modes. client-credentials is the machine mode: the run asks the token endpoint for a token with
the client’s secret and no person. headless is the mode for CI: it works only against a test environment
whose authorization endpoint answers the request directly with a redirect to your redirect URI that
carries a code, with no sign-in page in between. The redirect is read, never followed. When the endpoint
shows a sign-in instead, the checks that need a code are could-not-run (needs-input), never passed.
The matrix. A registration list runs the suite once for each revision and each method, prints each run
with its own summary and adds a Combined line made from the counters of all of them, not from their
scores. A check that is not-applicable for another method says so:
$ plune mcp conformance --url http://127.0.0.1:63496/mcp --suite oauth --config oauth-config.json --allow-origin http://127.0.0.1:63496 --check oauth-requires-auth --check oauth-reg-prereg --check oauth-reg-cimd --check oauth-authz-response-issplune mcp conformance · a server without serverInfo · http · protocol 2025-11-25, legacy era (the oauth suite runs once for each revision it covers) · registration preregisteredProfile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST oauth-requires-auth · not in the profile, not scored not-applicable MUST oauth-authz-response-iss (era) · not in the profile, not scored passed SHOULD oauth-reg-prereg not-applicable MUST oauth-reg-cimd (not-offered) · not in the profile, not scored
oauth: Score none = 0 of 0 | outside the score: pending 3, advisories 1Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.
plune mcp conformance · a server without serverInfo · http · protocol 2025-11-25, legacy era (the oauth suite runs once for each revision it covers) · registration cimdProfile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST oauth-requires-auth · not in the profile, not scored not-applicable MUST oauth-authz-response-iss (era) · not in the profile, not scored not-applicable SHOULD oauth-reg-prereg (not-offered) not-applicable MUST oauth-reg-cimd (not-offered) · not in the profile, not scored
oauth: Score none = 0 of 0 | outside the score: pending 3, advisories 1Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.
plune mcp conformance · a server without serverInfo · http · protocol 2026-07-28, modern era (the oauth suite runs once for each revision it covers) · registration preregisteredProfile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST oauth-requires-auth · not in the profile, not scored passed MUST oauth-authz-response-iss · not in the profile, not scored passed SHOULD oauth-reg-prereg not-applicable MUST oauth-reg-cimd (not-offered) · not in the profile, not scored
oauth: Score none = 0 of 0 | outside the score: pending 3, advisories 1Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.
plune mcp conformance · a server without serverInfo · http · protocol 2026-07-28, modern era (the oauth suite runs once for each revision it covers) · registration cimdProfile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST oauth-requires-auth · not in the profile, not scored could-not-run MUST oauth-authz-response-iss (needs-input) · not in the profile, not scored not-applicable SHOULD oauth-reg-prereg (not-offered) not-applicable MUST oauth-reg-cimd (not-offered) · not in the profile, not scored
oauth: Score none = 0 of 0 | outside the score: pending 3, advisories 1Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.
Combined over 4 runs: Score none = 0 of 0 | failed 0 | could-not-run 0 (the counters of the runs added up) | incomplete: exit 3.Exit code: 3.
$ plune mcp conformance --url http://127.0.0.1:63496/mcp --suite oauth --protocol-version 2026-07-28 --config oauth.yaml --allow-origin http://127.0.0.1:63496plune mcp conformance · a server without serverInfo · http · protocol 2026-07-28, modern era (the oauth suite runs once for each revision it covers) · registration preregisteredProfile plune-mcp-conformance 2026-11.2 · sha256 466f528105d9
passed MUST oauth-requires-auth · not in the profile, not scored passed MUST oauth-challenge-bearer · not in the profile, not scored passed MUST oauth-prm-reachable · not in the profile, not scored passed MUST oauth-challenge-metadata-url · not in the profile, not scored passed MUST oauth-prm-valid · not in the profile, not scored passed SHOULD oauth-challenge-scope passed MUST oauth-as-metadata-discoverable · not in the profile, not scored passed MUST oauth-as-issuer-matches · not in the profile, not scored not-applicable MUST oauth-as-https (not-offered) · not in the profile, not scored passed MUST oauth-as-pkce-advertised · not in the profile, not scored passed SHOULD oauth-as-pkce-s256 passed SHOULD oauth-as-iss-advertised failed SHOULD oauth-as-cimd-advertised passed MUST oauth-neg-invalid-token-401 · not in the profile, not scored could-not-run MUST oauth-neg-wrong-audience (needs-input) · not in the profile, not scored could-not-run MUST oauth-e2e-token-works (needs-input) · not in the profile, not scored could-not-run SHOULD oauth-neg-insufficient-scope (needs-input) passed MUST oauth-authz-response-iss · not in the profile, not scored passed MUST oauth-neg-pkce-wrong-verifier · not in the profile, not scored passed MUST oauth-neg-token-redirect-mismatch · not in the profile, not scored passed SHOULD oauth-reg-prereg not-applicable MAY oauth-reg-dcr (not-offered) not-applicable MUST oauth-reg-dcr-http-redirect-rejected (not-offered) · not in the profile, not scored not-applicable MUST oauth-reg-cimd (not-offered) · not in the profile, not scored could-not-run MUST oauth-token-format (needs-input) · not in the profile, not scored passed MUST oauth-neg-invalid-client · not in the profile, not scored passed MUST oauth-neg-authorize-redirect-mismatch · not in the profile, not scored
oauth: Score none = 0 of 0 | outside the score: pending 20, advisories 7Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.Exit code: 3.
In that run oauth-reg-prereg passed and oauth-as-cimd-advertised failed, but both are outside the score,
so the exit code is 3. Checks that need auth: client-credentials (oauth-e2e-token-works,
oauth-neg-wrong-audience, oauth-token-format) are could-not-run because the config above does not
ask for it.
Where secrets go
Section titled “Where secrets go”--allow-origin <origin> (repeatable) names an origin the run may ask besides the server’s own. Without it, a
resource_metadata URL or an authorization server on another origin is not asked, and the check is
could-not-run. It is also the only way a client secret leaves the run: the secret goes only to a token
endpoint whose origin is named there, and only over https, or http on a loopback
name. The server’s own origin is no exception: if the server is its own authorization server, its origin must be named
with --allow-origin too before the secret is sent to it. (Reading public metadata from the server’s own origin needs
no flag.) No redirect is followed anywhere. --header and --header-env are not used by this suite. A token, a
code, a verifier, a client secret or an Authorization value is masked in every output, in --out files and
in JUnit.
What the OAuth suite leaves behind
Section titled “What the OAuth suite leaves behind”The probes use the authorization server you point them at, and they do not clean it up:
- Codes and tokens. In one run,
oauth-authz-response-isstakes one authorization code, and the two probesoauth-neg-pkce-wrong-verifierandoauth-neg-token-redirect-mismatchtake one code each plus one shared control code and its token: at most 4 codes and 1 control token, and up to 3 tokens if both probes fail. Over both revisions that is up to 7 codes and 6 token requests (the older revision has noisscheck). Only a run with thepreregisteredmethod asks for codes, so a registration list does not multiply them. The other client-credentials checks issue tokens too (for example up to two foroauth-neg-insufficient-scope). No token is revoked; the codes expire on the server. - Clients. Dynamic registration creates a client on the authorization server, and is off unless
allowClientCreationistrue. After each registration the run deletes the client (DELETEon its registration URI, RFC 7592) when the server returned what that needs. When it could not, the result says so in plain words and names the client (Client dyn-1 remains on the authorization server), or says that a client may have been created and could not be named. One run makes at most 2 clients per revision, 4 over both, namedplune-conformance-test. A run interrupted with Ctrl-C between a registration and its deletion leaves the client and says nothing.
Nothing guarantees that no client or token is left. Use a test environment.
The Tasks and MCP Apps suites
Section titled “The Tasks and MCP Apps suites”Two suites check the extensions a server may offer: --suite tasks the Tasks extension
(io.modelcontextprotocol/tasks, 13 checks) and --suite apps MCP Apps (io.modelcontextprotocol/ui, 8 checks).
Both run alone or together with --suite protocol; neither runs together with --suite oauth. Every check of a suite is
not-applicable on a server that does not offer its extension: for Tasks, one that does not declare it, and any session
older than 2026-07-28; for Apps, one that declares no io.modelcontextprotocol/ui and has no tool with _meta.ui. No check of either suite is in the profile, so
a run of one of them alone always ends with exit code 3, even when a check failed: read the list of checks.
Both call a tool only when --fixture names it. The Tasks checks need a task from that tool, so without a fixture every
one is could-not-run (needs-input); a run makes at most 3 tasks and cancels the ones it did not see finish. The Apps
suite only reads the ui:// resources a tool or the list names, never runs their HTML, and calls a tool only for
apps-ui-tool-text-fallback.
These two outputs were captured with CLI 0.39.3 on 19 November 2026, against the test servers of the CLI’s repository,
each with a fixture.json that names the one tool to call:
$ plune mcp conformance --suite tasks --fixture fixture.json -- node tasks-server.mjs goodplune mcp conformance · tasks-good 1.0.0 · stdio · protocol 2026-07-28, modern era (server/discover answered a DiscoverResult)Profile plune-mcp-conformance 2026-11.3 · sha256 fd68f7f5dc2e
passed MUST tasks-result-type · not in the profile, not scored passed MUST tasks-undeclared-no-task · not in the profile, not scored passed MUST tasks-undeclared-capability-error · not in the profile, not scored passed MUST tasks-get-status-shape · not in the profile, not scored passed MUST tasks-status-payload · not in the profile, not scored passed SHOULD tasks-ttl-poll-integers passed MUST tasks-unknown-id-get · not in the profile, not scored passed SHOULD tasks-unknown-id-update-cancel passed MUST tasks-cancel-ack · not in the profile, not scored passed MUST tasks-update-ack · not in the profile, not scored not-applicable SHOULD tasks-failed-status-message (not-offered) not-applicable SHOULD tasks-http-name-header (transport) passed MUST tasks-input-request-keys-unique · not in the profile, not scored
tasks: Score none = 0 of 0 | outside the score: pending 9, advisories 4Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.Exit code: 3.
$ plune mcp conformance --suite apps --fixture fixture.json -- node apps-server.mjs goodplune mcp conformance · apps-good 1.0.0 · stdio · protocol 2025-11-25, legacy era (server/discover answered error -32601, not a modern one: legacy, the initialize handshake)Profile plune-mcp-conformance 2026-11.3 · sha256 fd68f7f5dc2e
passed MUST apps-ui-resource-uri-scheme · not in the profile, not scored passed MUST apps-ui-resource-exists · not in the profile, not scored passed MUST apps-ui-mime-type · not in the profile, not scored passed MUST apps-ui-content-payload · not in the profile, not scored passed SHOULD apps-ui-listed-mime passed SHOULD apps-ui-visibility-values passed MUST apps-ui-tool-text-fallback · not in the profile, not scored passed SHOULD apps-ui-meta-shapes
apps: Score none = 0 of 0 | outside the score: pending 5, advisories 3Verdict: incomplete (exit 3): nothing was measured, so nothing is proven.Exit code: 3.
The exit code is the gate. Exit 1 is a failed scored check, 2 a server that could not be tested or a wrong
flag, and 3 “not established”: a scored check could not run, or nothing was measured. A job that must stay
green until a profile counts more checks can let 3 through and fail on 1 and 2:
# .github/workflows/mcp-conformance.yml — a stdio server built in the jobname: MCP conformanceon: pull_request:jobs: conformance: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 22 - run: npm ci && npm run build - name: Check the server against the spec run: | code=0 npx -y @plune-ai/cli mcp conformance --out junit-xml=conformance.xml -- node dist/server.js || code=$? # 0 passed, 3 not established: let both through; 1 a failed check and 2 a wrong run fail the job if [ "$code" -eq 1 ] || [ "$code" -eq 2 ]; then exit "$code"; fi - uses: actions/upload-artifact@v4 if: always() with: name: mcp-conformance path: conformance.xmlFor a server that runs over HTTP, start it in the job, wait until it answers, and give the command its URL:
- name: Start the server run: node dist/server.js & - name: Wait for it run: | for i in $(seq 1 30); do curl -s -o /dev/null http://127.0.0.1:3000/mcp && break; sleep 1; done - name: Check the server against the spec run: | code=0 npx -y @plune-ai/cli mcp conformance --url http://127.0.0.1:3000/mcp --out junit-xml=conformance.xml || code=$? if [ "$code" -eq 1 ] || [ "$code" -eq 2 ]; then exit "$code"; fiBoth workflows were run with CLI 0.39.3 on 19 November 2026 against the test servers of a sandbox repository, the HTTP
one starting its HTTP server instead of dist/server.js: the stdio run ended with exit 3 and a green job
(https://github.com/plune-ai/mcp-action-sandbox/actions/runs/44590443548), the HTTP run with exit 1 from three
failed checks and a red job (https://github.com/plune-ai/mcp-action-sandbox/actions/runs/44590443685). If you want 3 to fail the job too, drop the if and keep the exit code as it is.
--out junit-xml=<file> writes one <testcase> per selected check, named mcp.conformance.<suite> and the check id,
so the same id is the same case on every run. A passed check is an empty case; a could-not-run is an <error>, never a
pass, even when it leaves the exit code alone:
$ plune mcp conformance --check life-initialize-result --check tools-structured-conformant --out junit-xml=conf.xml -- node server.mjs good<?xml version="1.0" encoding="UTF-8"?><testsuites name="plune mcp conformance" tests="2" failures="0" errors="1" skipped="0"> <testsuite name="plune mcp conformance" tests="2" failures="0" errors="1" skipped="0"> <testcase classname="mcp.conformance.protocol" name="life-initialize-result"/> <testcase classname="mcp.conformance.protocol" name="tools-structured-conformant"> <error type="could-not-run" message="could-not-run: needs-input: No --fixture: no tool is called without one. Name a safe tool and its arguments in a fixture file to run this check.">needs-input: No --fixture: no tool is called without one. Name a safe tool and its arguments in a fixture file to run this check.expected: The structuredContent of a call (from the fixture) conforms to the tool's outputSchema.actual: No --fixture: no tool is called without one. Name a safe tool and its arguments in a fixture file to run this check.fix:spec: https://modelcontextprotocol.io/specification/2025-11-25/server/tools (Output Schema)</error> </testcase> </testsuite></testsuites>Exit code: 0.
--format json gives the whole result. Two of its parts, for a run of three checks: the first entry of
checks, and the summary (an object assembled from those two parts, not the whole file):
$ plune mcp conformance --format json --check life-initialize-result --check tools-structured-conformant --check tools-name-format --out json=result.json -- node server.mjs good{ "checks[0]": { "id": "life-initialize-result", "suite": "protocol", "kind": "check", "class": "deterministic", "level": "MUST", "state": "passed", "bucket": "passed", "expected": "The result of initialize holds the protocol version the server chose, its capabilities and its serverInfo.", "actual": "protocolVersion 2025-11-25, capabilities {tools, resources, prompts, logging, completions}, serverInfo conformance-good 1.0.0.", "fix": "", "specRef": "https://modelcontextprotocol.io/specification/2025-11-25/basic/lifecycle (Initialization)" }, "summary": { "passed": [ "life-initialize-result" ], "failed": [], "couldNotRun": [], "needsInput": [ "tools-structured-conformant" ], "notApplicable": [], "pending": [], "advisories": [ "tools-name-format" ], "controls": [], "score": 100, "applicable": 1, "verdict": "passed" }}Exit code: 0.
What the suite does not prove
Section titled “What the suite does not prove”- It does not make a server secure.
passedsays the server answered the suite’s requests as the text asks; it says nothing of authorization beyond the OAuth checks, of input handling, of what the tools do, or of load. - It checks the specification, not whether your tools are good. For what in a tool list makes a model choose the wrong tool, see Lint an MCP server; for what changed in a contract, see Snapshot and diff; for the basics, Test an MCP server.
- It does not test with a model or an agent. Whether an agent can use the server is a different question.
- A level is the reading of one text.
MUSTis used only where the cited section says so, and where it was derived the entry says so; where the text leaves a choice,--explain <id>says which reading the check took. - Most checks do not count yet. The profile lists 31 of 132, so a green exit code covers those 31.