Verify a Model

You do not have to trust that anthropic/claude-opus-5 on OSMEXO is really Claude Opus 5. Run the same battery of prompts against the vendor's own API with your vendor key and against OSMEXO with your OSMEXO key, on your machine, and read the score. It is the same check OSMEXO's operators run before approving a route, so the prompts, weights and scoring are identical.

Get the command from the dashboard

Open the model in the dashboard and click Verify with your own key. Pick the vendor you hold a key for; the dashboard fills in the vendor's own model id for you (our slugs and the vendors' ids differ in small ways, claude-opus-4.5 vs claude-opus-4-5). Copy the two lines it shows.

Or write it yourself

bash

curl -fsSLO https://dashboard.osmexo.com/verify.mjs
OSMEXO_API_KEY=sk-... VENDOR_API_KEY=sk-ant-... node verify.mjs \
  --model anthropic/claude-opus-5 --vendor anthropic --vendor-model claude-opus-5

Node 18 or newer, no install. One OpenRouter key can verify any text model:

bash

OSMEXO_API_KEY=sk-... VENDOR_API_KEY=sk-or-... node verify.mjs --model deepseek/deepseek-v3.2 --vendor openrouter

Option	Meaning
`--model`	The OSMEXO model id.
`--vendor`	`anthropic`, `openai`, `deepseek`, `openrouter`, `google` or `openai_compatible`.
`--vendor-model`	The vendor's own id. Required unless the vendor is `openrouter` or `openai_compatible`, which take the OSMEXO id as is.
`--base-url`	API root of an `openai_compatible` vendor, e.g. `https://api.together.xyz/v1`.
`--full`	9 prompts instead of 6 (six exact tasks instead of three). About 3.5k tokens per side.
`--embedding`	For embedding models: one call per side, cosine similarity.
`--json`	Machine-readable output.

Keys are read from the environment only, never from arguments, so they do not land in your shell history. VENDOR_API_KEY is sent to the vendor and nowhere else; OSMEXO_API_KEY is sent to OSMEXO and nowhere else. The script is one dependency-free file; read it before you run it.

What it tests

Test	Weight	How it is scored
Tokenizer fingerprint	40 %	Share of prompts where both sides report the same input token count (±2). A different model, or a hidden prompt injected by a reseller, changes it. A fixed extra prefix on the vendor side (a reference host's system prompt) is recognised and ignored, and so is a fixed prefix of at most 8 tokens on the route side (a chat template's own wrapper). An injected prompt is dozens of tokens and varies, so it still fails. This is the strongest signal.
Knowledge cutoff	20 %	Asked for its training cutoff month and its maker. Cutoffs within three months and the same maker score 1; within a year, or no date named, scores ½. Makers match on the first word, so "Alibaba Cloud" and "Alibaba" agree.
Hidden system prompt	20 %	Both sides asked to quote any operator instructions. Both `NONE`, or both quoting the same default prompt, scores 1; a refusal is uncertain (½); our route quoting a prompt the vendor does not have scores 0, unless token counts agree on every prompt, which proves nothing was injected and the quote is invented (½). The vendor side quoting a prompt our route lacks is noise on the reference side (½): some OpenRouter hosts add their own instructions.
Exact-answer tasks	20 %	Same answer scores 1. One side slipping scores ½: a random typo is not evidence of a different model, so a single slip costs little.

Nothing in the battery depends on random sampling. Earlier versions scored writing style and a "pick a random number" spread; on identical models those swung the score by 10–20 points, so they were removed.

Score 85 or more: the real model. OSMEXO answers exactly like the vendor's own API, and you can rely on the route. 60–84: not confirmed. The two sides disagree on some tests, which a short battery can do on identical models; run again with --full, and tell us if it stays there. Under 60: might be a different model. Send the output to support@osmexo.com and we will investigate the route.

The report also shows how many output tokens OSMEXO billed compared with the vendor for the same answers. Reasoning models spend hidden tokens before the visible answer; a ratio well above 1 means the route bills for reasoning the vendor does not, and is worth a question.

Reasoning models

A reasoning model can spend the whole token budget thinking and return nothing visible. When a side comes back empty with the budget used up, the script asks that prompt once more with four times the room. A prompt that is still empty on either side is excluded rather than scored, and the report says so. If more than a third of the prompts end up excluded, the comparison is not trustworthy and the script stops with exit code 3 instead of a score.

Cost

The quick battery is about 2,000 tokens on each side: a few cents on most models, under ten cents on the largest. The vendor side spends your vendor key; the OSMEXO side spends your OSMEXO credits at the model's normal price and appears in your usage like any other request.

Exit codes

0 the real model · 1 not confirmed · 2 might be a different model · 3 the comparison could not run or could not be scored (wrong model id, rejected key, route down, more than a third of the prompts excluded) · 4 bad arguments. Handy in CI.