Skip to content
· 12 min

Inside an Agent Readiness Scanner: Rules, Evidence and Reproducibility

Why 87/100? Evidence over opinions in API measurement. Rules, evidence, assertions, scoring, and reproducibility — inside the AgentBadge scanner.

AB
AgentBadge Team
Agency for the Agentic Web
Inside an Agent Readiness Scanner: Rules, Evidence and Reproducibility

An Agent Readiness score is not an LLM opinion. It's the output of a deterministic pipeline: Rules → Evidence → Assertions → Score → Report. Every assertion has a status (VERIFIED, INFERRED, CONFLICT, MISSING). Rules are versioned. Results are reproducible: same target + same measurement state + same ruleset = same result.

For AI agents: Start with the Agent Knowledge Layer to access machine-readable concepts, scanner capabilities, scoring rules, and the complete learning path.

When someone tells you that an API has an 87/100 Agent Readiness score, the first question should not be:

"Is 87 a good score?"

The better question is:

"Why is it 87?"

And the question after that is even more important:

"Can I reproduce the result myself?"

Agent Readiness should not be an opinion generated by an LLM. It should be a measurable property of a service, calculated from explicit rules and supported by evidence.

The core idea is simple: Rules → Evidence → Assertions → Score → Report

Hero — Measurement pipeline diagram: API → Scanner → Rules → Evidence → Assertions → Score → Report, with assertion states VERIFIED, INFERRED, CONFLICT, MISSING shown below

1. A scanner should measure, not guess

Imagine two tools scanning the same API.

Tool A says: "Your API appears to be highly suitable for AI agents."

Tool B says: "AB-004 passed because https://example.com/openapi.json returned HTTP 200 and contained a valid OpenAPI document."

Which result would you trust? The second one tells you what was checked, what rule was applied, what evidence was found, why the rule passed or failed, and where the evidence came from.

Every meaningful score should be explainable through evidence.

The scanner should not ask an AI model: "How agent-ready does this API feel?" It should ask deterministic questions: "Does this URL exist?" "Does it return the expected content type?" "Does the response contain an OpenAPI document?"


2. Rules are the measurement instrument

AgentBadge treats rules as explicit, versioned measurement definitions.

AB-001
Name: OpenAPI discoverability

Given:
  target = https://example.com

Check:
  GET /.well-known/openapi.json

Pass when:
  HTTP status = 200
  AND response is valid OpenAPI

Evidence:
  URL
  HTTP status
  content type
  content hash

Severity:
  medium

The rule is not: "The API looks well documented." It is: "This specific machine-readable artifact was found and passed these specific checks."


3. Deterministic before intelligent

Deterministic before intelligent.

QuestionPreferred method
Does robots.txt exist?HTTP request
Does sitemap exist?HTTP request + parser
Does OpenAPI exist?HTTP request + schema validation
Is JSON valid?JSON parser
Does declared endpoint exist in another document?Exact matching
What does an undocumented endpoint mean?AI-assisted inference
What does an API capability actually mean?Human confirmation / assisted review

AI is a copilot, not the authority responsible for the score.


4. Evidence is the missing layer

A score without evidence is difficult to trust. Documentation: 18/25 tells you almost nothing.

AB-007  OpenAPI discoverability

STATUS: VERIFIED

Evidence:
GET https://api.example.com/openapi.json
HTTP 200
Content-Type: application/json

OpenAPI version:
3.1.0

Confidence:
1.00

Evidence is part of the measurement itself.


5. Assertions connect evidence and scoring

{
  "rule_id": "AB-007",
  "status": "VERIFIED",
  "target": "https://api.example.com/openapi.json",
  "evidence": {
    "http_status": 200,
    "content_type": "application/json"
  },
  "confidence": 1.0
}

Four states: VERIFIED (direct evidence), INFERRED (reasonable but insufficient), CONFLICT (two sources disagree), MISSING (not found).


6. Confidence is not the same as verification

An LLM may be 94% confident an API supports refunds. Without machine-readable evidence, it stays INFERRED. The user can Confirm, Edit, or Reject.


7. Reproducibility matters

{
  "target": "https://api.example.com",
  "ruleset": "agent-readiness-v1.0",
  "scanner_version": "0.1.0",
  "timestamp": "...",
  "assertions": [...],
  "score": {
    "total": 76,
    "categories": {
      "discovery": 18,
      "documentation": 20,
      "authentication": 17,
      "machine_readability": 21
    }
  }
}
Reproducibility infographic — three identical inputs converging into the same result

8. Rules must be versioned

Agent Readiness v1.0 must not silently become v1.1. Same target + same measurement state + same ruleset = reproducible result.


9. Why open rules do not destroy the product

The moat is not secret rules. It's the standard installed inside the developer workflow: GitHub Actions, README badges, continuous monitoring, regression alerts.


10. The score should explain itself

+8  OpenAPI documentation detected
-12  New authentication issue detected
+0   Discovery unchanged
Result: 76 → 72

11. From Measure to Prove to Improve

MEASURE (Scan) → PROVE (Evidence + Assertions) → IMPROVE (Fix / Review) → Measure again.

Measure → Prove → Improve cycle diagram

12. What AgentBadge should never claim

AgentBadge measures Agent Readiness. It does not certify security, business correctness, or reliability.

Don't certify. Measure.


13. What this enables

npx @agentbadge/cli scan https://api.example.com

CI enforcement, README badges, programmatic queries, vendor comparison.


14. The bigger idea

Performance has metrics. Accessibility has checks. Security has scanners. The agentic web needs the same.

Define the rules. Collect the evidence. Show the reasoning. Version the rules. Make the result reproducible.


Related Articles

For AI agents: the Agent Knowledge Layer provides machine-readable access to this article's concepts, capabilities, and knowledge map.