Local CLI that verifies citations in agent / deep-research Markdown reports · github.com/larrylot/citeguard
citeguard check report.md --json returns deterministic per-URL verdicts: resolve, title soft-match, optional claim–source overlap. No API keys. No telemetry. Offline --fixtures mode for CI.
pip install -e . citeguard check fixtures/mixed.md --fixtures --json
Top-level object from ReportResult.to_dict():
{
"path": "fixtures/mixed.md",
"summary": {
"clean": 3,
"dead": 1,
"title_mismatch": 1,
"total": 5
},
"citations": [
{
"url": "https://docs.github.com/en/actions",
"link_text": "GitHub Actions documentation",
"kind": "link",
"line": 3,
"verdict": "clean",
"status": 200,
"final_url": "https://docs.github.com/en/actions",
"page_title": "GitHub Actions documentation - GitHub Docs",
"title_score": 1.0,
"overlap_score": 0.8,
"claim": "GitHub Actions automates workflows in your repository",
"error": null,
"reasons": ["resolved; title/claim checks passed"]
},
{
"url": "https://example.invalid/dead-citation-404",
"link_text": "Vanished paper",
"kind": "link",
"line": 7,
"verdict": "dead",
"status": 404,
"final_url": "https://example.invalid/dead-citation-404",
"page_title": null,
"title_score": null,
"overlap_score": null,
"claim": "A fabricated cite points here: .",
"error": "http_404",
"reasons": ["http_404"]
}
]
}
| verdict | meaning |
|---|---|
| clean | Resolved; title/claim checks passed |
| dead | 404 or network failure |
| http_error | Non-success HTTP (e.g. 500) |
| title_mismatch | Link text soft-match vs <title> too low |
| claim_weak | Claim sentence tokens barely appear in page text |
| redirect_suspect | Cross-host redirect without strong title match |
| unresolved | URL missing from fixtures catalog (fixtures mode only) |
| field | notes |
|---|---|
summary | Counts per verdict + total |
citations[].kind | link | footnote | bare |
title_score | 0–1 soft overlap of link text vs page title |
overlap_score | 0–1 claim vs page body token overlap (optional; disable with --no-overlap) |
reasons | Human-readable explanation list |
Honest comparison (FACT / ASSUMPTION labeled): docs/vs-alternatives.md — vs “ask ChatGPT”, LinkChecker, html-proofer, ReportBench and related benches.
Planted CI fixtures: fixtures/. Broader failure corpus: corpus/ (Citation Failure Corpus).