How evidence is read, checked and read again

Models research, extract and parse. A deterministic engine ranks. Every quote passes an adversarial refute pass.

• Research, extraction, parsing. Language models read the sources, copy quotes character for character with the address and the day they were read, and propose a value or a score on the category's published band ladder.
• Ranking. The order is computed by the engine from the published criteria, weights and scores. The build rebuilds every published score from rankings.json alone and fails when one does not match.
• Refute pass. The last sentence of the policy is the rule for every quote. The records below show how much of the catalog can be checked against it today: 120 of 121 categories.

1. Research pass. A model reads primary sources: the provider's own pages, documents and contracts, and public onchain or market data. Each quote is recorded word for word with its address and read date, and each score is placed on the category's band ladder. The result is a draft. Nothing is published from a draft.
2. Adversarial refute pass. A separate run starts from the assumption that each quote and score is wrong. It fetches the page again, checks that the quote is there word for word, checks that the quote supports the value, and checks the score against the band ladder, not only against the address. A value that cannot be refuted stands; otherwise it is corrected or marked as not verifiable.
3. Second verify pass. Research that has passed is verified a second time before it is relied on. The files keep the draft and the final verified result, not each pass separately, so the counts below compare the draft with the final result.
4. Re-read on a cadence. The published cadence is to read every category's evidence again within 45 days. The oldest read date among a category's rows decides. A category counts as overdue 7 days after that, and the build fails when any category is overdue. A re-read runs the refute pass again against the current page.
5. Who approves. Methodology versions are approved by Mathias Siemonsmeier, who is named on each entry in the methodology changelog. No external reviewer signs a category today.

Overdue: DAO Governance Tools (oldest read 2026-08-06, due 2026-09-27), Market-Data APIs (oldest read 2026-08-06, due 2026-09-27), NFT Lending (oldest read 2026-08-06, due 2026-09-27), Token Screeners (oldest read 2026-08-06, due 2026-09-27), Seed Phrase Backup (oldest read 2026-08-06, due 2026-09-27), Shared Sequencers (oldest read 2026-08-06, due 2026-09-27), Tokenised Commodities (oldest read 2026-08-06, due 2026-09-27), Wrapped Assets (oldest read 2026-08-06, due 2026-09-27), Sentiment Analytics (oldest read 2026-08-07, due 2026-09-28), Relayers (oldest read 2026-08-11, due 2026-10-02), Incentive Distribution (oldest read 2026-08-17, due 2026-10-08), Wallets (oldest read 2026-08-17, due 2026-10-08), Exchanges (oldest read 2026-08-18, due 2026-10-09). Carrying no read date, so the cadence cannot be measured: Payment Gateways. Next due: 2026-11-04, 1 category.

Evidence age on 2026-11-04 (the day this page was built), over every placement with a read date: median 37 days, upper decile from 51 days, oldest 90 days; 803 of 1005 within their category's 45-day cadence. 13 placements carry no read date and are counted apart, never as new. The build archives the same figures daily as of its own day (evidenceAge in /data/kpi-snapshot.json), so the trend is public.

A caught error here is a criterion score the verify pass changed. Evidence text and pricing text are not counted: the verify pass rewrites them to record its own fetch, so a changed sentence is not by itself a caught error. Not counted: travel_rule_compliance-draft.json (no verified file beside it).

A correction can be a reading error or a change on the provider's side since the last read. The files do not say which, so neither is claimed. 1 file also records whether the research pass's quote was found word for word when the page was fetched again: it was not in 10 of 317 (3.2%) fields, counting fields where the research pass gave no quote.

120 of 121 categories have a committed draft and verified file (27) or a committed re-read (120). Every rate on this page describes those files only. It is not an estimate for the other 1, which have neither file committed; their counts are published when their re-read lands.

Some events cannot wait for the calendar. Any one of these opens an extraordinary review of the affected placements:

• Regulator enforcement. A published enforcement action by a regulator that names the provider.
• Withdrawal halt. Customer withdrawals halted for more than 24 hours.
• Exploit. A loss from an exploit of more than $1M, or of more than 5% of the protocol’s total value locked.
• Missed proof of reserves. A proof-of-reserves period the provider publishes passes without its attestation.

• The checks that look for these events are a watcher that reads regulator feeds and DefiLlama’s hacks list every 6 hours while its scheduler runs, and the incident census, which every build holds against the hacks list as read at least once a quarter and which fails the build when an exploit on a ranked placement has no review. A person confirms each event against its source before a review opens, and it is logged below with the page that reported it.
• The affected placements are re-read against their sources within 72 hours.
• Until the re-read is done, the review is listed on this page with its label, the time it was opened and the time the re-read is due.
• The label never changes a score or a position. If the re-read changes evidence, the change is published in the data changelog like any other, and an order that moves is recomputed from the new evidence by the same engine.

• near_intents in Intent Networks: re-read on 2026-10-20. All nine quoted passages behind its four published criteria still read word for word on its documentation on 2026-10-20, so no score moves. None of the four criteria reads contract or bridge security, so the $3.8M loss on 1 October falls outside what intents ranks; the post-failure note on /methodology/calibration says so. Opened 2026-10-20T16:55:00Z, due 2026-10-23T16:55:00.000Z. Report · Its rank that day
• ledger in Hardware Wallets: re-read on 2026-10-20. Re-read refute-by-default, because Ledger is ChainChoice’s only live affiliate partner: no published criterion’s evidence changes. Ledger says the implant sits inside the enclosure, reads the 24 words off the screen at setup and does not modify the secure element, firmware or signing, and independent teardowns (Tibane Labs, Mark Karpelès) describe it reading the display, not the chip, so the losses ($93M on DefiLlama’s hacks list, a total Ledger has not verified) are not a device-level fund loss on the ladder, and Genuine Check is not bypassed: Ledger documents it to reject counterfeits and says it ‘cannot detect unauthorized physical modifications to the hardware, such as spying implants’. Where the units were opened is not established; they were sold through CryptoBilis, a reseller on Ledger’s own official-reseller list whose sales are now paused, and no report names the Nano S Plus. Opened 2026-10-20T16:55:00Z, due 2026-10-23T16:55:00.000Z. Report · Its rank that day
• ledger-nano-x in Hardware Wallets: re-read on 2026-10-20. Re-read refute-by-default, because Ledger is ChainChoice’s only live affiliate partner: no published criterion’s evidence changes. Ledger says the implant sits inside the enclosure, reads the 24 words off the screen at setup and does not modify the secure element, firmware or signing, and independent teardowns (Tibane Labs, Mark Karpelès) describe it reading the display, not the chip, so the losses ($93M on DefiLlama’s hacks list, a total Ledger has not verified) are not a device-level fund loss on the ladder, and Genuine Check is not bypassed: Ledger documents it to reject counterfeits and says it ‘cannot detect unauthorized physical modifications to the hardware, such as spying implants’. Where the units were opened is not established; they were sold through CryptoBilis, a reseller on Ledger’s own official-reseller list whose sales are now paused, and Tibane Labs found a similar implant in Nano X units bought through Japanese resale channels, with no loss linked to them. Opened 2026-10-20T16:55:00Z, due 2026-10-23T16:55:00.000Z. Report · Its rank that day

• Correction log: published mistakes, what changed and how each was caught.
• Data changelog: every observed change to a published fact, beside the value it replaced.
• Methodology changelog: every change to criteria, weights and constants.
• Review calendar: each category's oldest read date and due date, as the build computes them.
• Review calendar for your calendar app (.ics): every category's next review as an all-day event, from the same dates.