How the audit works

What we compare with

Three sources, each of which settles a different question. Every one of them is a federal record or the plan’s own file, and none of them is our opinion. They do not share a cadence, and a check is only as fresh as the file behind it.

We copy each registry to a dated, immutable snapshot before a run uses it, so a finding can be reproduced against the exact file it was measured against.

About 78,000 of the exclusion list’s rows carry no NPI. We drop them before matching rather than matching on a name. That limits how much we find. It does not weaken anything we assert.

How a finding is counted

A rate is only as good as what it is divided by, so the unit is stated before the number.

1. The unit is the published directory, not the issuer. One file can carry as many as 24 registrations, so a finding attaches to the file that carries it.
2. One record is one finding. A record listed under many plan identifiers is still one record. Counting plan associations instead would inflate every rate by whatever a payer chose as its plan structure, which is not a property of directory quality.
3. Records and providers do not scale into one another. The ratio between them runs from 1.0 to 13.9 across directories, so both are published and neither is derived from the other.
4. The rate is records over records. A directory’s rate is the records carrying a qualified finding, divided by the records evaluated in that directory, expressed per 10,000. The Index for the whole panel is the same arithmetic at once: qualified records over evaluated records, not an average of the per-directory rates.
5. Nothing is ranked until the directory was read in full. Zero over zero is not a clean directory, it is an unmeasured one.

Fault and drift

One crawl produces two measurements. A federal fact older than the file’s own edition is the plan’s fault: it published an edition after the fact existed and carried the row anyway. A fact newer than the edition is drift, still wrong in the directory members read today, but not something the plan had an edition in which to fix.

Rankings use drift. A directory that stopped republishing accumulates less fault by construction, so ranking on fault would put the worst-maintained directories at the better end of the table. A finding that carries no federal date is neither, and is never counted as drift by default.

Where 346 and 183 both come from

The CMS machine-readable URL file for 2026 lists 346 rows, one per issuer per state, and the widely quoted figure for Marketplace issuers is 183. Both are right, and 346 is a strict superset. CMS’s own plan attributes file for the same year divides our 346 exactly: 183 medical issuers in the Individual market, 162 stand-alone dental, and one medical issuer that files only into SHOP. There is no residual.

A HIOS issuer ID is a registration rather than a company: Delta Dental holds 24 of them across 22 states, one company that CMS regulates twenty-four times. We derived 183 by applying CMS’s own classification, and we have not located the CMS publication that states it directly, so it stays a derivation rather than a quotation. Accessibility findings are counted per directory index, and the other checks per record, so the counts are not meant to be added together.

Review and response

A plan sees a finding about its own directory before anyone else does. We are reading a file from the outside, and reading from the outside is exactly the position that can be wrong about something the plan can see and we cannot.

1. Notice goes to subscribers. A plan that subscribes at findings@rollcall.health receives every finding that names it, with the evidence, when it publishes, and can respond on the same thread. Plans that have not subscribed are not privately noticed. For them, publication is the notice.
2. Publication follows the run, not a notice window. A verified, reviewed run publishes on its own cadence; nothing is held back waiting for a reply, and silence is never published as agreement.
3. Silence is not agreement. A finding nobody answered is published as unanswered. We never describe a plan that did not reply as having confirmed anything.
4. A finding can be disputed at any point, including after it is published. There is no window on that. We read the evidence when it arrives, and if it shows the finding is wrong we withdraw it and publish the correction in the same place, dated like everything else here.
5. A disputed finding stays visible and stays flagged. It is marked as disputed, with the date, until it is either withdrawn or restated. Where a plan wants its response published, it goes up in the plan’s own words or not at all.

One case we cannot settle: when the plan’s argument is that the federal record itself is wrong, the correction has to happen at NPPES or at HHS-OIG, because those files are not ours to edit. We re-read both every run, so the finding clears on its own once the federal record does. Findings and evidence both go to findings@rollcall.health.

The tests cannot change silently

A measurement whose tests drift is not a measurement. The method on this page is frozen per version, and the system that runs it has no way to alter it quietly.

• Constants are frozen per method version. The baseline, the dispersion adjustment and the eligibility floor are fixed when a version ships. A directory's band moves only when its own rate moves, never because the yardstick moved.
• A method change is a new series, never a restatement. Re-scoring is published as a parallel version over the same stored observations, with a visible break. History is never rewritten under the old name.
• A new check is never backfilled. A check added this year cannot create findings in last year's runs. What was not measured stays unmeasured.
• Machines run the tests; people change them. Agents operate the weekly pass and draft its review. A change to what is tested, and the decision to publish, are made by a person and recorded.