Why automated scanners only catch 30–40% of accessibility issues

Every accessibility tool vendor — including us — quotes that 30-40% number. Here is the actual research behind it, what automated tools genuinely do well, and what they will never catch.

Every accessibility tool vendor cites the same statistic: automated scanners catch about 30-40% of WCAG issues. Plumb, axe DevTools, WAVE, Siteimprove, SortSite — all of them. Where does that number come from, and what does it actually mean for your audit strategy?

The original research

The most-cited source is research from Deque Systems (the makers of axe-core), who analyzed which WCAG success criteria are amenable to programmatic detection vs. which require human judgment. Their published number: roughly 30-40% of WCAG 2.0 failures can be reliably detected by automated tools, with the remainder requiring manual evaluation.

Subsequent academic work — including the WebAIM Million annual analysis — has produced consistent results: automated scanners detect well-defined, programmatically-checkable failures with very high accuracy (low false-positive rate is the trade-off for low recall on judgment criteria), and they cannot detect issues that depend on whether content is “meaningful,” “descriptive,” or “logical.”

What automated tools actually do well

These categories of WCAG failure are reliably detected by axe-core and equivalent rules engines:

• Color contrast (1.4.3, 1.4.11) — the math is unambiguous; tools compute it perfectly.
• Missing or empty alt attributes (1.1.1) — tools detect the absence; they cannot evaluate quality.
• Form labels (1.3.1, 3.3.2) — programmatic association between <label> and <input>.
• Heading structure (1.3.1) — sequential hierarchy without gaps.
• ARIA validity (4.1.2) — invalid roles, conflicting attributes, required-children violations.
• Document language (3.1.1) — presence of the lang attribute.
• Duplicate IDs (4.1.1, removed in 2.2 but historically significant).
• Tab order traps (2.1.2) — focusable elements that the user can't escape.

These checks have very low false-positive rates — when axe-core flags a contrast issue, it is almost certainly a real failure.

What automated tools cannot detect

These categories require human judgment and are where the missing 60-70% lives:

• Alt-text quality. A tool sees that an alt attribute exists. It cannot tell whether alt="image" describes the image meaningfully (it doesn't), whether alt="Logo of Acme Corporation" is appropriate context (depends on the surrounding content), or whether the alt text helps a screen reader user accomplish their task.
• Link purpose from text (2.4.4). “Read more” and “Click here” pass automated checks (the link has text). Whether they describe the destination is a judgment call.
• Heading clarity. Tools verify hierarchy. They cannot evaluate whether the heading text actually summarizes the section.
• Reading order. The DOM order may differ from the visual order (CSS grid, flexbox order property). Whether the actual reading order is logical is a judgment call.
• Information conveyed by color alone (1.4.1). Tools detect contrast. They cannot determine whether a chart relies on red/green differentiation that color-blind users will miss.
• Cognitive load and language clarity (3.1.5). Tools cannot evaluate whether your jargon-heavy form instructions are understandable.
• Audio/video accuracy (1.2.x). Tools detect that captions exist. They cannot evaluate whether the captions are accurate or whether audio descriptions sufficiently describe visual content.
• Custom widget behavior. Tools verify ARIA attribute presence on a custom dropdown. They cannot verify that the dropdown actually announces state changes correctly to a screen reader.

Why “30-40%” under-counts what scanners do for you

The 30-40% number measures coverage of the WCAG criteria. But automated tools also produce most of the volume of findings on a typical site. A 50-page e-commerce site might have:

• 200 instances of insufficient contrast (one button style repeated across the site).
• 40 missing alt attributes (decorative icons that should be empty alt).
• 15 form fields without programmatic labels.
• 3 judgment issues — bad alt text, vague link purpose, color-only state.

Automated tools find the 255 high-volume programmatic issues in 30 seconds. A manual auditor would still find the 3 judgment issues, but they'd also find the same 255 — and bill 40 hours doing it. The right combination is automated tools for breadth and a manual auditor for depth.

How AI augmentation closes part of the gap

Modern multimodal AI models with vision and content understanding can detect a meaningful subset of judgment issues that traditional scanners cannot:

• Alt-text quality — the AI reads the image and the alt text and compares.
• Link purpose — the AI reads link text in context and flags vague phrasing.
• Heading clarity — the AI evaluates whether headings summarize their sections.
• Color-only information — the AI inspects screenshots and identifies information conveyed solely by color cues.

AI doesn't replace a manual audit (it has its own false-positive risks), but it materially expands the coverage of automated scanning — pushing the 30-40% number toward 50-60% in our internal benchmarks. Plumb's paid plans apply AI augmentation to every page; the report clearly distinguishes AI-detected findings from axe-core findings so you can weigh confidence accordingly.

The honest recommendation

For most businesses:

1. Run automated + AI-augmented scans on every release. This is what we do at Plumb. Catches the high-volume issues and expands into judgment territory.
2. Hire a manual accessibility auditor for major launches. A one-day audit of critical user flows produces findings worth far more than their cost. Our audits are designed to feed into a manual auditor's process — they read our report and skip the 80% of work we already did.
3. Test with assistive-technology users for critical flows annually. The only way to know if your account-creation flow works for a JAWS user is to watch a JAWS user attempt it.

Anyone who tells you a single tool — automated or manual — covers all your accessibility needs is selling you something. Real conformance is layered.