Accessibility

What axe-core Actually Catches — and the Part It Doesn't

10 min readAuthor: imola

WCAG test automation

I pulled apart axe-core's rule set to find out exactly where the tool stops. The answer is more specific than "automation catches about a third."

axe-core 4.12.1 ships 105 rules. At default settings, 96 of them are active, and between them they touch 15 of the 31 WCAG 2.2 Level A success criteria. Turn on every rule the library has and you get 16. The other 15 criteria have no rule at all — not a weak rule, not a partial one. Nothing.

That's not a criticism of axe. It's the most rigorous open-source accessibility engine there is, and most of the gaps are things no static analyser could ever decide. But if your team treats a clean axe run as "we tested accessibility", this is the size of what you didn't test.

And the covered half is thinner than the number suggests, which is the part worth staying for.

How I got these numbers

Straight from the package, not from a blog post:

npm install axe-core          # 4.12.1
node -e "const axe=require('axe-core');
         console.log(axe.getRules().length)"

Every rule carries tags. Conformance rules carry their criterion — wcag111, wcag244, wcag412. So mapping rules onto criteria is just reading the tags off the rule set and intersecting them with the 31 Level A criteria in the WCAG 2.2 Level A checklist. Rules with enabled: false are off unless you switch them on.

Run it yourself against whatever version you have. Rule sets change, and this one is a snapshot of 4.12.1.

Which Level A criteria does axe-core have no rule for?

Sixteen of the thirty-one, at default settings. Here they are, with why the gap exists.

Criterion Why there's no rule
1.2.1 Audio-only / Video-only There is a rule (audio-caption) but it's off by default
1.2.3 Audio Description Whether narration covers what's on screen is a content judgment
1.3.2 Meaningful Sequence Requires knowing the intended reading order — that's in someone's head
1.3.3 Sensory Characteristics Needs a machine to understand that "the button on the right" is an instruction
2.1.2 No Keyboard Trap Needs a live interaction session, not a DOM snapshot
2.1.4 Character Key Shortcuts Shortcut behaviour lives in event handlers, not markup
2.3.1 Three Flashes Requires measuring rendered animation over time
2.4.3 Focus Order Whether the order "preserves meaning" is a judgment about meaning
2.5.1 Pointer Gestures Needs to know a gesture exists and whether an alternative does
2.5.2 Pointer Cancellation Down-event vs up-event behaviour is runtime, not structure
2.5.4 Motion Actuation Device-motion handlers aren't visible in the DOM
3.2.1 On Focus Requires focusing everything and observing consequences
3.2.2 On Input Same — needs interaction, then a judgment about "change of context"
3.2.6 Consistent Help Requires comparing several pages against each other
3.3.1 Error Identification Whether an error message is useful is a language judgment
3.3.7 Redundant Entry Requires running a multi-step process end to end

Read the right-hand column. Nearly every entry comes down to one of two things: it needs interaction over time, or it needs someone to decide what a thing means. A rule engine inspecting a rendered DOM can do neither, and no version bump is going to change that.

Half of Level A isn't unimplemented in axe. It's unimplementable by anything that reads a page instead of using it.

The covered ones aren't fully covered either

This is the part that catches teams out. "Has a rule" and "is verified" are different claims, and the gap between them is where audits go wrong.

1.1.1 Non-text Content has seven rules. The main one, image-alt, is described in the library as: "Ensure <img> elements have alternative text or a role of none or presentation." Presence. Not quality. alt="image" passes. alt="DSC_00417.jpg" passes. A sales chart with alt="chart" passes clean.

2.4.4 Link Purpose is covered by link-name — "Ensure links have discernible text." Eight "Read more" links on one page, all with discernible text, all green. The criterion asks whether the purpose is clear. The rule asks whether the string is empty. [TODO: internal link → "Read more" links and aria-label]

3.3.2 Labels or Instructions is the strangest of them. Exactly one rule carries the wcag332 tag: form-field-multiple-labels, or "Ensure form field does not have multiple label elements." That checks a field doesn't have too many labels. The rule that checks a label exists at all (label) is tagged under 4.1.2 instead. So on the criterion that covers whether your form tells people what to type, axe's dedicated contribution is a duplicate-label check.

2.5.3 Label in Name has one rule, label-content-name-mismatch, and it's flagged experimental in the rule set. It's on by default and it's genuinely useful — but "experimental" is the library telling you its false-positive rate isn't settled.

None of this is axe being sloppy. Each rule is doing the honest, decidable part of a criterion that also has an indecidable part. The tool reports on the part it can see, and your report doesn't come with a note saying which part that was.

What about the 30 rules that aren't WCAG at all?

Of the 105 rules, 30 carry only the best-practice tag. They aren't mapped to any success criterion at all: heading-order, region, skip-link, landmark-one-main, page-has-heading-one, empty-heading, tabindex, and two dozen more.

They're good advice, and several are things I'd raise in review anyway. But they aren't conformance findings. Paste an axe report into a compliance document without separating them and you've mixed "this violates WCAG" with "Deque thinks this is a nice idea". Those two claims get treated very differently in an audit. Only one of them is arguable.

It cuts the other way as well. heading-order, the one that asks whether your outline is logical, is best-practice only. Meanwhile 1.3.1 Info and Relationships is a real Level A criterion, and the one about structure. It's carried by rules on lists, tables and <p> elements styled to look like headings. So the check most testers think of as the heading check isn't the one holding the conformance weight.

What's switched off when you press run?

Nine rules are disabled by default in 4.12.1:

aria-roledescription, audio-caption, color-contrast-enhanced, duplicate-id-active, duplicate-id, identical-links-same-purpose, landmark-complementary-is-top-level, meta-refresh-no-exceptions, target-size

Two of those matter more than the rest.

audio-caption is the only rule touching 1.2.1. Off by default, so out of the box that criterion drops to zero coverage — which is the whole difference between 16 and 15.

target-size implements 2.5.8 Target Size (Minimum), one of the new WCAG 2.2 AA criteria. If your compliance target is 2.2 AA and nobody enabled it, the criterion your stakeholders think is being checked is not running.

Seven more rules are marked experimental, including p-as-heading and css-orientation-lock. Enabled, but caveat-flagged by their own authors.

So where does the manual pass start?

Right where the table above starts. That list isn't a list of gaps to feel bad about — it's a work plan, and it's the most useful output of this whole exercise.

  1. Run axe first, on every build. It clears the mechanical failures in seconds and it catches regressions a human sweep won't. Nobody should be hand-checking for missing alt attributes in 2026.
  2. Turn on the disabled rules you actually need. At minimum target-size if you're claiming 2.2 AA, and audio-caption if you ship audio.
  3. Split the report before it reaches anyone else. WCAG-tagged findings in one column, best-practice in another. Ten minutes, and it stops a compliance conversation going sideways.
  4. Treat every "covered" criterion as half-done. For 1.1.1 and 2.4.4, the tool told you the string exists. Reading it is still yours.
  5. Book the interaction passes. Keyboard (2.1.2, 3.2.1), pointer (2.5.1, 2.5.2), process (3.3.7, 3.2.6). None of these can be done from a snapshot, so they need a person driving the product.
  6. Listen to something. 3.3.1, 1.2.3 and the meaning half of 1.1.1 come down to what a person actually hears. [TODO: internal link → Screen reader walkthrough]

Steps 5 and 6 are the ones teams skip, and they're where the findings that fail an audit live.

Quick FAQ

How much of WCAG can automated testing verify? By criterion count, in axe-core 4.12.1: 15 of 31 Level A criteria have at least one default-enabled rule, and several of those only partially. Deque, which builds axe, reports a much higher figure by issue volume. That measures something different. A handful of automatable failures like colour contrast appear on nearly every page, which is what inflates it. Both numbers can be true. Only one of them tells you what's left to test.

Is axe-core better than the other scanners? It's the engine inside a lot of them, including the accessibility panel in Chrome DevTools. Arguing about which wrapper to use mostly misses the point — the coverage ceiling is the same, because the hard criteria are hard for structural reasons.

Does a clean axe run mean anything at all? Yes. It means no machine-detectable failure is present, which is a real thing to know and worth automating. It doesn't mean the page conforms, and it doesn't mean anyone can use it.

Should I fail the build on axe violations? On WCAG-tagged violations, yes. On best-practice rules, that's a team decision, and I'd make it consciously rather than by leaving the default on.

Which version are these numbers from? axe-core 4.12.1, read from the installed package. Rule sets change between releases — re-run the check rather than citing this page in two years.

The handoff

If you take one thing from this: the value of running axe isn't the report. It's that the report tells you precisely where your job starts. Sixteen Level A criteria, plus the judgment half of the fifteen it does touch. That's the scope of the manual pass, and now you can put a number on it in a planning meeting.

The next question is what that manual pass sounds like. Most of the remaining criteria come down to what assistive tech actually announces, and that's a skill you build by listening, not by reading. Which is what the Pearly Quality accessibility workshop is for: small group, live, hands on your own product, starting exactly where the scanner stopped. [TODO: capture link → Screen Reader Testing Cheat Sheet download]

Rule counts, tags and descriptions in this article were read from axe-core 4.12.1 via axe.getRules() and the library's internal rule set on 4 August 2026, not from secondary sources. Criterion levels are from WCAG 2.2 (W3C Recommendation, updated 12 December 2024).