Never heal an assertion

Self-healing automation has a simple promise. When a developer renames a button or restructures a page, the test finds the element anyway instead of failing. For a large suite, that promise is worth a lot: in most mature suites I’ve worked with, broken locators, not real defects, are the single biggest source of red builds.

But there’s a version of self-healing that is worse than none at all: one that heals its way to a pass when the application is actually broken. The rule that prevents it is short.

A locator is healable. An assertion’s expected value is sacred.

What the rule means in practice

A locator answers “which element?” If the Submit button moved from one container to another, finding it again is a fair repair. The test’s intent hasn’t changed.

An assertion answers “is the application right?” If the test expects the total to read $1,240.00 and the page shows $1,204.00, there is nothing to heal. That’s the defect the test exists to catch. Any system that “adapts” the expected value, or quietly swaps in a different element whose text happens to match, has turned your test suite into a rubber stamp.

Five design decisions that keep healing honest

The healing toolkit I’ve designed and built follows a handful of principles. None of them are exotic. Together, they’re the difference between a tool a release manager trusts and one they switch off.

1. Heal against a snapshot, not the live site. Candidates are validated offline against a captured DOM snapshot from the failing run. That makes every heal reproducible and reviewable, and it means healing never pokes at a live environment.

2. Accept a candidate only if it matches exactly one element. If a proposed locator matches zero elements, it’s wrong. If it matches three, it’s a guess. Exactly one, or it escalates to a person.

3. Keep healing separate from the verdict. By default, healing only captures a proposed fix. The test still fails, and the fix arrives as a pull request. Teams that opt into in-flight retries get a distinct status, PASSED_WITH_HEAL, so a healed pass is never mistaken for a clean one.

4. Keep AI out of the decision. The core of the healer is deterministic: candidate generation, validation, and the exactly-one rule. AI helps write the pull request description. It never decides whether a heal is safe, and the whole toolkit runs without AI in CI.

5. Roll out canary-first. Healing starts on a slice of the suite, and its proposals are reviewed before it earns wider scope, the same way you’d roll out any change to production.

Make a heal a data edit

Healing works best when locators live outside the test code, as externalized descriptors. Then a heal is a one-line change to a data file that anyone can review, rather than a rewrite of test logic.

// checkout.locators.json: the proposed heal, as a reviewable diff
  "submitOrder": {
-   "xpath": "//div[@class='actions']/button[2]"
+   "role": "button", "name": "Place order"
  }
// matched: 1 element in snapshot run-4812 | assertions touched: 0

Notice that the heal also upgrades the locator, from a positional XPath to a role and accessible name. Every heal leaves the suite a little sturdier than it found it.

Be honest about the limits

No healer fixes everything. A raw XPath or CSS locator pointing at an element with no accessible name gives the healer nothing to anchor on, so it produces no candidates and hands the failure to a person. That’s the correct behavior. The long-term fix is upstream: guardrails at commit time that reject hard-coded, nameless locators before they ever enter the suite.

Where this runs

We expose the healer as an MCP server over stdio, so the same tools work from Claude, Copilot Chat, or Roo Code in the IDE, with a matching command-line interface for CI. Engineers can ask their assistant to diagnose a failure and review the proposed heal without leaving their editor. The same principles now extend to native mobile apps through Appium.

← All insights