UI automation creates value when it gives teams dependable feedback about product quality. Yet ordinary interface changes can consume a surprising amount of time in locator maintenance: a renamed button, a modified attribute, or a changed hierarchy can break many tests even when the intended business behavior has not changed. Pandit has seen teams spend more time debugging test failures than investigating product issues. That shift is subtle, but it can erode trust in automation and change the value it delivers.

Pranav Pandit, a Staff Quality Engineer with more than 15 years of experience in mobile and web automation, published When AI maintains your tests: Designing safe self-healing automation in Ministry of Testing on August 4, 2026. Its central design choice is clear: AI should be a controlled fallback after a locator fails, not the primary test execution engine. This article brings together the supplied study notes and the source article. Code is illustrative, not a verified reproduction of the author’s implementation; additional engineering recommendations are labeled separately.

Locator maintenance is a hidden cost

Frameworks such as Appium, Espresso, and XCUITest use locators to find and interact with interface elements. Because locators depend on the interface representation, even a small text or hierarchy change can trigger an “element not found” failure. Engineers inspect the logs, update the locator, and rerun the tests. A shared component change can break dozens of checks at once.

Why not let an LLM drive every interaction? Deterministic automation is predictable and easier to diagnose. AI can adapt, but its decisions are probabilistic and may be confidently wrong. A safer hybrid keeps the normal path deterministic and invokes AI only for a specific, eligible locator failure.

Two paths, two different responsibilities

SituationSystem behavior
Original locator matchesContinue the test using the regular deterministic path
Eligible locator lookup failsCheck for a previously validated recovery in cache
No valid cached recoveryParse intent and action, detect screen context, reduce UI hierarchy, and request one candidate or an explicit abstention
Candidate fails validationPreserve the original failure and record why the candidate was rejected
Candidate passes validationExecute the step, retain the original assertions, then log the outcome and cache the recovery if appropriate

A business assertion failure, backend outage, or application crash must not be relabeled as a locator issue just because the test failed. And finding a replacement locator does not make the test pass: the original business assertions still need to run.

The recovery pipeline

Parse the locator and its use

The parser combines the failed locator with its use in the test. That can make the intended text, element type, action, and workflow hint explicit.

CHECKOUT_CONTINUE_BUTTON = {
    "by": "xpath",
    "value": "//android.widget.Button[@text='Continue']"
}
click(CHECKOUT_CONTINUE_BUTTON)

parsed = {
    "text": "Continue",
    "class": "android.widget.Button",
    "action": "click",
    "context": "checkout",
    "element_type": "button"
}

This turns implicit instructions into structured signals. A variable name such as CHECKOUT_CONTINUE_BUTTON can help, but it is a hint—not definitive evidence of business intent.

Detect the screen and workflow context

A button labeled “Continue” may exist in both a modal and the screen behind it. A unique, visible, enabled match can still be the wrong control. Useful context includes the active screen, workflow stage, foreground dialog, and surrounding test step—not just matching text.

Prune XML without discarding useful structure

Large mobile UI hierarchies can increase model input and ambiguity. Pruning should keep relevant candidates while retaining enough parent, sibling, and modal relationships to interpret them.

def prune_xml(nodes, signals):
    candidates = []
    for node in nodes:
        text = (node.text or "").lower()
        if signals["text"].lower() in text:
            if node.class_name in {"Button", "TextView"}:
                candidates.append(node)
    return candidates[:5]

This intentionally simplified example uses exact text matching. It would miss a legitimate rename from “Continue” to “Next.” A production system must handle approved text changes without losing the relationships needed to select the right control.

Ask the model for a constrained proposal

Do not ask the model to “fix the test.” Provide the structured intent, action, screen context, and reduced hierarchy. Require one stable locator where possible, and a clear EMPTY response when there is not enough evidence.

Task: Propose a locator for the failed test step.
Inputs: intent, action, screen context, pruned hierarchy.
Constraints:
- Match the intended interaction and screen context.
- Prefer stable attributes.
- Reject ambiguous or unsupported candidates.
- Return EMPTY when evidence is insufficient.
Output: One locator or EMPTY.

Abstention is a safety feature. A clear refusal is better than a confident but unsupported recovery that hides a product problem.

Validate separately from generation

A model proposes a locator; deterministic checks decide whether it is eligible to run. The source discussion includes checks for exactly one match, expected element type, visibility, and interaction readiness.

def validate_candidate(driver, locator, action):
    elements = driver.find_elements(*locator)
    if len(elements) != 1:
        return False
    element = elements[0]
    if not element.is_displayed():
        return False
    if action == "click" and not element.is_enabled():
        return False
    return True

This illustrates only basic checks. is_enabled() is not a complete clickability test. Real implementations must also consider overlays, active windows, platform behavior, and state changes between validation and execution. A successful click is still not proof that the intended business outcome occurred.

Cache and logs are reliability infrastructure

If the same interface change affects dozens of tests, repeating an LLM call for each test is wasteful. Cache a previously validated recovery, but scope it carefully: application version, platform, screen, action, original locator, and validity period may all affect whether the result remains safe.

Audit logs should make it possible to answer: Why did recovery start? What evidence did the model receive? Why did the validator accept or reject the proposal? Did the original test’s business assertions pass afterward? Without those details, self-healing exchanges an obvious failure for an opaque decision.

Three ways to adopt recovery

StrategyWhat it doesTrade-off
Runtime-onlyUses the recovered locator for the current execution; does not change repository codeLowest code-change risk, but execution outcomes still need monitoring
Human-reviewedProposes a code change or pull request for an engineer to reviewBalances assistance with traceability and existing code review
Auto-mergeWrites and merges a repair without human approvalHighest risk; should not be the default

Start with runtime-only recovery and collect evidence. Consider automatically preparing pull requests only after the system demonstrates reliable behavior. Merging code is a separate authorization level from recovering one test execution; it needs stricter evidence, regression checks, rollback, and permission controls.

What the 40-plus failure case teaches

The supplied notes describe a button renamed from “Continue” to “Next,” causing more than 40 tests to fail. The system recovered most of them, but some candidates matched similar controls in the wrong workflow.

The central challenge was not simply finding an element. It was confirming that the element belonged to the business step under test. The team focused on context and constraints rather than only trying a more capable model.

There is another quality risk: if a label change violates a requirement or accessibility agreement, healing the locator must not silently normalize that product regression. Whether recovery is acceptable depends on what the test is meant to protect.

Limitations worth keeping visible

The source discusses four persistent constraints:

  1. Intent remains uncertain. Similar elements can still be confused.
  2. UI relationships can be lost. Pruning may discard the hierarchy needed for a correct decision.
  3. Prompts are brittle. A change that helps one case can regress another.
  4. Recovery adds latency and cost. Model calls introduce additional processing and infrastructure expense.

A high healing rate is not a success metric by itself. The system might recover more failures while also increasing incorrect recoveries—the more dangerous outcome.

Engineering extensions: how to show recovery is safe

The recommendations below extend the article; they are not claims about features the author has implemented.

Distinguish recovery states. Track not attempted, rejected, recovered, and verified separately. Recovered means the interaction ran; verified means the relevant business postcondition also passed.

Check postconditions. After clicking “Next” in checkout, assert that the expected next stage appears. Structural validation answers whether the element can be operated; business assertions answer whether the right action happened.

Build a regression evaluation set. Include renamed labels, duplicate controls, dialogs, reordered hierarchies, expired cache entries, cross-screen matches, and genuine product defects. Version the prompt and record the model and pruning configuration used for each run; rerun the cases whenever any of them changes. This makes regressions traceable instead of relying on memory of the last successful result.

Fail closed when evidence is weak. Preserve the original failure and diagnostics when the candidate is ambiguous. Apply stricter policies to payments, deletion, permission changes, and other high-impact actions.

Protect data. UI hierarchies and logs may contain personal or business-sensitive information. Minimize and redact inputs sent to an external model, then define retention and access controls.

For a pilot, track the share of locator-related failures, candidate acceptance, business-postcondition pass rate, incorrect recoveries, review time, added latency, and cost per recovery. Incorrect recovery and masked real defects should be primary safety guardrails.

What should teams explore next?

The article identifies four directions: prompts tailored to element types; regression testing for prompt changes; validation of business outcomes rather than interactions alone; and richer screen and user-flow context. These can be introduced gradually; they do not require starting with autonomous code modification.

The source article also leaves useful questions for a team discussion:

  • Would you trust AI to update locators in your test suite?
  • How much validation is enough before accepting an AI-generated fix?
  • Where should the boundary sit between automation and human oversight?
  • If you have tried a similar approach, what worked and what did not?

Self-healing is not about keeping every test green. It is about reducing avoidable maintenance without weakening the suite’s ability to expose real product failures. Deterministic automation keeps execution predictable; AI proposes a constrained recovery; validation determines whether it is safe; and traceable evidence keeps engineers in control.

An automation system that repairs a locator but quietly clicks the wrong control is not more reliable. It is a harder-to-detect quality risk.

References

Prepared from the supplied detailed notes and checked against the original article and its further-reading links. Code is illustrative; the engineering extensions are identified separately.

ACP (Agent Client Protocol)Active MemoryAgent AuthorizationAgent CardAgent Evals
Share