Stackness
Sign up
All moves

Use an AI agent to play QA tester as a simulated first-time user

by Stackness

AI Tools

Rather than only running scripted test suites, developers give an agent a skill or MCP-driven simulator that lets it interact with an app like a first-time user would, taking screenshots and recording tool calls for later inspection. Multiple simulated personas can be run through a simple audit workflow to catch confusion points a developer would miss by testing their own app. The technique treats UX and exploratory QA as something an agent can perform at scale rather than a purely manual task.

What it is

Instead of relying only on scripted unit and integration tests, you point a coding agent at a running app and let it click, type, and navigate the way a real visitor would, using a skill or an MCP-driven simulator rather than a fixed test script. The agent can drive a browser through a tool like Playwright or control a mobile simulator, and its screenshots and tool calls can be inspected afterward to see exactly what it did and why.

How to do it

  • Give the agent a role and context, not just a one-line instruction. One experiment found that a bare prompt like "test the website and look for UX errors" found far fewer issues than a prompt that specified a persona, a way of working, and a report format, and performance improved further once a full checklist based on usability heuristics and accessibility guidelines was added.
  • Define several personas instead of one generic tester. An open-source agent skill called First User Audit runs an app through four behaviors: an impatient user who gives up quickly, a skeptic who checks pricing and terms, a mobile user at a narrow viewport, and a "breaker" who tries invalid inputs, double submissions, and back navigation.
  • Have the agent identify what a first successful outcome looks like, explore the onboarding journey under each persona, record blockers with evidence and reproduction steps, and propose fixes before verifying them in a fresh session.
  • If your tooling supports it, run the agent against a simulator with a live watch page so a person can see taps, swipes, and text entry as they happen alongside screenshots and an interaction list.

When it helps

This approach catches friction a developer would never notice because they already know where every button leads, including confusing states during waiting, failure, retries, and small screens.

Pitfalls

Results vary significantly by run and by how the prompt is written, so a single pass can under-report problems. Simulated personas do not replace real usability testing, and agents should be kept away from production data or live payments during these runs.

Sources