RunPull Request Testing

Exploration Agent

View as Markdown

The exploration agent is an AI agent that tests each PR without any saved steps. It reads the pull request, writes a test plan tailored to that change, and executes it in a real browser (or mobile simulator). Great for surfacing issues nobody thought to write a test for yet.

Exploration agent PR comment

For flows you already know matter and want verified the same way every time, pair it with Regression testing.

See Pull Request Testing for prerequisites, supported providers, and troubleshooting.

How it plans and executes

For every PR the agent works in two phases:

  1. Plan - it reads the PR, decides what is worth testing, and writes an ordered list of steps tailored to that change.
  2. Execute - it runs those steps in a real browser or mobile simulator and reports the result, just like a saved test.

The plan is tailored to each PR and considers:

  • PR title and description - understands intent.
  • Changed files - focuses on affected areas when the project has a connected GitHub repository. With a connection it can read the actual diffs, not just file names.
  • Application context - project memory, test accounts, uploaded files.

It covers functional checks, visual verification, and edge cases relevant to the changes.

The PR comment first shows the planned steps as a table, then updates in place with completed/total counts and per-step results as the run finishes.

If the agent decides a PR has no user-visible effect to test (for example, a backend-only or config-only change), it skips execution and the PR check reports a neutral “Tests skipped” status with the reason, instead of a pass or fail.

Setup

For Vercel previews, connect GitHub, connect Vercel, and select the Vercel project that should be allowed to trigger tests. Add a bypass token only if previews are protected. No step authoring is required.

Remote CI and GitHub Enterprise

If your repository is on GitHub Enterprise or another CI system that cannot use the TesterArmy GitHub App webhook, trigger the exploration agent directly:

$curl -X POST "https://tester.army/api/v1/projects/$PROJECT_ID/pull-request-runs" \
>  -H "Authorization: Bearer $TESTERARMY_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "targetUrl": "https://preview.example.com",
>    "pullRequest": {
>      "number": 123,
>      "title": "Add onboarding flow",
>      "description": "PR body or summary",
>      "commitSha": "'$GITHUB_SHA'"
>    }
>  }'

The endpoint is metadata-first: do not send repository identifiers, changed files, or patches. When a supported GitHub connection exists for the project, TesterArmy resolves the configured repository, enriches the run with changed files, and posts best-effort PR comments/checks. Without a connection, the agent plans from PR title/body, project memory, target URL, and saved credentials.