Local Development

Use the CLI to validate changes against your local dev server before pushing.

Set your target URL

Set the target once for the session:

$
export TESTERARMY_TARGET_URL="http://localhost:3000"

Or pass it per-run:

$
ta tests run <testId> --url http://localhost:3000

Run tests

Start your dev server, then run dashboard-defined tests locally:

$
# Run a single test
$
ta tests run <testId>
$
$
# Run all tests in a group
$
ta tests run --group <groupId> --project <id>

Add project memory

Use memories to give the agent durable project context before it runs tests:

$
echo '{"category":"site_structure","title":"Auth route","content":"Login is at /login","importance":"high"}' | ta memories create --project <projectId>
$
ta memories list --project <projectId>
$
ta memories delete <memoryId> --project <projectId>

Headed mode

Watch the AI agent work in a visible browser:

$
ta tests run <testId> --headed

Useful for debugging flaky tests or understanding agent behavior.

Debug mode

Save detailed logs and transcripts to .testerarmy/:

$
ta tests run <testId> --debug

Check .testerarmy/<timestamp>/debug-run.json for the full stream/tool transcript.

Inspect artifacts

Every run writes to .testerarmy/<timestamp>/:

• run-meta.json - run metadata
• result.json - pass/fail results
• debug-run.json - full transcript (with --debug)

List recent runs:

$
ta runs list --project <projectId>

Typical workflow

$
# 1) Start dev server
$
pnpm dev
$
$
# 2) Set target
$
export TESTERARMY_TARGET_URL="http://localhost:3000"
$
$
# 3) Make changes
$
$
# 4) Run targeted test
$
ta tests run <testId>
$
$
# 5) Check results, iterate
$
ta runs list --project <projectId>

CI usage

Set TESTERARMY_API_KEY and TESTERARMY_TARGET_URL as environment variables in your pipeline:

$
ta tests run --group <groupId> --project <id> --parallel 3 --json

Exit code 1 on any test failure - your pipeline fails automatically.

1
- name: Run QA tests
2
  run: npx testerarmy tests run --group <groupId> --project <id> --parallel 3 --json
3
  env:
4
    TESTERARMY_API_KEY: ${{ secrets.TESTERARMY_API_KEY }}
5
    TESTERARMY_TARGET_URL: ${{ steps.deploy.outputs.url }}