Automate TestingCI and Custom Deployments

Any CI or Custom Deployment

View as Markdown

Trigger TesterArmy after any CI system or deployment platform finishes deploying your application. This works with GitHub Actions, Jenkins, Buildkite, CircleCI, shell scripts, and other systems that can send an HTTP request.

For GitLab, use the focused GitLab CI recipe.

How it works

  1. Create and manually verify a test group.
  2. Enable a signed webhook for the group.
  3. After deployment, your pipeline sends the saved environment ID or deployed URL to the webhook.
  4. TesterArmy queues every enabled test in the group and returns the run IDs.
  5. Full results appear in the TesterArmy dashboard.

The target URL must be reachable from the public internet. “Custom deployment” does not mean a private TesterArmy runner or access to private-network URLs.

1. Enable a signed group webhook

  1. Open your project → Tests tab.
  2. Expand the test group you want to run on each deploy.
  3. Select the trigger control and toggle Enable webhook on.
  4. Copy the URL immediately. The secret is only available in the session that creates it.

The URL looks like:

https://tester.army/api/v1/groups/webhook/{id}/{secret}

The secret is part of the URL, so store the entire value as a masked CI secret. See Signed Group Webhook Reference for payload fields, mobile overrides, responses, and limits.

2. Choose the run target

Use a saved environment for stable staging or QA deployments:

1{
2  "commitSha": "abc123",
3  "projectEnvironmentId": "00000000-0000-0000-0000-000000000000"
4}

Use targetUrl for temporary deployments and review applications:

1{
2  "commitSha": "abc123",
3  "targetUrl": "https://preview.example.com",
4  "environment": "preview"
5}

projectEnvironmentId selects a saved URL and its credential overrides. environment only labels the result as production, staging, or preview; it does not select a saved environment.

3. Call the webhook after deployment

GitHub Actions

1- name: Run TesterArmy tests
2  run: |
3    curl -X POST "${{ secrets.TESTERARMY_WEBHOOK_URL }}" \
4      -H "Content-Type: application/json" \
5      -d "{\"commitSha\":\"${{ github.sha }}\",\"targetUrl\":\"${{ steps.deploy.outputs.url }}\",\"environment\":\"preview\"}"

Any shell

$curl -X POST "$TESTERARMY_WEBHOOK_URL" \
>  -H "Content-Type: application/json" \
>  -d '{"commitSha":"'"$COMMIT_SHA"'","targetUrl":"'"$DEPLOY_URL"'","environment":"staging"}'

Use curl --fail-with-body when an authentication, validation, or usage-limit response should fail the CI job. A successful response means TesterArmy accepted the runs; it does not mean the tests have already passed.

Results

  • TesterArmy dashboard: Always contains the run status, test outcome, steps, screenshots, video, and issues.
  • GitHub Check Run: Created on a best-effort basis only when commitSha belongs to the repository connected through the native GitHub integration.
  • GitHub PR comment: Not created by the signed webhook because it does not accept a PR number.
  • GitLab status or merge request comment: Not supported by this trigger.

For a walkthrough focused on a staging environment, see Testing Staging Environment.

When to use the API instead

Use POST /api/v1/groups/{groupId}/runs with an API key when you need richer response metadata, result polling, or GitHub PR context. Unlike the signed webhook, the API-key endpoint accepts prNumber for best-effort GitHub PR reporting when the project has an active GitHub connection.

For change-specific exploration, use POST /api/v1/projects/{projectId}/pull-request-runs. Without a native GitHub connection, planning uses the PR metadata supplied by the caller and cannot read repository diffs or post provider-native results.

Troubleshooting

No check run on the PR

  1. Missing commitSha - the tests run, but no commit check can be created.
  2. Commit not in the connected repository - TesterArmy cannot attach the check to an unrelated SHA.
  3. GitHub App not connected - connect the repository through GitHub.
  4. Permissions - the GitHub App needs Checks: Read & Write.

401 Unauthorized

The webhook ID or secret is wrong. The plaintext secret cannot be recovered, but you can use Regenerate in the group’s trigger panel and replace the URL in your CI secret manager. The previous URL stops working immediately.

403 Forbidden

The webhook is disabled. Open the group’s trigger panel and toggle Enable webhook back on.

Tests run against the wrong URL

Tests use targetUrl from the payload if provided, otherwise the URL configured in your project settings. Check both.

Target URL is rejected

TesterArmy blocks localhost, private IP ranges, and internal hostnames. Deploy the application to a publicly reachable URL or expose it through an appropriate secure tunnel.