Automate TestingCI and Custom Deployments

Run from GitLab CI

View as Markdown

GitLab CI can run a TesterArmy test group after it deploys your application. This is a CI recipe, not a native GitLab repository integration.

TesterArmy does not currently connect to GitLab repositories, read merge request diffs, or post GitLab commit statuses and merge request comments. Full results are available in the TesterArmy dashboard.

Prerequisites

  1. Create and manually run the tests you want to automate.
  2. Add them to a test group.
  3. Enable the group’s signed webhook and copy its URL.
  4. Add the URL to GitLab CI/CD variables as a masked variable named TESTERARMY_WEBHOOK_URL.

If the deployment URL is stable, create a saved environment and store its ID as TESTERARMY_ENVIRONMENT_ID. For temporary review apps, send the deployment URL as targetUrl.

Stable staging environment

Trigger the group after the staging deployment succeeds:

1testerarmy:
2  stage: test
3  needs: [deploy_staging]
4  script:
5    - |
6      curl --fail-with-body -X POST "$TESTERARMY_WEBHOOK_URL" \
7        -H "Content-Type: application/json" \
8        -d "{\"commitSha\":\"$CI_COMMIT_SHA\",\"projectEnvironmentId\":\"$TESTERARMY_ENVIRONMENT_ID\"}"

The saved environment supplies the URL and any environment-specific credential overrides.

GitLab review app

When the deployment job exposes a temporary review URL, pass it directly:

1testerarmy:
2  stage: test
3  needs: [deploy_review]
4  rules:
5    - if: $CI_MERGE_REQUEST_ID
6  script:
7    - |
8      curl --fail-with-body -X POST "$TESTERARMY_WEBHOOK_URL" \
9        -H "Content-Type: application/json" \
10        -d "{\"commitSha\":\"$CI_COMMIT_SHA\",\"targetUrl\":\"$CI_ENVIRONMENT_URL\",\"environment\":\"preview\"}"

The review URL must be publicly reachable. Configure site access or custom headers when the deployment is protected.

Results and pipeline behavior

The webhook returns the queued TesterArmy run IDs. A successful HTTP response means the runs were accepted, not that every test passed.

  • Full steps, screenshots, videos, and issues appear in TesterArmy.
  • This webhook does not update a GitLab commit status or merge request comment.
  • commitSha is retained as run metadata. It only enables a GitHub Check Run when the TesterArmy project is separately connected to a GitHub repository containing that commit.

Use the API-key group-run endpoint instead when the pipeline needs richer response metadata or will poll individual run results. Open the API Reference tab for endpoint details.

Troubleshooting

401 Unauthorized

The secret-bearing webhook URL is incorrect. The plaintext secret cannot be recovered or regenerated from the dashboard after its creation session, so keep the original URL in your GitLab CI/CD variables.

403 Forbidden

The group webhook is disabled. Re-enable it from the group’s trigger controls.

Tests use the wrong URL or account

Use projectEnvironmentId to select a saved environment and its credential overrides. The environment field only labels the result; it does not select a saved environment.