Connect GitHub

Connect one GitHub.com repository to a TesterArmy project to run tests against pull request deployments and report results in GitHub.

The native GitHub integration can:

• Discover pull requests and deployment previews.
• Read PR metadata and changed files for the exploration agent.
• Run a saved regression group, the exploration agent, or both.
• Create best-effort GitHub Check Runs and PR comments.
• Filter automatic runs by the PR’s target branch.

1. Install the GitHub App

1. Open your TesterArmy project.
2. Go to Integrations -> GitHub.
3. Select Install GitHub App.
4. Choose the GitHub organization or account and grant access to the repositories TesterArmy should test.
5. Return to TesterArmy and select Refresh if the installation is not shown yet.

If TesterArmy asks you to reconnect your GitHub account, select Connect GitHub account before continuing.

2. Connect a repository

1. Select the GitHub Installation.
2. Select the Repository for this TesterArmy project.
3. Optionally select a Target branch. Choose main, for example, to run automatic tests only for PRs targeting main.
4. Select Save connection.

Each TesterArmy project connects to one GitHub repository. A repository connection cannot be shared by multiple TesterArmy projects.

3. Choose PR coverage

Open the project’s PR Testing tab and enable one or both modes:

• Regression testing runs the enabled tests in a selected group.
• Exploration agent creates a plan for each PR from its code and metadata.

4. Connect preview delivery

Choose how TesterArmy receives the deployed URL:

• Vercel for verified Vercel preview deployments.
• Coolify for Coolify deployment webhooks.
• Any CI or custom deployment when your pipeline creates the preview URL itself.

GitHub reporting behavior

GitHub reporting is best effort and does not block the TesterArmy run. Check Runs conclude success when all tests pass. Failed, cancelled, skipped, or mixed batches use a neutral conclusion with details in the check title and summary.

Saved regression groups and exploration runs can maintain separate PR comments because they report different kinds of coverage.

Limitations

• The native integration supports GitHub.com. GitHub Enterprise Server cannot install this GitHub App flow.
• TesterArmy reads only the repository connected to the project.
• Automatic PR testing requires a deployed URL that TesterArmy’s hosted runners can reach.

For a non-native or GitHub Enterprise Server pipeline, use an API or webhook trigger. Without an active supported GitHub connection, exploration planning uses the PR metadata supplied by the caller and cannot read repository changes or post GitHub results.

Troubleshooting

Repository is missing

Update the GitHub App installation to grant access to the repository, then return to TesterArmy and select Refresh.

GitHub permissions need reconnect

Select Connect GitHub account in the integration panel, complete authorization, and retry the connection.

No PR run appears

Confirm that PR testing is enabled, the deployment completed successfully, the preview provider is configured, and the PR targets the selected branch.