Skip to main content

​
CI/CD Integration (GitHub & Azure)

TMS ONE integrates with GitHub Actions and Azure DevOps Pipelines so you can trigger automated Playwright test runs from your existing CI/CD pipeline — for example, as part of a deployment workflow.

​
How test execution works in TMS ONE

Before setting up CI/CD, it helps to understand how TMS ONE actually runs Playwright tests. When a test is executed — whether from the UI or from a pipeline — TMS ONE runs the Playwright scripts on its own server infrastructure. You do not need Playwright installed in your own environment or pipeline. The execution flow works like this: 
Trigger (UI or CI/CD pipeline)
        ↓
TMS ONE API receives the request
        ↓
Job is queued in a Redis-backed queue (Bull)
        ↓
A worker picks up the job and runs:
  → npx playwright test (JavaScript)
  → dotnet test (C#)
        ↓
HTML report generated, zipped, uploaded to S3
        ↓
Results saved to database
        ↓
Real-time status pushed to the UI via WebSocket
        ↓
Results visible in Execution History
Key point: The queue can handle up to 10 concurrent test runs. If multiple runs are triggered at once, they are queued and processed as workers become available. The API responds immediately when a run is submitted — the test execution happens in the background.

​
How the CI/CD path works

The CI/CD integration works through a simple mechanism:
  1. You authenticate TMS ONE with your GitHub or Azure account
  2. TMS ONE connects to your repository
  3. When you click Run Workflow, TMS ONE sends a workflow_dispatch event to your GitHub Actions workflow (or triggers an Azure Pipeline)
  4. Your pipeline runs — and the workflow file you add to your repo contains a single step that calls back to the TMS ONE API to start the test run
  5. TMS ONE queues and executes the tests on its server, exactly the same as a direct UI execution
The CI/CD pipeline itself does not run Playwright — it simply tells TMS ONE to start a test run. All execution happens on the TMS ONE server. 
TMS ONE UI (Run Workflow)
        ↓
GitHub / Azure receives dispatch event
        ↓
Pipeline job executes:
  curl https://api.tmsone.app/api/v1/playwright/run-tests-ciCD?id=...&projectId=...&email=...
        ↓
TMS ONE API queues and runs the Playwright scripts on its server
        ↓
Results saved and visible in TMS ONE Execution History
This means your pipeline doesn’t need any Playwright dependencies, test files, or browser setup. The workflow file is intentionally minimal.

​
Setting up the integration

​
Step 1 — Authenticate

Before connecting a repository, authenticate TMS ONE with your Git provider. The Authenticate button appears in:
  • The API Runner sidebar
  • The Test Builder → Automation execute page (the Execute Tests view)
Click Authenticate and choose your provider: GitHub Authentication — you are redirected to GitHub to authorise TMS ONE. Once authorised, you are returned to TMS ONE and your GitHub account is connected. Azure Authentication — you are redirected to Microsoft login. Sign in with your Azure DevOps account. Once authorised, your Azure organisation is connected.
Authentication is per-user and persists across sessions. You only need to authenticate once. To switch accounts, authenticate again using the same dropdown.

​
Step 2 — Add the sample workflow file to your repository

From the Run Workflow modal, download the sample workflow file for your provider. Add it to your repository:
  • GitHub: Place the file in .github/workflows/ in your repository
  • Azure DevOps: Add the YAML file to your pipeline configuration
The sample workflow contains a single step — a curl call to the TMS ONE run endpoint. Before committing, replace the placeholder values with your actual script IDs, project ID, and email address. These values identify which scripts to run and which account to attribute the run to.
The workflow is triggered via workflow_dispatch (a manual/API trigger). It runs on a standard ubuntu-latest runner and makes a single HTTP GET request to the TMS ONE API. No Playwright installation or test files are needed in the workflow.
The pipeline is triggered the same way. The sample uses a self-hosted pool by default — you can switch this to ubuntu-latest or another hosted agent if preferred, since no special dependencies are required.
After downloading the sample, replace the placeholder values (project ID, email) with your actual TMS ONE project details before committing the file to your repository.

​
Step 3 — Connect repository & run

Once authenticated and your workflow file is in place, click Run Workflow on the Test Builder → Automation execute page. Fill in the Connect Repository & Run Workflow modal: For GitHub:
​
Provider
select
required
Select Connect GitHub Repository.
​
GitHub Organisation
text
Auto-populated from your authenticated GitHub account. Read-only.
​
Repository
select
required
The repository where your workflow file lives.
​
Initial Branch
select
required
The branch to dispatch the workflow against.
​
Workflow ID
string
required
The filename of your GitHub Actions workflow file.
Example: ci-workflow.yml
For Azure DevOps:
​
Provider
select
required
Select Connect Azure Repository.
​
Azure Organisation / Project
text
Auto-populated from your authenticated Azure account. Read-only.
​
Repository
select
required
The Azure DevOps repository.
​
Initial Branch
select
required
The branch to run the pipeline against.
​
Workflow ID
string
required
The pipeline filename.
Example: azure-pipeline.yml
Click Run Workflow. TMS ONE dispatches the event to your provider. Your pipeline starts, calls back to TMS ONE, and the test run is queued on the TMS ONE server.

​
Viewing results

Results from CI/CD-triggered runs appear in Execution History under the Automation tab — exactly the same as runs triggered directly from the UI. Each entry shows pass/fail counts, which scripts ran, and a link to download the full HTML report. While a run is in progress, the execute page shows an “Execution is in progress” indicator. Real-time status updates are pushed to your browser automatically.

​
CI/CD in the API Runner

In the API Runner, the same GitHub/Azure authentication also lets you commit and push API collections directly from TMS ONE to your connected repository. This keeps your API test collections version-controlled alongside your application code. After authenticating, a Commit & Push option becomes available on your API collections.

​
What’s next?

Test Builder

Generate and manage Playwright automation scripts

Execution History

View results from all test runs