Environments

Most apps ship to more than one place. You probably run the same code against prod, staging, maybe a uat for stakeholders, plus a per-PR preview deploy. Test plans care about behavior, not URLs, and you don't want to maintain three copies of every plan with different base URLs hardcoded into the prompt.

Environments are the unit you target with a run. A project owns one or more environments, each with its own URL, notification config, test cookies, HTTP headers, and proxy country. The same test plan runs against any of them, picked by name at trigger time.

What an environment owns

Each environment in a project carries its own copy of:

One environment per project is flagged as the default. When the run API is called without an env name, the default is used.

URL prepending in practice

When an env has a URL set, it's stitched onto the front of every test plan prompt that runs against it:

Env URL: https://staging.myapp.com

Test plan: "Go to the pricing page and verify the free tier is listed"

Effective prompt: "This test run is for https://staging.myapp.com.

Go to the pricing page and verify the free tier is listed"

This is the part that lets one test plan run against three deployments with no edits. The plan stays written in terms of behavior ("the pricing page", "the free tier"), and the env decides which host that behavior is checked against.

Creating an environment

Every new project comes with a single prod environment that picks up the default URL and notification config from the create form. Add more from the project edit page:

1. Open Dashboard → Projects → click your project.
2. Scroll to Environments, click + New Environment.
3. Fill in:
   • Name (required): e.g. staging, uat, preview.
   • URL (optional): the base URL for this env.
   • Notifications: Webhook, Slack, Teams. Each has its own URL and notify-on gate.
   • Test config: cookies + headers, same shape as project-level config but scoped to this env.
   • Proxy country: Custom plans can route through a non-US country.
4. Create Environment.

You can rename prod (it's just the default name), promote any env to default with Set Default, or soft-delete one you no longer need. The default env can't be deleted directly. Promote another env first.

Targeting an env from the API

Pass env as a string in the run request. It matches the env name on every project the run touches.

# Run all plans in a project against staging
curl -X POST https://test-lab.ai/api/v1/run \
  -H "Authorization: Bearer tl_xxxxx" \
  -H "Content-Type: application/json" \
  -d '{"projectId": YOUR_PROJECT_ID, "env": "staging"}'

# Run a specific plan against the preview deploy for PR 42
curl -X POST https://test-lab.ai/api/v1/run \
  -H "Authorization: Bearer tl_xxxxx" \
  -H "Content-Type: application/json" \
  -d '{"testPlanIds": [123], "env": "preview-42"}'

# Use the project's default env (prod, in most setups)
curl -X POST https://test-lab.ai/api/v1/run \
  -H "Authorization: Bearer tl_xxxxx" \
  -H "Content-Type: application/json" \
  -d '{"projectId": YOUR_PROJECT_ID}'

Strict batch behavior

When you pass env and use a batch selector (projectId or label) that fans out across multiple projects, the runner resolves the env name on every project up front. If any project is missing a live env with that name, the whole request rejects with a 404 listing the projects at fault. No jobs queue.

{
  "error": "Environment \"staging\" not found for project(s): 17, 23"
}

That's the safe default. Better to fail loudly than to silently skip the projects without staging and run the rest, because "the test passed" with half your suite quietly missing is the worst outcome.

If the env exists on every targeted project, every plan resolves to that env. Plans share a batch but each one looks up its own project's env config (URLs and notifications can differ across projects under the same env name).

Omitting env

If you don't pass env, every plan in the batch resolves to its own project's default env. This is the right call for the most common case: a CI run that hits prod with no special targeting.

Notification routing

Each env's webhook, Slack, and Teams configs are independent. A typical setup:

The third notify-on mode, failed_excluding_flaky, is useful on noisier envs: a run that failed but where the healing pass identified the failure as flaky won't alert. See Webhooks and Slack Notifications for the full setup, including HMAC signature verification.

Build ID deep links

If a notification carries a buildId (which it will, if your CI passed one in the run request), the Build ID URL template on the env turns it into a clickable link inside the Slack/Teams message. Common templates:

# GitHub commit
https://github.com/owner/repo/commit/{buildId}

# GitLab pipeline
https://gitlab.com/owner/repo/-/pipelines/{buildId}

# Internal dashboard
https://internal.dev/builds/{buildId}

{buildId} is the only substitution token. Whatever string you pass in the run API as buildId lands there.

Common patterns

CI matrix: PR previews + staging gates

Most teams wire up the same suite at three points in the deploy pipeline:

• PR open / push → runs against the per-PR preview env (env: "preview-42" or whichever name your preview infra mints).
• Merge to main → runs against staging once the deploy lands. Slack alerts go to a dev channel.
• Cron schedule → runs against prod every few hours. Failures page oncall.

Same plans, three triggers, three different envs targeted. Add a fourth env (a dev pointing at localhost via the Tunnel Extension) and developers get the same suite locally.

Per-tenant testing

If your app is multi-tenant and the tenant lives in the URL (acme.myapp.com, globex.myapp.com), envs are a clean place to encode it. Spin up acme-prod, globex-prod, and so on. Each one has its own URL and its own notification target if you want per-customer alerting.

Cookie / header isolation

Cookies and headers configured on a staging env don't leak into prod. This matters for bypass tokens, feature flag overrides, and test-only auth: you want them on the dev envs, not in production runs. Configure them on the env that should see them and you're done.

Pipelines and envs

Pipelines (multi-step plans where pre-steps share browser state with the main test) inherit the env value from the run request. Every step in the pipeline targets the same env. The main plan's env URL gets prepended to every step's prompt.

That's the right default for the cases pipelines exist to solve: log in to staging, then test the staging dashboard. Trying to log in on prod and assert on staging halfway through would be both confusing and a security footgun.

Cookie & header inheritance

Test cookies and headers on an environment merge with the project-level and test-plan-level configs the same way the rest of the cookie hierarchy works:

Project cookies → Environment cookies → Test Plan cookies → Runtime cookies (API)

Later levels override earlier ones on a name + domain (cookies) or name (headers) match. So a project-wide tracking_consent cookie can sit at the project level, while env-specific bypass tokens sit on the env, and a test plan can override one cookie for a specific edge case. See Cookie & Header Authentication for the full mechanics.

Geolocation per env

Each env has its own default proxy country. A run inherits its env's proxy country unless the test plan overrides it. So you can have prod running through us, eu-prod running through de, and a prod-jp env running through jp, all in the same project, with the same test plans. See Geolocation for the proxy network and country list.

What an env is not

A few things envs deliberately don't do:

• They aren't account boundaries. Environments live inside a project, and projects belong to one account. They're not a way to share configuration across accounts or teams.
• They aren't credential stores. Sensitive values still live in Credentials, which the AI agent never sees in plaintext. Cookies and headers on an env can reference credentials via {{credentials.name}} substitution.
• They aren't a feature-flag system. If you want to test a flag variation, set the flag-control cookie on the env and let the test run normally.

Migrating from a single-env setup

If you've been running all your plans against one URL (e.g. just prod) and want to add staging:

1. Open the project, click + New Environment, name it staging.
2. Set the staging URL and notification routes (Slack, webhook, etc.).
3. Copy any test cookies / headers your staging env needs (often the same as prod, sometimes a bypass token added).
4. From your CI, add "env": "staging" to the run request that fires after staging deploys.

No test plan changes needed. The same prompts that ran against prod now run against staging when the run is targeted that way.

Next Steps