Webhooks
Receive HTTP notifications when test runs complete, with HMAC-signed payloads
Webhooks
Webhooks notify your server when test runs complete, eliminating the need for polling. Configuration lives on each environment, so a project's prod and staging envs can post to different endpoints with different gating.
Setup
- Go to Dashboard → Projects → click the project.
- Click the Edit (pencil) icon on the environment you want to wire up.
- In the Notifications section, paste your endpoint into HTTP Webhook URL.
- Choose Notify on:
- All runs – every terminal status (default for new webhooks, preserves CI integrations that need every result)
- Failed runs only – only
failed,incomplete,credits_exhausted,timeout
- Save. A Webhook Secret is auto-generated and shown once. You can rotate it later from the same screen.
Webhooks are configured per environment, not per project. The same project's prod env can post to your production alerting webhook, while staging posts to a noisier dev-loop webhook. See Slack Notifications for chat-channel routing along the same lines.
Webhook Payload
When a test reaches a terminal state, Test-Lab POSTs the following JSON to your webhook URL:
{
"event": "run.completed",
"jobId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"testPlanId": YOUR_TEST_PLAN_ID,
"buildId": "YOUR_BUILD_ID",
"status": "completed",
"result": {
"summary": {
"result": "passed",
"notes": "All acceptance criteria verified."
},
"acceptanceCriteria": [
{
"description": "Homepage loads",
"status": "passed",
"evidence": ["Page loaded in 1.2s"]
}
],
"steps": [...],
"issues": []
},
"error": null,
"completedAt": "2024-01-15T10:32:05.000Z"
}Event Types
| Event | Status | Meaning |
|---|---|---|
run.completed | completed | Run finished, all assertions passed |
run.failed | failed | Run finished, assertions caught a real regression |
run.incomplete | incomplete | Agent ran but couldn't produce a final report |
run.credits_exhausted | credits_exhausted | Account ran out of credits mid-run |
run.timeout | timeout | Run hung past the 20-minute deadline (no executor callback received) |
Treat incomplete, credits_exhausted, and timeout as distinct from failed. A failed run executed and caught a real regression. The other three mean the scenario was never actually tested. Don't show them as green in your dashboards.
For run.timeout, result is always null (no executor data was uploaded). All other events carry the structured executor output.
Headers
| Header | Description |
|---|---|
Content-Type | application/json |
X-TestLab-Signature | HMAC-SHA256 signature |
X-TestLab-Event | Event type |
User-Agent | TestLab-Webhook/1.0 |
Verifying Signatures
Always verify webhook signatures to ensure requests are from Test-Lab.
The X-TestLab-Signature header contains:
sha256=<hex-encoded-hmac>Node.js Example
import crypto from 'crypto';
function verifySignature(payload, signature, secret) {
const expectedSignature = 'sha256=' + crypto
.createHmac('sha256', secret)
.update(payload)
.digest('hex');
// Timing-safe comparison
return crypto.timingSafeEqual(
Buffer.from(signature),
Buffer.from(expectedSignature)
);
}
// Express.js handler
app.post('/webhook', express.raw({ type: 'application/json' }), (req, res) => {
const signature = req.headers['x-testlab-signature'];
const payload = req.body.toString();
if (!verifySignature(payload, signature, process.env.WEBHOOK_SECRET)) {
return res.status(401).send('Invalid signature');
}
const event = JSON.parse(payload);
console.log(`Test ${event.jobId}: ${event.result?.summary?.result}`);
res.status(200).send('OK');
});Python Example
import hmac
import hashlib
from flask import Flask, request
app = Flask(__name__)
def verify_signature(payload: bytes, signature: str, secret: str) -> bool:
expected = 'sha256=' + hmac.new(
secret.encode(),
payload,
hashlib.sha256
).hexdigest()
return hmac.compare_digest(signature, expected)
@app.route('/webhook', methods=['POST'])
def webhook():
signature = request.headers.get('X-TestLab-Signature')
payload = request.get_data()
if not verify_signature(payload, signature, os.environ['WEBHOOK_SECRET']):
return 'Invalid signature', 401
event = request.json
print(f"Test {event['jobId']}: {event['result']['summary']['result']}")
return 'OK', 200Go Example
package main
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"io"
"net/http"
)
func verifySignature(payload []byte, signature, secret string) bool {
mac := hmac.New(sha256.New, []byte(secret))
mac.Write(payload)
expected := "sha256=" + hex.EncodeToString(mac.Sum(nil))
return hmac.Equal([]byte(signature), []byte(expected))
}
func webhookHandler(w http.ResponseWriter, r *http.Request) {
payload, _ := io.ReadAll(r.Body)
signature := r.Header.Get("X-TestLab-Signature")
if !verifySignature(payload, signature, os.Getenv("WEBHOOK_SECRET")) {
http.Error(w, "Invalid signature", http.StatusUnauthorized)
return
}
// Process event...
w.WriteHeader(http.StatusOK)
}Payload Fields
| Field | Type | Description |
|---|---|---|
event | string | One of run.completed, run.failed, run.incomplete, run.credits_exhausted, run.timeout |
jobId | string | Job UUID |
testPlanId | number | null | Test plan ID (null for ad-hoc runs) |
buildId | string | null | CI build ID if provided when triggering the run |
status | string | One of completed, failed, incomplete, credits_exhausted, timeout |
result | object | null | Structured executor output (steps, criteria, summary, issues). Always null for run.timeout |
error | string | null | Error message if execution failed or the run timed out |
completedAt | string | ISO timestamp of when the terminal state was recorded |
Best Practices
Respond Quickly
Return a 200 status within 30 seconds. Process events asynchronously if needed:
app.post('/webhook', (req, res) => {
// Respond immediately
res.status(200).send('OK');
// Process asynchronously
processWebhook(req.body).catch(console.error);
});Handle Retries
Webhooks are delivered once. If your server is down, you'll miss the event.
Store the Secret Securely
Never commit your webhook secret to version control. Use environment variables:
WEBHOOK_SECRET=your-secret-hereTesting Webhooks
Local Development
Use tools like ngrok to expose your local server:
ngrok http 3000
# Use the ngrok URL as your webhook URLVerify Configuration
After setting up, trigger a test and check your webhook endpoint receives the payload.
Troubleshooting
Not Receiving Webhooks
- Verify the webhook URL is correct and publicly accessible
- Check your server logs for incoming requests
- Ensure your server returns
200status
Signature Verification Failing
- Use the raw request body (before JSON parsing)
- Ensure you're using the correct webhook secret
- Check for encoding issues
Slack and Teams
For chat-channel notifications with rich formatting (status emojis, clickable build links), see Slack Notifications. Slack is configured side-by-side with HTTP Webhooks on the same env, and each channel has its own notify_on setting so you can mix and match (e.g. webhook on every run, Slack on failures only).