@checkly/playwright-reporter

Official Playwright reporter for Checkly. Automatically upload test results, screenshots, videos, and traces to gain visibility into your end-to-end tests.

Fully compatible with Playwright's built-in JSON reporter - use it as a drop-in replacement that adds Checkly integration.

Installation

npm install --save-dev @checkly/playwright-reporter

Requirements:

• Node.js 20.19+ or 22.12+
• Playwright >= 1.40.0
• A Checkly account

Quick Start

1. Get Your Credentials

1. Go to Account Settings > General to find your Account ID
2. Go to User Settings > API Keys to create an API key

2. Configure Playwright

Add the reporter to your playwright.config.ts:

import { defineConfig } from '@playwright/test';
import { createChecklyReporter } from '@checkly/playwright-reporter';

export default defineConfig({
  reporter: [
    ['list'],
    createChecklyReporter(),
  ],
});

Tip: Using createChecklyReporter() provides full intellisense for configuration options. Alternatively, use the array syntax: ['@checkly/playwright-reporter', { ... }]

3. Set Environment Variables

export CHECKLY_API_KEY=your_api_key
export CHECKLY_ACCOUNT_ID=your_account_id

Or pass them inline:

CHECKLY_API_KEY=... CHECKLY_ACCOUNT_ID=... npx playwright test

4. Run Tests

npx playwright test

You'll see a session URL in the output:

View test results: https://app.checklyhq.com/test-sessions/abc123

[...test execution...]

✓ Check the results: https://app.checklyhq.com/test-sessions/abc123

Configuration Options

import { createChecklyReporter } from '@checkly/playwright-reporter';

createChecklyReporter({
  outputDir: 'test-results',
  sessionName: 'My Test Suite',
  dryRun: false,
  verbose: false,
  showProgress: true,
  showSummaryTable: true,
})

Output files (written to outputDir):

• checkly-report.json - JSON test report
• checkly-report.zip - ZIP archive with report and assets

Environment Variables

Environment variables take precedence over options passed in the config.

What Gets Uploaded

• Test results and status (passed/failed/flaky)
• Execution duration and timing
• Screenshots (on failure or explicit capture)
• Video recordings
• Playwright traces
• Console logs and network requests (extracted from traces)

Secret Scrubbing

The reporter automatically scrubs sensitive values from JSON reports, trace files, attachments and logs before upload.

By default, any environment variable whose name starts with CHECKLY_SECRET_ is scrubbed. You can configure additional scrubbing strategies:

createChecklyReporter({
  scrubbing: {
    envVars: ['DB_PASSWORD', 'STRIPE_KEY'],
    autoDetect: true,
    prefix: 'CHECKLY_SECRET_',
    replacement: '[REDACTED]',
  },
})

Set scrubbing: false to disable scrubbing entirely.

Flaky Test Detection

The reporter automatically detects flaky tests:

• Flaky test: A test that failed initially but passed after retry
• Degraded status: Set when there are flaky tests but no complete failures
• Overall status: passed if all tests eventually pass, failed otherwise

Running Locally (No Upload)

For local development without uploading to Checkly:

# Simply don't set CHECKLY_API_KEY or CHECKLY_ACCOUNT_ID
npx playwright test

What happens:

• The reporter prints that credentials are missing and skips the upload
• Tests run normally
• With outputDir set, the reporter writes a scrubbed checkly-report.json there

To get checkly-report.zip locally, use dry run mode.

Dry Run Mode

Create local JSON and ZIP files without uploading:

createChecklyReporter({
  dryRun: true,
})

Conditional dry run (skip uploads when no credentials):

createChecklyReporter({
  dryRun: !process.env.CHECKLY_API_KEY,
})

CI/CD Integration

GitHub Actions

name: Playwright Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run Playwright tests
        env:
          CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
          CHECKLY_ACCOUNT_ID: ${{ secrets.CHECKLY_ACCOUNT_ID }}
        run: npx playwright test

GitLab CI

test:
  image: mcr.microsoft.com/playwright:v1.57.0-jammy
  stage: test
  script:
    - npm ci
    - npx playwright test
  variables:
    CHECKLY_API_KEY: $CHECKLY_API_KEY
    CHECKLY_ACCOUNT_ID: $CHECKLY_ACCOUNT_ID

Setting up variables:

1. Go to your GitLab project
2. Navigate to Settings > CI/CD > Variables
3. Add CHECKLY_API_KEY (check "Mask variable")
4. Add CHECKLY_ACCOUNT_ID (check "Mask variable")

Multiple Reporters

Combine with other Playwright reporters:

import { createChecklyReporter } from '@checkly/playwright-reporter';

export default defineConfig({
  reporter: [
    createChecklyReporter(),
    ['html', { outputFolder: 'playwright-report' }],
    ['list'],
    ['junit', { outputFile: 'test-results/junit.xml' }],
  ],
});

Understanding Test Sessions

The reporter creates suite-level test sessions, not individual test file results.

When you run npx playwright test:

1. One test session is created, named after your directory
2. One test result for the entire run
3. All assets uploaded together in a single ZIP file

Benefits:

• Consolidated view of your entire test suite
• Efficient storage
• Track overall pass/fail rates over time

Security

Always use environment variables for credentials. Never commit API keys to version control.

// DON'T DO THIS
createChecklyReporter({
  apiKey: 'chk_api_...',  // Hardcoded credentials!
})

// DO THIS
createChecklyReporter()  // Reads from environment

Use the Secret Scrubbing feature to ensure sensitive values don't leak into uploaded reports and traces.

Documentation

For detailed documentation, visit checklyhq.com/docs/detect/testing/playwright-reporter.

License

Apache 2.0