reporter

Playwright reporter that uploads results to Playwright Reports Server - https://github.com/CyborgTests/playwright-reports-server

Install

npm i -D @cyborgtests/reporter

Basic Configuration

In your playwright.config.ts or playwright.config.js:

  reporter: [
    // blob reporter is required, produced zip would be uploaded
    ['blob', { outputFile: 'test-results/blob.zip' }],
    [
      '@cyborgtests/reporter',
      {
        // true by default. Use this if you need to skip this reporter for some cases (local executions for example)
        enabled: true,
        /**
         * Your server url
         * @see https://github.com/CyborgTests/playwright-reports-server
         */
        url: 'https://your server instance.com',
        // Set token if your server instance has authentication enabled
        token: '1234',
        // Timeout for reporter HTTP requests to finish, default 60000ms, increase if you have slow server and big requests.
        requestTimeout: 60000,
        // Relative path to your blob. Required.
        reportPath: 'test-results/blob.zip',
        // Any custom metadata to attach to this blob (strings)
        resultDetails: {
          branch: process.env.CI_COMMIT_BRANCH,
          foo: 'bar',
          bar: 'baz'
        },
        // Automatically trigger HTML report generation after tests finish. Shards supported. false by default
        triggerReportGeneration: false
      },
    ],
  ],

Then run your tests, if you see [ReporterPlaywrightReportsServer] 🎭 HTML Report is available at: ... - your blob results were successfully sent to server!

Shards

Auto-generation of report after all shards completed is supported. But you must specify testRun and triggerReportGeneration: true:

• In reporter configuration pass

resultDetails: {
  ...
  // testRun required, it should be same for all shards!
  testRun: 'my-awesome-test-run-12'
  ...
},
triggerReportGeneration: true

Reporter passes current shard number and total shard count to server, and after all shards uploaded - report will be generated by server for all blobs in this testRun

Test Quarantine Feature

The Test Quarantine feature allows you to automatically skip tests that have been marked as unstable or flaky directly from the Playwright Reports Server UI. This helps to prevent unstable tests from blocking your CI/CD pipelines.

How It Works

1. Tests are marked as quarantined in the Playwright Reports Server manually via web UI (Analytics page) or automatically (if specified on Settings page)
2. Before test execution, the reporter fetches the list of quarantined tests from the server
3. The reporter writes this list to a local JSON file
4. During test execution, each test is checked against the quarantine list
5. Quarantined tests are automatically skipped with the reason stored in the server

Enabling Test Quarantine

To enable automatic skipping of quarantined tests:

1. Import the extended test fixture from the reporter package
2. Enable skipQuarantinedTests in the reporter configuration
3. Use the extended test fixture in your config

import { defineConfig } from '@playwright/test';
import { test } from '@cyborgtests/reporter';

export default defineConfig({
  reporter: [
    ['blob', { outputFile: 'test-results/blob.zip' }],
    [
      '@cyborgtests/reporter',
      {
        url: 'http://localhost:3000',
        reportPath: 'test-results/blob.zip',
        // Specify the project name to fetch quarantined tests for
        resultDetails: {
          project: 'my-project',
        },
        // Enable test quarantine
        skipQuarantinedTests: true,
        // Optional: Custom path for the quarantine file (default: './quarantine.json')
        quarantineFilePath: './quarantine.json',
      },
    ],
  ],
  // Use the extended test fixture that checks quarantine status
  test: test,
});

The checkQuarantine Fixture

The reporter exports an extended test fixture that includes a checkQuarantine hook which is responsible for checking if a test is quarantined.

Configuration Options

What Happens During Test Execution

When skipQuarantinedTests is enabled:

1. Before tests run (onBegin):

   • The reporter fetches quarantined tests from /api/tests?status=quarantined&project=<your_project>
   • Results are written to quarantineFilePath

2. During each test (checkQuarantine fixture):

   • The fixture reads the quarantine file
   • Checks if testId matches any quarantined test ID
   • If matched skips the test

3. After tests complete (onEnd):

   • Results are uploaded to the server
   • Report is generated if triggerReportGeneration is true

Troubleshooting

Quarantine file not found warning:

[checkQuarantinedTests] Quarantine file not found at ./quarantine.json, proceeding without skipping tests.

Ensure the reporter can fetch quarantined tests from the server (check network and authentication).

Tests not being skipped:

• Verify skipQuarantinedTests: true is set
• Verify you're using the extended test fixture: import { test } from '@cyborgtests/reporter'
• Check the project in resultDetails matches the project name in the server, if you do not have the project - it can be skipped
• Ensure the quarantine file is being generated correctly

All tests being skipped:

• Check that the server's test management thresholds are configured appropriately
• Verify tests are correctly marked as quarantined in the server UI (Analytics page)