@code-pushup/ci

🔎🔬 Quality metrics for your software project. 📉🔍

1. ⚙️ Configure what you want to track using your favourite tools.
2. 🤖 Integrate it in your CI.
3. 🌈 Visualize reports in a beautiful dashboard.

This package exports provider-agnostic core logic for running Code PushUp in CI pipelines. It serves as the base for the following provider integrations:

Setup

npm install @code-pushup/ci

yarn add @code-pushup/ci

pnpm add @code-pushup/ci

Usage

The runInCI function implements the full CI flow:

import { runInCI } from '@code-pushup/ci';

const result = await runInCI(
  {
    /* Git refs */
  },
  {
    /* Provider API client */
  },
  {
    /* Options */
  },
);

Parameters

Git refs

For each CI run, you must pass in the commit SHA and Git ref (e.g. main) of what was pushed. These values can be detected from the CI environment, the details depend on which provider is being used.

If only the head is supplied, then Code PushUp will collect a new report and optionally upload it to portal (depending on your Code PushUp config). If triggered by a pull request, then specify the base ref as well. This will additionally compare reports from both source and target branches and post a comment to the PR.

Provider API client

The PR flow requires interacting with the Git provider's API to post a comparison comment. Wrap these requests in functions and pass them in as an object which configures the provider.

A Comment object has the following required properties:

Options

Optionally, you can override default options for further customization:

[^1]: By default, the code-pushup.config file is autodetected as described in @code-pushup/cli docs.

[^2]: The {task} pattern is replaced with the task value, so the default behaviour is to list projects using npx nx show projects --with-target=code-pushup --json. The nxProjectsFilter options gives Nx users the flexibility to filter projects in alternative ways supported by the Nx CLI (e.g. --affected, --projects, --exclude, --type) - refer to options in Nx docs for details.

Standalone mode

By default, it is assumed that Code PushUp is set up to run on the whole repo with one command (standalone mode). If you want to run Code PushUp on multiple projects separately, you should enable monorepo mode.

Standalone result

In standalone mode, the resolved object will include paths to report files (JSON and Markdown formats), as well as diff files, comment ID and new issues in case of PR comparisons.

const result = await runInCI(refs, api);

if (result.mode === 'standalone') {
  const {
    // output files, can be uploaded as job artifact
    files: { current, comparison },
    // ID of created/updated PR comment
    commentId,
    // array of source code issues, can be used to annotate changed files in PR
    newIssues,
  } = result;
}

Monorepo mode

For monorepo setups, Code PushUp reports can be collected and compared individually per project. All project comparisons are then combined into a single PR comment.

Use the monorepo option to activate monorepo mode:

await runInCI(refs, api, {
  monorepo: true,
});

The runInCI function will try to detect which monorepo tool you're using from the file system. The following tools are supported out of the box:

• Nx
• Turborepo
• Yarn workspaces
• pnpm workspace
• npm workspaces

If you're using one of these tools, you can also skip auto-detection by setting monorepo option to 'nx', 'turbo', 'yarn', 'pnpm' or 'npm'.

If none of these tools are detected, then the fallback is to run Code PushUp in all folders which have a package.json file. If that's not what you want, then you can also configure folder patterns using the projects option (supports globs):

await runInCI(refs, api, {
  monorepo: true,
  projects: ['frontend', 'backend/*'],
});

Based on which monorepo tool is used, Code PushUp CLI commands will be executed using a package.json script, Nx target, Turbo task, or binary executable (as fallback). By default, these are expected to be called code-pushup, but you can override the name using the task option:

await runInCI(refs, api, {
  monorepo: 'nx',
  task: 'analyze', // custom Nx target
});

Parallel tasks

By default, tasks are run sequentially for each project in the monorepo. The parallel option enables parallel execution for tools which support it (Nx, Turborepo, PNPM, Yarn 2+).

await runInCI(refs, api, {
  monorepo: true,
  parallel: true,
});

The maximum number of concurrent tasks can be set by passing in a number instead of a boolean:

await runInCI(refs, api, {
  monorepo: true,
  parallel: 3,
});

Monorepo result

In monorepo mode, the resolved object includes the merged diff at the top-level, as well as a list of projects. Each project has its own report files and issues.

const result = await runInCI(refs, api);

if (result.mode === 'monorepo') {
  const {
    // array of objects with result for each project
    projects,
    // ID of created/updated PR comment
    commentId,
    // merged report-diff.md used in PR comment, can also be uploaded as job artifact
    files: { comparison },
  } = result;

  for (const project of projects) {
    const {
      // detected project name (from package.json, project.json or folder name)
      name,
      // output files, can be uploaded as job artifacts
      files: { current, comparison },
      // array of source code issues, can be used to annotate changed files in PR
      newIssues,
    } = project;
  }
}

Advanced usage

Faster CI runs with configPatterns

By default, the print-config command is run sequentially for each project in order to reliably detect how code-pushup is configured - specifically, where to read output files from (persist config) and whether the portal may be used as a cache (upload config). This allows for each project to be configured in its own way without breaking anything, but for large monorepos these extra code-pushup print-config executions can accumulate and significantly slow down CI pipelines.

As a more scalable alternative, configPatterns may be provided. A user declares upfront how every project is configured, which allows print-config to be skipped. It's the user's responsibility to ensure this configuration holds for every project (it won't be checked). The configPatterns support string interpolation, substituting {projectName} with each project's (slugified) name. Other than that, each project's code-pushup.config must have exactly the same persist and upload configurations.

await runInCI(refs, api, {
  monorepo: true,
  configPatterns: {
    persist: {
      outputDir: '.code-pushup/{projectName}',
      filename: 'report',
      format: ['json', 'md'],
    },
    // optional: will use portal as cache when comparing reports in PRs
    upload: {
      server: 'https://api.code-pushup.example.com/graphql',
      apiKey: 'cp_...',
      organization: 'example',
      project: '{projectName}',
    },
  },
});

Search latest commits for previous report

When comparing reports, the report for the base branch can be cached. If a project has an upload configuration, then the Portal API is queried for a report matching that commit. If no such report was uploaded, then the report is looked up in CI artifacts (implemented by downloadReportArtifact in ProviderApiClient). If there's no report to be found, then the base branch is checked out and the previous report is collected.

In some scenarios, there may not be a report for the latest commit in main branch, but some other recent commit may have a usable report - e.g. if nxProjectsFilter is used with --affected flag. In that case, the searchCommits option can be enabled. Then a limited number of recent commits in the main branch will be checked.

await runInCI(refs, api, {
  monorepo: 'nx',
  nxProjectsFilter: '--with-target=code-pushup --affected',
  // checks 10 most recent commits by default
  searchCommits: true,
  // optionally, number of searched commits may be extended up to 100
  // searchCommits: 30
});