Skip to content

Connect Plune to your CI

Your CI needs one secret: PLUNE_TOKEN, an API token from Plune (Account → API tokens), saved in the CI’s secret store. The keys of your storage bucket never go into CI. They stay in Plune, and the CI gets a link to write one file at a time, for 15 minutes — see What the CI sees.

There are three ways in. Only the live reporter is a package of its own; the other two read the report your framework already writes:

WayForHow
Live reporterPlaywright@plune-ai/playwright in playwright.config.ts — sends results from inside the run, no step after it
GitHub Actionany report, on GitHub Actionsone step after the tests: plune-ai/report-action@v1
plune run importany report, in any CInpx -y @plune-ai/cli run import <report or folder>

The report is what your framework already writes: a JUnit XML file, a folder of JUnit XML files (Maven, Gradle; 0.32.0), Playwright’s JSON report, an allure-results folder, or a k6 summary or Lighthouse report (0.33.0). The format is detected.

The GitHub Action is a thin wrapper over @plune-ai/cli: the step runs plune run import from a pinned CLI version, and nothing is installed into your project. It is not on the GitHub Marketplace yet, so use it by repository and tag, plune-ai/report-action@v1, as below.

- uses: plune-ai/report-action@v1
if: ${{ always() }}
with:
token: ${{ secrets.PLUNE_TOKEN }}
report: reports/junit.xml

A failed test makes the test step fail, and GitHub Actions skips the steps after a failed one. Failed tests are the reason to report, so the report step carries if: ${{ always() }}. A red test does not fail the import itself. To fail the job on the platform’s verdict instead, set fail-on-failures: 'true' on the action, or --fail-on-failures on the command — see Gate a release on the platform’s verdict. A report step that must never turn the build red takes continue-on-error: true; leave it off a step with fail-on-failures, whose job is to turn it red.

Secrets are not given to workflows triggered by pull requests from forks; the step then fails with a message saying the token is empty.

In any other CI, run the import after the tests whatever they returned, with PLUNE_TOKEN in the environment:

Terminal window
npx -y @plune-ai/cli run import reports/junit.xml

The CLI needs Node.js 20 or newer. Run it where the report was written: a file a test attached is found by a path counted from the report’s folder.

- run: pytest --junitxml=reports/junit.xml -o junit_logging=system-out
- uses: plune-ai/report-action@v1
if: ${{ always() }}
with:
token: ${{ secrets.PLUNE_TOKEN }}
report: reports/junit.xml

In any CI:

Terminal window
pytest --junitxml=reports/junit.xml -o junit_logging=system-out
npx -y @plune-ai/cli run import reports/junit.xml

junit_logging=system-out keeps what a test prints in the report. A test that saves a file as reports/shots/cart.png and prints [[ATTACHMENT|shots/cart.png]] on a line of its own gets that file sent with its result: the path counts from the report’s folder, reports — see Attachments in JUnit XML.

Surefire writes a JUnit XML file per test class (TEST-<class>.xml) into target/surefire-reports. Point the import at that folder and the files arrive as one run; nothing to add to the pom.xml. Needs @plune-ai/cli 0.32.0 or newer, which the action’s default cli-version is.

- run: mvn -B test
- uses: plune-ai/report-action@v1
if: ${{ always() }}
with:
token: ${{ secrets.PLUNE_TOKEN }}
report: target/surefire-reports

In any CI:

Terminal window
mvn -B test
npx -y @plune-ai/cli run import target/surefire-reports

Every *.xml directly in the folder that is a JUnit report is read. Other files (Surefire’s *.txt) are not, and an *.xml that is not JUnit is skipped with a line in the log.

  • Keys. A test is matched to its case by classname#name, and Surefire’s names have no parentheses: cart.CartTest#addsAnItem. Gradle and Allure keep them, so a project that moves between Surefire and Gradle, or Surefire and Allure, meets its cases under other keys.
  • Duration. Surefire writes no timestamp on a suite, so a Maven test’s span ends when the import reads the report; its duration is the test’s own time. That needs @plune-ai/cli 0.32.1 or newer.
  • Files. A file reaches a test only through a [[ATTACHMENT|path]] line in that test’s own output (System.out.println), and the path counts from target/surefire-reports, so the file has to be below it. With redirectTestOutputToFile set to true, Surefire moves that output out of the XML, and the line does not reach the report.

For a test’s steps, and files attached with Allure.addAttachment(...), use Allure. With allure.results.directory=target/allure-results set, point the import at target/allure-results instead; without it, Allure writes allure-results in the working folder — see Java, JUnit 5 (allure-jupiter).

Gradle writes a JUnit XML file per test class into build/test-results/test. Point the import at that folder; nothing to add to the build. Needs @plune-ai/cli 0.32.0 or newer, as for Maven.

- run: ./gradlew test
- uses: plune-ai/report-action@v1
if: ${{ always() }}
with:
token: ${{ secrets.PLUNE_TOKEN }}
report: build/test-results/test

In any CI:

Terminal window
./gradlew test
npx -y @plune-ai/cli run import build/test-results/test

The subfolder Gradle writes there (binary/) is not read.

  • Keys. Gradle keeps the method’s parentheses: cart.CartTest#addsAnItem(). Surefire drops them, so a move to Maven meets the cases under other keys.
  • Files. A [[ATTACHMENT|path]] line counts from build/test-results/test, and reaches its test only with reports.junitXml.outputPerTestCase = true in the test task. By default Gradle keeps a test’s output at suite level, where the import does not look for the line.

With Allure, set the test task’s system property allure.results.directory to build/allure-results and point the import there; without it, Allure writes allure-results in the working folder — Java, JUnit 5 (allure-jupiter).

Jest writes JUnit XML through the jest-junit reporter, installed beside Jest:

- run: npx jest --reporters=default --reporters=jest-junit
- uses: plune-ai/report-action@v1
if: ${{ always() }}
with:
token: ${{ secrets.PLUNE_TOKEN }}
report: junit.xml

In any CI:

Terminal window
npx jest --reporters=default --reporters=jest-junit
npx -y @plune-ai/cli run import junit.xml

Vitest writes JUnit XML itself. Keep the default reporter beside junit, so one run prints to the log and writes the file:

- run: npx vitest run --reporter=default --reporter=junit --outputFile.junit=junit.xml
- uses: plune-ai/report-action@v1
if: ${{ always() }}
with:
token: ${{ secrets.PLUNE_TOKEN }}
report: junit.xml

In any CI:

Terminal window
npx vitest run --reporter=default --reporter=junit --outputFile.junit=junit.xml
npx -y @plune-ai/cli run import junit.xml

Add the JunitXml.TestLogger package to the test project; dotnet test then writes a JUnit XML file.

# bash. Under pwsh (the default shell on Windows runners) write $env:RESULTS_DIR in run:, or set shell: bash.
- run: dotnet test --logger "junit;LogFilePath=$RESULTS_DIR/report.xml"
env:
RESULTS_DIR: ${{ github.workspace }}/test-results
- uses: plune-ai/report-action@v1
if: ${{ always() }}
with:
token: ${{ secrets.PLUNE_TOKEN }}
report: test-results/report.xml

In any CI (bash):

Terminal window
export RESULTS_DIR="$PWD/test-results"
dotnet test --logger "junit;LogFilePath=$RESULTS_DIR/report.xml"
npx -y @plune-ai/cli run import test-results/report.xml

To attach a file, have the test save it in the report’s folder and write [[ATTACHMENT|cart.png]] through ITestOutputHelper.WriteLine: xUnit v2 does not capture Console.WriteLine, so a line printed that way never reaches the report.

Two routes. The live reporter sends results from inside the Playwright run: a batch once 500 results are waiting (PLUNE_BATCH_SIZE sets fewer), and the rest when the suite ends, so no import step follows it. Install it beside Playwright and add it to the config:

Terminal window
npm install --save-dev @plune-ai/playwright
playwright.config.ts
reporter: [['list'], ['@plune-ai/playwright']],
- run: npx playwright test
env:
PLUNE_TOKEN: ${{ secrets.PLUNE_TOKEN }}
PLUNE_RUN: ${{ github.run_id }}-${{ github.run_attempt }}

Shards land in one run when every shard has the same PLUNE_RUN and PLUNE_SHARED_RUN: "1". Then no shard closes the run, and one job after them closes it with plune run finish <id>. Setup, the other variables and shards are in Report Playwright results as they run.

Or, with no package added, import Playwright’s JSON report after the run:

- run: npx playwright test --reporter=json
env:
PLAYWRIGHT_JSON_OUTPUT_NAME: playwright-report.json
- uses: plune-ai/report-action@v1
if: ${{ always() }}
with:
token: ${{ secrets.PLUNE_TOKEN }}
report: playwright-report.json

In any CI:

Terminal window
PLAYWRIGHT_JSON_OUTPUT_NAME=playwright-report.json npx playwright test --reporter=json
npx -y @plune-ai/cli run import playwright-report.json

The import finds each test’s screenshots where the report says they are, so run it where the report was written.

From @plune-ai/cli 0.33.0, plune run import also reads a k6 end-of-test summary and a Lighthouse JSON report. Plune runs no load or page-speed tests of its own: it takes the results of the tools that do, as a run.

Terminal window
k6 run --summary-export=summary.json script.js
npx -y @plune-ai/cli run import summary.json --create
lighthouse https://example.com --output=json --output-path=lighthouse.json
npx -y @plune-ai/cli run import lighthouse.json --create --min-score performance=80 --min-score seo=95

Each is recognised by its own keys, never by the file name; --format k6 or --format lighthouse says it outright.

  • A k6 threshold is one result, named after the metric and the expression (http_req_duration p(95)<500): passed when k6 says it held, failed when it was crossed, with the measured values as the message. A summary with no thresholds is refused with exit code 2 and nothing is imported.
  • A Lighthouse category is one result, named after the category, the page and the form factor (performance (https://example.com/, mobile)), with the score from 0 to 100. It passes at or above the bar: --min-score sets the bar per category, and a category you do not name needs 90. The GitHub Action has no input for --min-score, so through it every category needs 90.

As with any report, a result reaches the run only once it has a case. A first import finds none, so --create offers them to the review queue, and imports record them once a person has approved them there.

From then on, a crossed threshold or a category under its bar is a failed result, and only that: the run finishes as usual and the command exits 0. To fail the job on it, add --fail-on-failures — see Gate a release on the platform’s verdict. A result with no case yet does not count towards that verdict.

k6 itself exits 99 when a threshold is crossed, so give the import a step of its own, with if: ${{ always() }} as above.

When the jobs of a matrix import k6 summaries into one run under one --key, each job names its file apart, because a threshold is known by the file’s base name, not its folder:

- run: k6 run --summary-export=summary-${{ matrix.scenario }}.json script.js

Give every job the same key with the attempt in it, so that Re-run failed jobs starts a new run instead of joining the first attempt’s. On the action that is key, plus shared-run: 'true' on every job but the one that ends the run, or shard: k/N on each shard. On the command it is --key with PLUNE_SHARED_RUN=1 or PLUNE_SHARD=k/N — see Several jobs, one run. The action’s other inputs (run-title, env, labels, create, fail-on-missing-attachments, …) are listed in its README.