Connect Plune to your CI
Your CI needs one secret: PLUNE_TOKEN, an API token from
Plune (Account → API tokens), saved in the CI’s secret store. The keys of your
storage bucket never go into CI. They stay in Plune, and the CI gets a link to
write one file at a time, for 15 minutes — see What the CI sees.
There are three ways in. Only the live reporter is a package of its own; the other two read the report your framework already writes:
| Way | For | How |
|---|---|---|
| Live reporter | Playwright | @plune-ai/playwright in playwright.config.ts — sends results from inside the run, no step after it |
| GitHub Action | any report, on GitHub Actions | one step after the tests: plune-ai/report-action@v1 |
plune run import | any report, in any CI | npx -y @plune-ai/cli run import <report or folder> |
The report is what your framework already writes: a JUnit XML file, a folder of JUnit XML files
(Maven, Gradle; 0.32.0), Playwright’s JSON report, an allure-results
folder, or a k6 summary or Lighthouse report (0.33.0).
The format is detected.
The GitHub Action is a thin wrapper over
@plune-ai/cli: the step runs plune run import from a pinned CLI version, and nothing is installed
into your project. It is not on the GitHub Marketplace yet, so use it by repository and tag,
plune-ai/report-action@v1, as below.
The report step runs even when tests fail
Section titled “The report step runs even when tests fail”- uses: plune-ai/report-action@v1 if: ${{ always() }} with: token: ${{ secrets.PLUNE_TOKEN }} report: reports/junit.xmlA failed test makes the test step fail, and GitHub Actions skips the steps after a failed one. Failed
tests are the reason to report, so the report step carries if: ${{ always() }}. A red test does not
fail the import itself. To fail the job on the platform’s verdict instead, set
fail-on-failures: 'true' on the action, or --fail-on-failures on the command — see
Gate a release on the platform’s verdict.
A report step that must never turn the build red takes
continue-on-error: true; leave it off a step with fail-on-failures, whose job is to turn it red.
Secrets are not given to workflows triggered by pull requests from forks; the step then fails with a message saying the token is empty.
In any other CI, run the import after the tests whatever they returned, with PLUNE_TOKEN in the
environment:
npx -y @plune-ai/cli run import reports/junit.xmlThe CLI needs Node.js 20 or newer. Run it where the report was written: a file a test attached is found by a path counted from the report’s folder.
pytest
Section titled “pytest”- run: pytest --junitxml=reports/junit.xml -o junit_logging=system-out- uses: plune-ai/report-action@v1 if: ${{ always() }} with: token: ${{ secrets.PLUNE_TOKEN }} report: reports/junit.xmlIn any CI:
pytest --junitxml=reports/junit.xml -o junit_logging=system-outnpx -y @plune-ai/cli run import reports/junit.xmljunit_logging=system-out keeps what a test prints in the report. A test that saves a file as
reports/shots/cart.png and prints [[ATTACHMENT|shots/cart.png]] on a line of its own gets that file
sent with its result: the path counts from the report’s folder, reports — see
Attachments in JUnit XML.
Java with Maven
Section titled “Java with Maven”Surefire writes a JUnit XML file per test class (TEST-<class>.xml) into target/surefire-reports.
Point the import at that folder and the files arrive as one run; nothing to add to the pom.xml. Needs
@plune-ai/cli 0.32.0 or newer, which the action’s default cli-version is.
- run: mvn -B test- uses: plune-ai/report-action@v1 if: ${{ always() }} with: token: ${{ secrets.PLUNE_TOKEN }} report: target/surefire-reportsIn any CI:
mvn -B testnpx -y @plune-ai/cli run import target/surefire-reportsEvery *.xml directly in the folder that is a JUnit report is read. Other files (Surefire’s *.txt)
are not, and an *.xml that is not JUnit is skipped with a line in the log.
- Keys. A test is matched to its case by
classname#name, and Surefire’s names have no parentheses:cart.CartTest#addsAnItem. Gradle and Allure keep them, so a project that moves between Surefire and Gradle, or Surefire and Allure, meets its cases under other keys. - Duration. Surefire writes no
timestampon a suite, so a Maven test’s span ends when the import reads the report; its duration is the test’s owntime. That needs@plune-ai/cli0.32.1 or newer. - Files. A file reaches a test only through a
[[ATTACHMENT|path]]line in that test’s own output (System.out.println), and the path counts fromtarget/surefire-reports, so the file has to be below it. WithredirectTestOutputToFileset totrue, Surefire moves that output out of the XML, and the line does not reach the report.
For a test’s steps, and files attached with Allure.addAttachment(...), use Allure. With
allure.results.directory=target/allure-results set, point the import at target/allure-results
instead; without it, Allure writes allure-results in the working folder — see
Java, JUnit 5 (allure-jupiter).
Java with Gradle
Section titled “Java with Gradle”Gradle writes a JUnit XML file per test class into build/test-results/test. Point the import at that
folder; nothing to add to the build. Needs @plune-ai/cli 0.32.0 or newer, as for Maven.
- run: ./gradlew test- uses: plune-ai/report-action@v1 if: ${{ always() }} with: token: ${{ secrets.PLUNE_TOKEN }} report: build/test-results/testIn any CI:
./gradlew testnpx -y @plune-ai/cli run import build/test-results/testThe subfolder Gradle writes there (binary/) is not read.
- Keys. Gradle keeps the method’s parentheses:
cart.CartTest#addsAnItem(). Surefire drops them, so a move to Maven meets the cases under other keys. - Files. A
[[ATTACHMENT|path]]line counts frombuild/test-results/test, and reaches its test only withreports.junitXml.outputPerTestCase = truein the test task. By default Gradle keeps a test’s output at suite level, where the import does not look for the line.
With Allure, set the test task’s system property allure.results.directory to build/allure-results
and point the import there; without it, Allure writes allure-results in the working folder —
Java, JUnit 5 (allure-jupiter).
Jest writes JUnit XML through the jest-junit reporter, installed beside Jest:
- run: npx jest --reporters=default --reporters=jest-junit- uses: plune-ai/report-action@v1 if: ${{ always() }} with: token: ${{ secrets.PLUNE_TOKEN }} report: junit.xmlIn any CI:
npx jest --reporters=default --reporters=jest-junitnpx -y @plune-ai/cli run import junit.xmlVitest
Section titled “Vitest”Vitest writes JUnit XML itself. Keep the default reporter beside junit, so one run prints to the
log and writes the file:
- run: npx vitest run --reporter=default --reporter=junit --outputFile.junit=junit.xml- uses: plune-ai/report-action@v1 if: ${{ always() }} with: token: ${{ secrets.PLUNE_TOKEN }} report: junit.xmlIn any CI:
npx vitest run --reporter=default --reporter=junit --outputFile.junit=junit.xmlnpx -y @plune-ai/cli run import junit.xml.NET (xUnit)
Section titled “.NET (xUnit)”Add the JunitXml.TestLogger package to the test
project; dotnet test then writes a JUnit XML file.
# bash. Under pwsh (the default shell on Windows runners) write $env:RESULTS_DIR in run:, or set shell: bash.- run: dotnet test --logger "junit;LogFilePath=$RESULTS_DIR/report.xml" env: RESULTS_DIR: ${{ github.workspace }}/test-results- uses: plune-ai/report-action@v1 if: ${{ always() }} with: token: ${{ secrets.PLUNE_TOKEN }} report: test-results/report.xmlIn any CI (bash):
export RESULTS_DIR="$PWD/test-results"dotnet test --logger "junit;LogFilePath=$RESULTS_DIR/report.xml"npx -y @plune-ai/cli run import test-results/report.xmlTo attach a file, have the test save it in the report’s folder and write [[ATTACHMENT|cart.png]]
through ITestOutputHelper.WriteLine: xUnit v2 does not capture Console.WriteLine, so a line printed
that way never reaches the report.
Playwright
Section titled “Playwright”Two routes. The live reporter sends results from inside the Playwright run: a batch once 500
results are waiting (PLUNE_BATCH_SIZE sets fewer), and the rest when the suite ends, so no import
step follows it. Install it beside Playwright and add it to the config:
npm install --save-dev @plune-ai/playwrightreporter: [['list'], ['@plune-ai/playwright']],- run: npx playwright test env: PLUNE_TOKEN: ${{ secrets.PLUNE_TOKEN }} PLUNE_RUN: ${{ github.run_id }}-${{ github.run_attempt }}Shards land in one run when every shard has the same PLUNE_RUN and PLUNE_SHARED_RUN: "1". Then
no shard closes the run, and one job after them closes it with plune run finish <id>. Setup, the
other variables and shards are in
Report Playwright results as they run.
Or, with no package added, import Playwright’s JSON report after the run:
- run: npx playwright test --reporter=json env: PLAYWRIGHT_JSON_OUTPUT_NAME: playwright-report.json- uses: plune-ai/report-action@v1 if: ${{ always() }} with: token: ${{ secrets.PLUNE_TOKEN }} report: playwright-report.jsonIn any CI:
PLAYWRIGHT_JSON_OUTPUT_NAME=playwright-report.json npx playwright test --reporter=jsonnpx -y @plune-ai/cli run import playwright-report.jsonThe import finds each test’s screenshots where the report says they are, so run it where the report was written.
k6 and Lighthouse
Section titled “k6 and Lighthouse”From @plune-ai/cli 0.33.0, plune run import also reads a k6 end-of-test summary and a Lighthouse JSON
report. Plune runs no load or page-speed tests of its own: it takes the results of the tools that do, as a run.
k6 run --summary-export=summary.json script.jsnpx -y @plune-ai/cli run import summary.json --create
lighthouse https://example.com --output=json --output-path=lighthouse.jsonnpx -y @plune-ai/cli run import lighthouse.json --create --min-score performance=80 --min-score seo=95Each is recognised by its own keys, never by the file name; --format k6 or --format lighthouse says it
outright.
- A k6 threshold is one result, named after the metric and the expression
(
http_req_duration p(95)<500): passed when k6 says it held, failed when it was crossed, with the measured values as the message. A summary with no thresholds is refused with exit code 2 and nothing is imported. - A Lighthouse category is one result, named after the category, the page and the form factor
(
performance (https://example.com/, mobile)), with the score from 0 to 100. It passes at or above the bar:--min-scoresets the bar per category, and a category you do not name needs 90. The GitHub Action has no input for--min-score, so through it every category needs 90.
As with any report, a result reaches the run only once it has a case. A first import finds none, so
--create offers them to the review queue, and
imports record them once a person has approved them there.
From then on, a crossed threshold or a category under its bar is a failed result, and only that: the run
finishes as usual and the command exits 0. To fail the job on it, add --fail-on-failures — see
Gate a release on the platform’s verdict.
A result with no case yet does not count towards that verdict.
k6 itself exits 99 when a threshold is crossed, so give the import a step of its own, with
if: ${{ always() }} as above.
When the jobs of a matrix import k6 summaries into one run under one --key, each job names its file apart,
because a threshold is known by the file’s base name, not its folder:
- run: k6 run --summary-export=summary-${{ matrix.scenario }}.json script.jsSeveral jobs, one run
Section titled “Several jobs, one run”Give every job the same key with the attempt in it, so that Re-run failed jobs starts a new run
instead of joining the first attempt’s. On the action that is key, plus shared-run: 'true' on every
job but the one that ends the run, or shard: k/N on each shard. On the command it is --key with
PLUNE_SHARED_RUN=1 or PLUNE_SHARD=k/N — see
Several jobs, one run. The action’s other
inputs (run-title, env, labels, create, fail-on-missing-attachments, …) are listed in
its README.