Interpreting the diff

On each PR the Action runs your suite on the PR head and the base branch, then plune diffs them. The result shows up in three places: the sticky comment, the check status, and the action outputs.

The transition table

Every eval is classified by how it moved between the base branch and the PR. Only a genuine passed → failed transition is a regression:

Transition	Meaning	Gates?
`passed → failed`	regression	yes (when `fail-on-regression: true`)
`failed → passed`	improvement	no
`failed → failed`	pre-existing failure	no
absent → failed	new failure (new eval)	no
`errored` (either side)	execution error, not a quality change	no

The key idea: an eval that was already failing on the base branch does not turn the check red. Only quality you had and lost gates the merge.

The sticky comment

The Action posts one comment per PR and updates it in place on every push — it never spawns duplicates. The body summarizes regressions and improvements; the headline is the same one-line summary exposed as an output: N regression(s), M improvement(s).

Consuming the outputs

The Action exposes three outputs you can use in later steps:

Output	Description
`has-regression`	`'true'` when at least one `passed → failed` regression was detected
`regressions`	number of regressions
`summary`	one-line diff summary

- uses: plune-ai/eval-action@v1
  id: plune
  with:
    config: plune.yaml
- name: Annotate
  if: ${{ steps.plune.outputs.has-regression == 'true' }}
  run: echo "::warning::${{ steps.plune.outputs.summary }}"

Next steps

fail-on-regression as a required check Turn has-regression into a merge-blocking branch protection rule.