fail-on-regression as a required check

By default the Action only comments — it never blocks a merge. To make a regression actually stop a PR, you need two things: opt into the gate, and make the check required on your protected branch.

1. Opt into the gate

Set fail-on-regression: 'true'. Now the Action exits non-zero (the check goes red) whenever a passed → failed regression is detected:

- uses: plune-ai/eval-action@v1
  with:
    config: plune.yaml
    fail-on-regression: 'true'

A red check alone doesn’t prevent a merge — a maintainer can still merge a failing PR — until you make it required.

2. Make the check required

1. Go to Settings → Branches → Branch protection rules and add (or edit) a rule for main.
2. Enable Require status checks to pass before merging.
3. Search for the Plune job’s check name — it’s the job id from your workflow (e.g. plune) — and select it.
4. Save. Now a regression-red check blocks the merge button.

Tip

The check only appears in the list after it has run at least once, so open a throwaway PR first if you don’t see it.

Forks can't be gated on the comment

On fork PRs the comment may be skipped (read-only token), but the check conclusion still gates — the regression exit code stands regardless of the comment. See Fork pull requests.

Next steps

Fork pull requests How the gate and comment behave on PRs from forks — safely.