The review queue

Plune generates test cases. It does not add them.

Everything a machine proposes lands in a queue and waits for a person, and that is the single rule the rest of the product is arranged around. A tool that writes tests into your suite on its own is a tool you stop trusting after the first bad one — and then you stop reading its output at all, which costs more than it ever saved.

Three kinds of entry

They arrive from different directions and take the same decision. Exactly one of the three fields is ever set on an entry, and the database holds that rule, not just the schema.

Proposal — a case a generator wrote. It carries a proposedCase: steps, an expectation, and the mode that produced it. This is what an ingest from Cairn queues.

Discovered test — a test your runner reported that this project has no case for. It carries a discoveredTest, and the difference from a proposal is the whole reason it is its own kind: a reporter sees a result, never the source, so it has no steps to state and no expectation to claim. The entry testifies that a case is missing instead of describing one — keys, title, location, and what the test did on that run. This is what --create and POST /v1/review-items/discovered produce, and it is the ordinary path for onboarding an existing suite: import the report, decide once, and every later import matches instead of offering.

Dispute — a result whose verdict deserves a human look rather than trust. It carries a resultId pointing at the result in question, and its comparison is shown side by side.

Three states, one decision

An entry is pending until somebody decides, and then it is approved or rejected. Approval is the only way a generated case enters your suite — there is no other door, not through the API and not through the dashboard. A discovered test has a second door, and you are the one who opens it: see trusted sources below.

1. Read it. GET /v1/review-items/{id} returns the entry with its comparison.

2. Decide. PATCH /v1/review-items/{id} with approved or rejected. Approving a proposal creates the test case and links it to the entry.

3. Or decide in bulk. POST /v1/review-items/decisions takes one verdict over an explicit list of ids — explicit, so a batch can never decide something you have not seen.

A rejection carries a reason

Rejecting takes a category, and it is required. A queue of rejected entries with no reasons answers “was this a bad proposal or a bad test?” the same way for both, which is to say not at all — and that answer is the only signal there is for whether generation is getting better.

Who may propose

Proposing is not a paid-seat action: a reader, the free role, may put a result forward with POST /v1/review-items, and on a case page Send to review is there for a reader too. The item is created pending and nothing else. A reader’s request that asks for another state, or carries a rejectionCategory, is refused 403 and nothing is queued, so a proposal can never be a verdict.

Deciding stays with a seat that writes. A reader cannot approve or reject, in bulk or one by one: the API answers 403 read-only member, and the queue shows a reader the entries with their state and no Approve, Reject or checkbox on a row, and no Approve or decline on an entry. A reader’s proposal counts in the same limit of 1 000 waiting items as anyone’s, and is recorded in the audit log with the reader as its author.

Why the queue is idempotent

Re-importing the same report does not re-offer what you already decided. A proposal matches an existing case through its external keys, so the second import of 2 000 tests after you accepted them reports 2000 accepted, 0 unmatched rather than filling the queue again.

That property is what makes onboarding a big suite survivable: the first import is a decision, and every one after it is a no-op.

Trusted sources

A discovered test is not something a machine invented. It exists in your repository — Plune did not propose it, it read it. So the queue asks a question whose answer you already know, and it asks it once per test.

That costs more than it looks. The queue holds 1 000 waiting items and one decision covers at most 200, so onboarding a suite of N tests is ceil(N/1000) imports plus ceil(N/200) decisions: 14 actions for 2 000 tests, 120 for 20 000. Neither ceiling is wrong on its own. Their product is.

So the permission is granted to the source, once:

curl -X PATCH https://beta-api.plune.ai/v1/settings \
  -H "Authorization: Bearer $PLUNE_TOKEN" \
  -H "content-type: application/json" \
  -d '{"trustedDiscoverySources": ["playwright"]}'

Or in the dashboard, under Settings → Trusted sources.

From then on a discovered test from that runner becomes a case immediately, active rather than waiting: over a test that already runs in your CI there is no second judgement to make. Every case it creates is written to the audit log with trusted-source as the reason, which is what answers “which of these did I actually read” six months later.

What trust does not do

It does not overturn a rejection: a test you declined stays declined, because “no” is an answer you already gave. It does not touch proposals — a case a generator wrote still waits for a person, with no exception. And it does not raise the 20 000 cases a project holds.

It does resolve what is already waiting: entries queued before you granted the permission become cases on the next import, so turning this on after you hit the wall is not too late.

PATCH /v1/settings replaces rather than merges, so send the settings your project has — a key you leave out is one you removed.

Checked before review

A proposal is checked when it arrives, before it is queued. A finding marks the proposal — a “flagged” badge on its row, a list on its page — and never blocks it: it is still in the queue and the decision is still yours. The finding names the rule and links here. A proposal with nothing found carries no mark.

Check: title-vague

The title has fewer than three words, so it does not say what the case verifies. “Login works” fits many cases; “Login rejects a wrong password” fits one.

Check: step-empty

A step has no letters or digits in it — blank, or only punctuation such as ....

Check: expected-vague

The expected result has fewer than two words. “ok” does not say what should happen.

Check: duplicate-case

A case of this project already has this title. Case and spacing are ignored. The case may be the one the proposal is meant to replace; the mark only asks you to look.

Check: duplicate-proposal

Another proposal waiting in the queue has this title, or an earlier case in the same Cairn run does.

Every decision is recorded

Approvals, rejections and corrections all write to the audit log, and a change can be put back from it — including a whole session’s worth at once. The log is not swept along with the data it describes: delete the run and its audit trail is still there, because a log that disappears with its subject only proves that nothing was kept.