Skip to content

Test Value Score

The Test Value Score of a case is one number from 0 to 100: how much the case has been worth running so far. Plune counts it from the case’s own results, the defects found through it and how long it takes, each time you ask, and shows every number it is made of. Nobody sets it, and it changes nothing on the case.

  • The case page. In the head, after the state badge: a gauge with the score, named Value 71 of 100. Under the tags, a Value block with the Value score, its four parts and the sum worked out with the case’s own numbers. See the case page.
  • The list of cases. Under the filters, Case list view switches between Tree and By value. By value is one flat list with the score on every row, Highest value first · equal values keep the tree order. The filters still apply, and the address carries the view as sort=value.

The Tree view shows no score.

The list of cases in the By value view: the switch Tree and By value under the filters, the line Highest value first · equal values keep the tree order, and each row with its score. The list of cases in the By value view: the switch Tree and By value under the filters, the line Highest value first · equal values keep the tree order, and each row with its score.
The Value block of a case: Value score with the line from N of the last 50 results, the parts Failures · weight 0.5, Defects · weight 0.3, Speed · weight 0.2 and History depth · scales the rest, each with its bar, the formula with the case's own numbers, and the line Not the Flaky flag. The Value block of a case: Value score with the line from N of the last 50 results, the parts Failures · weight 0.5, Defects · weight 0.3, Speed · weight 0.2 and History depth · scales the rest, each with its bar, the formula with the case's own numbers, and the line Not the Flaky flag.
Value = 50 + 100 × depth × (0.5 × failures + 0.3 × defects + 0.2 × speed − 0.5)

rounded to a whole number. Each part is a share from 0 to 1:

PartWhat it isWhat it is read from
Failures · weight 0.5failed ÷ counted resultsthe case’s last 50 results. A skipped or blocked result is not counted; of the rest, every one that did not pass is a failure.
Defects · weight 0.3(bugs + links) ÷ 3, at most 1the bugs that name the case, leaving out those in the Bin, and the links to tracker tickets on the case and on its results. Three or more is 1.
Speed · weight 0.21 − the case’s p50 ÷ the project’s p95, at least 0the durations of results in runs of the project that started in the last 90 days. A case with no measured result uses its estimate; with neither, speed is 0.5.
History depth · scales the restcounted results ÷ 50, at most 1the same last 50 results.

A case that fails often, has found defects and runs fast scores high. Depth pulls the score towards 50 while the history is short, so a case with two results cannot reach either end of the scale.

The Value block writes the sum out with the case’s own numbers, each part already multiplied by its weight, and says under each part what it was read from. A case that failed 2 of its 20 counted results, has one bug and takes 1.2 s where the project’s p95 is 8.0 s reads 2 failed of 20 runs, 1 bug + 0 links · 3 or more is 1.00, p50 1.2 s against the project’s p95 8.0 s and 20 of the last 50 results, and its sum is here 50 + 100 × 0.40 × (0.05 + 0.10 + 0.17 − 0.5) = 43.

A case with no counted result among its last 50 has depth 0, so it scores exactly 50, the middle, whatever its defects and duration. It shows a dashed outline with no history yet, and the Value block opens with No history yet: no result in the last 50, so the value is 50 — the middle. By value the case keeps its place instead of dropping to the bottom; the value moves with its first result.

Not the Flaky flag: the value is counted from the results and never sets or clears Flaky. The score neither reads nor writes the case’s Flaky mark.

  • GET /v1/test-cases/{id} carries the case’s value.
  • GET /v1/test-cases?value=1 adds a value map, by case id, for the cases of the page. Without it the list reads no score and the answer has no value key.
  • GET /v1/test-cases?sort=value lists the highest score first, ties in the tree’s order, and sends the value map. Any other sort, or none, is the tree’s order.

A value holds score, neutral (true for a case with no history), the weights, and parts: failures (value, runs, failures), depth (value, runs, limit), defects (value, bugs, links) and duration (value, p50Ms, projectP95Ms, and source: results, estimate or none). The score can be worked out again from these numbers by hand.