Direction Test
The test a change must pass before we say that a score rose or fell. Its direction must hold in at least 90% of the 10,000 runs behind the robustness band; for Legal, under all three ways of weighting its categories. Otherwise we show both values and make no claim.
What It Is
Before we say that a score rose or fell, the change must pass the direction test. This holds for the index, a component, a category, the LRRS, the balance gap and the plain average of the components. It holds for every way of saying it: a word, an arrow, a sign, a colour, a percentage or a caption. Showing two values side by side, in a table or in a chart, is not a claim.
The test recomputes the change in each of the 10,000 runs of the robustness band, so it cannot run without them. It passes when the change goes the same way as the published values in at least 90% of the runs. The runs do not move Legal, so for the Legal component and the LRRS the change must instead go the same way under all three ways of weighting the legal categories that we publish: rank-sum, equal and rank-order-centroid weights.
The result decides what we show:
- Passed: we state the direction.
- No change: the two published values are equal, so we say "no change", and no runs are needed.
- Failed, or the test could not run: we show both values, make no claim about the direction, and say so.
- Nothing to compare: there is no earlier score yet, and we say that.
Each page fixes in advance which changes it tests, over which period: the index over 30 days on our live pages, each component over 7 days on /mei, and each calendar month in our monthly reports. A change that fails is never swapped for another period or another score that passes. A single metric's value is a measurement, not a claim, and is shown as measured.
We adopted the test on 30 October 2026, in Methodology Decision 20. Our September and October 2026 reports, and the note of 30 October 2026 on our methodology page, call the index's 30-day test R1.
Why It Matters for the Machine Economy
A small rise or fall in the MEI can come from our choices of weights and bounds rather than from the machine economy. If another reasonable choice would reverse the direction, saying "the index rose" would claim more than we know. The test keeps every stated direction to what the data support, and readers can check it against the published values.
Real-World Example
In our October 2026 report, the index's rise and the moves of Payment, Physical and Macro passed the test in every run. Legal was 11 at both ends of the month, so the report made no claim about its direction. Run on our past scores, the test fails for the index's 30-day change on seven days from August to October 2026, and no page, report or briefing stated the index's direction on those days.
Related Terms
- Robustness Band — the runs this test uses
- LRRS — tested under its three ways of weighting its categories