Systematic Error and Noise
How to tell a rate that needs changing from a quarter that was unusual, the four-period rule, and why single-period corrections make plans worse.
A single period's variance tells you almost nothing. Capacity has real variation in it — absence clusters, work arrives unevenly, one claim runs long — and reacting to one period's difference is how an account acquires corrections that cancel each other out.
The variance in “Systematic Error and Noise” requires an unchanged plan and a trustworthy record of actual work. Teams exploring check this practical product resource for team productivity metrics can strengthen the actual side of that comparison while preserving corrections, category changes and human explanations.
The distinction that matters is between noise, which should be absorbed by contingency, and systematic error, which should change a figure.
As a separate benchmark for “Systematic Error and Noise”, see BCG people and organisation resources. Keep source data, calculation, override and decision visible as distinct steps.
The four-period rule
Change a rate only when the same error appears in the same direction in at least four consecutive periods, or when a specific cause is identified that will obviously recur.
Four is arbitrary and it is long enough that random variation has cancelled and short enough that the correction arrives within a year.
What systematic error looks like
Every period, the same sign. Supply two to five per cent below plan each quarter; support consistently above its allowance; one claimant's estimates consistently a third low.
The consistency is the signal. Size is not: a small consistent error is more actionable than a large random one, because it can be corrected exactly.
What noise looks like
Alternating signs, varying magnitude, and an explanation available for each instance. Three absences in one quarter and none in the next is noise even though both quarters missed the plan.
Noise is what contingency exists for, and an organisation that corrects for noise will find its contingency growing without its accuracy improving.
The series, which is the whole instrument
One row per period, six numbers, kept for two years. That table is the only way to distinguish the two, and it takes thirty seconds a period to maintain.
Organisations without the series argue about whether a problem is real, which is unresolvable, instead of looking at eight rows, which is not.
Correcting a rate properly
When the evidence is there, move the rate to the measured value rather than part way. Half-corrections produce a figure that is still wrong in the same direction and another four periods of the same conversation.
Record the change, its date and the evidence in the account's header, so that the series remains interpretable across the change.
The break in the series
A rate change, a reorganisation, a new system or a significant change in the work breaks comparability. Mark it in the series with a line and a note.
Series with unmarked breaks produce the most confident wrong conclusions in capacity management, because the step change is read as a trend.
Small teams and large variance
In a team of four or five, almost everything is noise: one person's absence is a large percentage, and four periods of data is four data points about a population of five.
For small teams, the honest position is that rates are approximate, the contingency is proportionally larger, and the series is still worth keeping — it just takes longer to say anything.
The temptation after a bad quarter
To add margin everywhere. It is understandable and it degrades the plan: the margin lands on the lines that were not wrong as well as the ones that were, and next period's comparison is against an inflated baseline.
One targeted correction beats five general ones, and the diagnosis from the previous page is what makes it targeted.
The temptation after a good quarter
To remove margin. Equally wrong and less common. A quarter that came in under plan is one data point, and a contingency removed on that basis will be missed the first time anything unusual happens.
What four periods of the same sign looks like
| Period | Supply variance | Support variance |
|---|---|---|
| Q1 | −3% | +18% |
| Q2 | −4% | +22% |
| Q3 | +1% | +15% |
| Q4 | −3% | +19% |
Supply is noise around zero; support is systematic and large. Correct the second, leave the first alone, and the next year's plan is better in the only place it was reliably wrong.
The rule for changing a figure
Change it when four periods agree, change it to the measured value, and write the date and evidence in the header.
Three clauses. They prevent both of the failure modes — the plan that never corrects and the plan that corrects every quarter — and between them those two account for most of what goes wrong after an account has been running a year.
What the series eventually gives you
A plan whose errors are small, random and symmetric. That is the realistic target: not accuracy, which is unavailable, but unbiased estimates with a known spread.
An organisation that reaches it can state its own planning error as a number, which is a position almost nobody is in and which takes about two years of six-number rows to reach.