How we measure.
This page sets out what we count, how each number is worked out, and when we call a batch a success. The same rules apply to every client of XPU Systems and to every batch. The scores that matter are yours, not ours.
What we count.
Six numbers, reported on every batch. Each one is a plain count or a ratio of two counts.
- Unit acceptance rate
- The share of units in a batch that you accepted after scoring your sample against the standard in the contract. This is the number the invoice is built from. A unit that failed, is being reworked, or is in dispute is not in the accepted count.
- Worked out as
accepted units / delivered units - Rework rate
- The share of delivered units that came back to us for rework, whether your sample caught them, our own check caught them after delivery, or you reported them. Rework is at our cost. A rising rework rate means our check before delivery is not holding, and that is our problem to fix, not yours.
- Worked out as
units returned for rework / delivered units - Turnaround per batch
- The time from a batch arriving with us to the same batch being delivered back finished, measured on the working calendar agreed in discovery. Reported for every batch against the turnaround written in the contract, so you can see at a glance which batches, if any, were late.
- Worked out as
time the batch was delivered finished, less time the batch was received - Volume delivered
- The number of finished units delivered in the period, counted per unit type and per batch. It is a plain count, not an estimate. It sits beside the accepted count so the gap between what was delivered and what you accepted is visible.
- Worked out as
sum of finished units delivered in the period, by unit type - Sample coverage
- The share of each batch that you actually scored. The contract sets a minimum sample per batch. Coverage shows whether that minimum was met, and if you choose to score more than the minimum, or every unit, it shows that too.
- Worked out as
units in your sample / delivered units in the batch - Disputed units
- Units where you and we do not agree on whether the standard was met. A disputed unit stays off the invoice while it is settled. We count them because a dispute nearly always means the wording of a check is loose, and the count tells us where to tighten it.
- Worked out as
disputed units / delivered units
How each number is worked out.
How the unit is defined
Discovery comes first. We look at the work as it runs today and write down what one finished unit is: what arrives, what leaves, and the source or rule the finished unit is checked against. If a workflow has more than one kind of output, each kind is its own unit with its own definition. A unit is something you can count and score, never something we estimate. You agree the definition before the pilot, and you keep it whether or not you go further.
How the standard is written
The standard is the test a unit must pass, and it lives in the contract. It is written as a short list of checks, each answered pass or fail against a source or rule you already use, so that two people scoring the same unit reach the same result. Anything that needs judgement to score is either rewritten as a check or left out of the standard. If a dispute shows that a check can be read two ways, we fix the wording with you and the fix carries a date. Units accepted under the old wording stay accepted.
How you draw and score the sample
Every unit is checked by us against the standard before it is delivered. Then you draw a sample from the delivered batch, at random and without telling us in advance which units, at or above the minimum size in the contract. Your scorer marks each sampled unit pass or fail, check by check, and records the result. You can score more than the minimum, up to the whole batch, whenever you want to. Your scores, not ours, are what the acceptance rate is built from.
What happens to failures
A unit that fails your score comes off the invoice at once. We rework it at our cost, it goes through the same test again, and it reaches the invoice only once you have accepted it. It is billed once, as one unit, however many times it was reworked. If a batch fails the sample, the whole batch is held off the invoice, we re-check every unit in it, and you draw a fresh sample from the redelivered batch. Every failure is logged against the check it failed, and that log is what we use to stop the same failure recurring.
When we call a batch a success.
A batch is a success when three things are true: your sample passed the standard in the contract, the batch was delivered inside the turnaround in the contract, and the number of units on the invoice is the number you accepted. Nothing else counts. Our own check passing does not make a batch a success, and neither does a high volume. The pilot is a success when a fixed batch of real data has met that test and you have the numbers on this page to decide with. The run is a success when every batch keeps meeting the same test, at the same standard, for as long as you choose to keep sending the work.
Questions about the numbers.
Anything else, write to hello@xpusystems.com.
How big does our sample need to be?
Big enough that a batch which does not meet the standard would show up in it, and small enough that your scorer can finish it in the time you have. Discovery sets the minimum for each unit type from the batch size and from how costly a missed failure would be, and the minimum goes in the contract. Sample coverage on every batch shows whether it was met.
Can we score every unit instead of a sample?
Yes. The sample size in the contract is a floor, not a ceiling. Score as many units as you like, up to all of them, and coverage records it. The cost is your scorer's time. One sensible way to use it: score the pilot batch in full, drop to the contract sample once the numbers hold, and go back to full scoring on any batch you have doubts about.
Do you check the units before we see them?
Yes, every one, against the same standard you score against. Your sample then checks our check. If our check were perfect the rework rate would be zero, so that number is the honest measure of how well it is holding, and it is reported on every batch.
Does a reworked unit count twice?
No. A unit is counted once in volume delivered, on its first delivery, and billed once, when you accept it. The rework is at our cost, adds nothing to the invoice, and appears only in the rework rate.
Where do these numbers come from, and can we see them?
From the batch record. Every batch has one: units received, delivered, sampled, scored, failed, reworked, disputed and accepted, with the time each step happened. The record comes with the batch, and the invoice is worked out from it, so the numbers you see are the numbers you are billed on. Your scores are entered by your scorer, not by us.
Do these numbers change what we pay?
No. The invoice is accepted units times the unit price agreed in discovery, and nothing on this page moves it. There are no bonuses for a high acceptance rate and no penalties written into the numbers, because the penalty for a failure is already built in: a failed unit is reworked at our cost and earns nothing. The numbers are there so you can see whether the work is holding and decide whether to carry on.
Want to know what your repetitive work is worth before discovery?
Estimate it