The readiness screen reported 1,081 missing pricing rules. About 33 of them were real
A quoting system for a fabrication business has a screen that tells an administrator whether the catalogue is ready to quote from. For each product it counts how many features are assigned and how many of them have a pricing rule attached, and flags what is missing.
On a catalogue that had just been imported and reconciled against its source workbook to the unit, that screen reported 1,081 missing pricing rules.
My first conclusion was that the import had failed. It had not. Roughly 1,048 of those were artefacts of how the screen counted, and only around 33 were real gaps. The screen was not detecting an incomplete catalogue. It was manufacturing an incomplete catalogue out of a join.
The join
The catalogue models a feature — a top, an undershelf, a splashback — as a company-wide thing that many products can offer. Each feature has variants, and those variants belong to the feature, not to any particular product.
That is the right model. A splashback is a splashback; defining it once and pointing 30 products at it is exactly the reuse that keeps the rule count down.
The readiness screen then asked, for a given product, how many variants exist across all the features that product offers. It got that number by joining the product’s feature assignments to every variant of each of those features, and treating the result as the number of pricing rules that ought to exist.
The flaw is in the phrase all the features that product offers. A feature might have eleven variants across the catalogue as a whole. A specific product might offer two of them. The other nine exist because some other product needs them. The screen counted all eleven as required and reported nine gaps that were not gaps — for that feature, on that product, multiplied across 57 products.
The pooling that makes the catalogue cheap to maintain is exactly what makes a naive completeness count wrong. The two are the same property viewed from different ends.
Why this is worse than a screen being merely wrong
An over-reporting completeness check does not fail loudly. It fails by being ignored.
The first time an administrator sees 1,081 missing rules on a catalogue that is demonstrably producing correct quotes, they learn that the number is not to be trusted. From then on the screen is decoration. The 33 real gaps are sitting in that list, correctly identified, and nobody will ever find them, because they are buried under a thousand items that experience has taught everyone to disregard.
That is the actual cost. A check with a high false-positive rate is not a partial check. It is a check that has been switched off by its users while continuing to render.
The counting rule, stated properly
The question the screen was trying to answer is: for every configuration a customer could actually order of this product, is there a pricing rule.
The set of configurations a customer could order is not the set of variants that exist. It is the set of variants this product has been configured to offer. Those are two different things and the system knows both — the catalogue records which combinations are offered as part of what the quoting interface renders.
So the fix is not clever. Count against the combinations the product actually exposes, and treat a wildcard rule — one that applies to all variants of a feature — as covering every variant it applies to, rather than looking for a rule per variant. Both of those were available in the data. The screen simply asked an easier question than the one it claimed to answer.
There is a second class of false gap sitting underneath. Some of what remained pointed at rules referenced in the source data but never defined anywhere — twelve of them. Those are real, in the sense that no rule exists, but they are not the administrator’s problem to solve by clicking around. They belong in a separate list labelled as source-data gaps, because the fix is for someone to write the missing rule, not for someone to attach an existing one.
Three categories, three lists. Not one number.
The general shape of this bug
I have now seen this pattern often enough to name it. It appears wherever a system reports completeness or coverage over a many-to-many model.
You have entities pooled for reuse. You have assignments that say which pool members a given parent actually uses. And you have a count that, for convenience, joins to the pool instead of the assignments — usually because the pool is one join away and the assignments are two, and the difference does not show up on small data.
The tell is that the reported number scales with catalogue size rather than with actual incompleteness. If adding a product elsewhere in the system increases the number of gaps reported against an unrelated product, the count is joining to the pool.
Two checks worth writing
The first is arithmetic. Before believing a coverage number, ask what its maximum could be. If the screen says 1,081 missing out of a possible 2,698 rules — a catalogue that is quoting real jobs every day — the number is not credible on its face, and it should have been challenged before it was implemented, not after.
The second is a reconciliation. The count of things the system thinks should exist has to be derivable from the source data independently. In this case the source mappings expanded to a known number of expected rules, and the live data matched it. Once you have that reconciliation, any screen reporting a large shortfall is contradicting a number you have already verified, and you know to look at the screen rather than the data.
The rule
A completeness check must count against what is actually offered, never against the pool that options are drawn from, and it must separate “not configured yet” from “referenced but never defined” because those go to different people.
And if a check reports a large number of problems on a system that is visibly working, suspect the check first. It is the cheaper hypothesis to test, and in this case it was the correct one.