Is the compliance percentage on my dashboard real?

The big number at the top of a compliance dashboard is the one everybody reads. In the product we are describing, that number does not come from the certificates. It comes from a table of saved snapshots, and nothing in the application ever writes to that table.

A comment in the database design says a scheduled job refreshes it. There is no such job. The only things that ever put figures there are the demo seeding script and a second script that fills the table in when the first has not.

So a company can have complete staff records, correct certificates, accurate expiry dates and a working requirement model, and the dashboard shows a dash where the percentage should be. Worse, nothing checks the figure against the certificates: a company with snapshots saying eighty-four per cent, and certificate data implying thirty, passes every check we had.

That is the cost. It is a decision made on a number that nothing has tested.

What was actually going on

The two scripts are both good pieces of work, which is what makes them worth writing about.

The first fills in missing snapshots. It works out whether a company has any, and if not, inserts a current one and several historical ones. It can be run again safely and produces the same rows. The second is a validator that runs nine named checks and fails if any does: that the organisation exists, that staff and sites exist, that snapshots exist, that the comparison with the prior period is not empty, and that there are scheduled training sessions in the windows the dashboard asks about.

Both are careful. Both are pointed at the wrong problem.

The thresholds give it away. The first script adds snapshots when there are fewer than two, because the change-versus-prior tile needs two rows and, with one, the trend chart is a single dot. It adds a training session when none falls inside the fourteen-day window, because a tile counts sessions in the next fourteen days. It adds another when fewer than two fall inside an eight-week window, because a chart looks wrong with a single bar. Those three counting rules were copied from the dashboard character for character.

So the script’s specification is “make these tiles non-empty”, not “these figures are true”. It compensates for a missing writer, and its test of success is the appearance of the screen.

Nobody decides to build this. The database is designed well first, and pre-adding up the percentage is the right call. The “refresh via job” comment is a note to self. Then the dashboard is built to read the table, and the job is not on anyone’s list, because it is not a screen and nobody asks for it in a demo. The demo comes up blank, somebody writes a seed. A company that was not seeded comes up blank, somebody writes the backfill. Somebody wants to know beforehand, and writes the validator. Each is a reasonable answer, and the missing job ends up further away each time, because a handled symptom creates no pressure.

What is still open

The job does not exist. Nothing was changed by this write-up, and this article records a finding, not a fix.

Building it means answering a question the product has not yet settled: what period a compliance snapshot covers, and whether the figure is a point-in-time measurement or an average over a period. Those give different numbers, and the current data cannot say which one it is.

What it did not fix

The validator checks that snapshots exist. It does not check they are consistent with the certificates. Its author’s problem was emptiness, not wrongness, so a validator like that will never contain the check that matters. It also only examines the organisation-wide figure, so a company with no site-level snapshots passes, then shows an empty bar chart and a blank “worst-performing site” field, which is exactly what the validator was written to prevent.

The change-versus-prior figure subtracts the two most recent snapshots with no check that they cover comparable periods. With a real job, snapshots would arrive at a fixed rhythm. Without one they arrive whenever a script ran, so “versus prior” means “versus whenever the last one happened to be written”. And the trend line is capped in length, so a long date range returns a truncated series, with nothing on screen saying the line was cut short.

The pattern, for anyone reading a compliance dashboard

Ask where the headline number comes from and what writes it. If the answer is a file of saved figures, ask what updates them and when they were last updated.

Then ask whether anything compares the figure against the underlying records. And be wary of any script whose job is to make a screen look populated: it is telling you about a missing writer.

Where this ends up

The compliance product this comes from is Sazinga Comply: staff, sites, certificates, expiry dates and a requirement model, per company. The headline percentage is the figure its dashboard puts at the top, which is why the job behind it is a deliverable with a name, not a line in a schema comment.