An audit log that tells you which events it does not understand yet

There is an endpoint in a distributor platform that can reassign fifty or more accounts to a different sales representative in a single call. No approval step, no second pair of eyes. If somebody runs it by mistake, or runs it deliberately and later denies it, the audit trail is the only record of what happened.

Which means the audit trail has to capture the specific accounts affected, not just a count. “Fifty accounts reassigned” is a fact about the size of the event. It is useless for reconstructing which fifty, and reconstructing which fifty is the entire reason the log exists.

That endpoint is a good lens on audit design, because it makes the requirement concrete: an audit log is only worth having if it can answer the question you will ask during an incident. Most audit logs are built to satisfy a compliance checkbox and are discovered to be inadequate on the one day they matter.

Do not write audit calls by hand

The first design decision, and the most important one: audit entries are not written by endpoint code.

Hand-written audit calls per endpoint get forgotten. Not sometimes — reliably, because adding one is a step that has no effect on whether the feature works, so it is the step that gets dropped under time pressure and the step a reviewer does not notice missing. And a log with gaps is worse than no log, because people trust it. The absence of an entry gets read as evidence that nothing happened, when it actually means somebody forgot a line six months ago.

So capture is central: one middleware mounted after authentication on every protected route, firing on the mutating HTTP methods only, writing a row when the response finishes.

That structure gives you several properties for free:

• New endpoints are covered by default. The failure mode inverts — instead of forgetting to add auditing, you would have to deliberately remove a router from the middleware.
• Failed calls are logged too. A rejected attempt to do something is often more interesting than a successful one, and endpoint-level audit code almost never covers the failure path because it sits after the work succeeded.
• The actor, path, module, status and outcome are captured uniformly, because they come from the request rather than from whatever each developer thought to include.

The payload is stored as a deliberately allow-listed snapshot rather than the whole request body. That is worth being explicit about: an audit log is a long-lived, widely-readable store, and putting raw request bodies into it means putting credentials, personal data and payment details into it. The allow-list is not a convenience, it is the thing that stops your audit log becoming your worst data exposure.

Labelling, and the trick worth stealing

Central capture gives you complete coverage of events. It does not give you meaningful names for them, because the middleware only knows the method and the path.

So event labels resolve through a registry: a lookup from method and path to a human-readable event type. Anything the registry does not recognise is recorded with a fallback label meaning “an unrecognised mutation”.

The trick is what you do with that fallback.

The operational procedure is to periodically filter the audit log for the unknown label, group by path and module, and add registry entries for the paths that matter. The gap list generates itself, from production traffic, ranked by how often each gap actually occurs.

Compare that to the alternative: someone reads the route table and writes registry entries for everything. That produces entries for routes nobody calls, misses routes added since, and gives you no information about which gaps are important. Mining the fallback gives you a list that is complete with respect to real usage and ordered by frequency, with no maintenance effort at all.

This pattern generalises well beyond auditing. Any classification layer over a stream of events should have an explicit unclassified bucket that you routinely inspect, rather than a default that silently absorbs the unknown into a plausible-looking category. A fallback you never read is a blind spot; a fallback you mine is a work queue.

What the audit still missed

Cross-checking the registry against the schema and the route table found real gaps, and they cluster in a way worth noting.

One entire router — the file upload router, handling photographs and product images — had no audit middleware mounted at all. Not a missing registry entry: no capture. That is the failure mode central capture is supposed to prevent, and it survived because mounting the middleware is still a per-router action. The lesson is that centralisation moves the forgettable step rather than eliminating it, and you should know where the remaining forgettable step is.

Authentication events — failed logins and token refreshes — were not logged at all, because the middleware deliberately skips the authentication routes. Sensible on the face of it: those routes are noisy and the middleware sits after authentication anyway. But failed logins are among the highest-value audit events in any system, and excluding them by a path rule means the exclusion is invisible.

Several mutating operations were captured but generically labelled, so they were present in the log and not findable in it.

The pattern across all three: skips and defaults are where audit coverage goes to die, and they are invisible in code review because they are written once at the top of a file and never revisited. Anything your capture layer deliberately excludes deserves a comment explaining why, and a periodic review of whether the reason still holds.

Retention, stated rather than assumed

Retention was explicitly deferred to a later phase rather than implemented and forgotten. That is worth mentioning because the failure I see more often is the opposite: a retention policy documented in a design document, believed by everyone, and never actually built.

In a related system, the specification said the audit table was partitioned by month so that old partitions could be archived and dropped. The table was not partitioned. Nothing was archiving anything. The specification described an operational story that did not exist, and the only reason anyone found out was that somebody checked the catalogue rather than the document.

Verify infrastructure claims against the system, not against the document that describes it. A design document is a statement of intent. Whether the intent was carried out is a different question with a different answer, and it is answered by querying the database.

The questions to ask of your own audit log

• If someone disputes an action taken last month, can you reconstruct exactly what changed, by whom, and to which specific records — not just that something changed?
• Are failed and rejected attempts logged, or only successes?
• Which routes are excluded from capture, and when was that list last reviewed?
• How do you find out about an event type your labelling does not recognise?
• Is anything in the log that should not be there — credentials, personal data, a full request body?
• Does the retention behaviour described in your documentation actually exist in the database?

The purpose of an audit log is to be trustworthy on a day when someone is arguing. That trust is built entirely from coverage you can demonstrate, and demonstrating it means knowing where the gaps are — which is why the most useful thing an audit system can do is tell you what it does not understand yet.