Why a promotion ended at the wrong hour

You run a promotion that ends on the 31st. You mean the end of the 31st, where your business is. But your software decides it is over by the clock in another country, so it stops at a different moment from the one you meant. Nobody complains, because five and a half hours at the edge of a day does not generate a support ticket. And the same software ends every customer’s free trial at the same wrong moment.

That is the actual finding behind this article, and we reached it by an odd route. A test run reported that a discount scheme which had expired yesterday was still being applied to orders. It went into our defect register as a money problem: expired promotions keep discounting. That is the kind of finding you act on immediately.

It was not a bug. It was a false alarm. But underneath the false alarm was a real one, smaller and more consequential.

What was actually going on

The test worked out “yesterday” from the date on the machine running it. The server judged the scheme against the date in UTC, the world’s reference clock. The business’s local time is five and a half hours ahead of UTC, so between midnight and half past five in the morning, local yesterday is still UTC today. The scheme was genuinely live by the server’s reckoning. The test was asking a different question from the one it thought it was asking, and only during those five and a half hours of each day.

The real bug sat under that. A discount scheme’s end date is a business date: a label a person puts on a day in a particular place. When a business says a promotion runs until the 31st, it means until the end of the 31st where the business is, not 23:59:59 by a clock somewhere else. The system judged it on the clock somewhere else, so on the last day the promotion did not stop when the business meant, and, as the test had shown without meaning to, it kept applying into the early hours after.

The system had a timezone setting for each customer company, stored, editable and shown in the admin screens. Nothing read it apart from the screen that edited it. A setting that exists and changes nothing is worse than none, because it tells everyone the problem has been handled.

The same mistake turned up twice more. A test of attendance records failed eight checks at a date rollover and passed when re-run. And the free-trial check compared against UTC, so every customer in that timezone had their trial end at the wrong time, by the same five and a half hours, automatically. Whether that cut their trial short or stretched it we cannot show from the record we have, and we have not claimed either.

What we changed

Two things, because there were two causes. The test was given a margin: it now asserts that something from seven days ago has expired, so a one-day skew can never pass for a real expiry again. And the software now works out the business date from the customer’s own timezone setting, taken from who is asking and not from something anyone could forget to pass. The discount check, the trial check and everything else asking “is today inside this range” use it.

There is a lesson about flaky tests in there. A test that fails near a boundary and passes when you re-run it at another hour is easy to write off. Here the test and the code were wrong in different directions but from the same confusion, so fixing one did not show the other. When something fails around a boundary, suspect both sides.

What it did not fix

The harm was small each time and invisible each time. Nobody reports a trial that ended a few hours away from when it should have, and nobody notices a promotion ending slightly off. Only in aggregate was it real, and it was never individually reportable, so we cannot give you a count of customers or orders affected. We do not have one.

The pattern, for anyone running promotions, trials or daily reports

Two kinds of date get called “a date”. One is an instant: when a payment cleared, when someone logged in. Store those on one world clock. The other is a business date: the day a promotion ends, the day a trial expires, which day’s sales a sale belongs to. Those must be worked out in the timezone of the business the decision is about, never the server’s and never the machine running the test.

Check three things in your own business. For each date that decides something, which kind is it? Does anything actually read your timezone setting? And when a test fails at a boundary and “fixes itself”, did anyone find out why?

Where this ends up

Deciding a date in the timezone of the business it concerns is the kind of rule that has to be settled before the first screen is drawn. It is part of how Field handles discount schemes, attendance and trials for each company, and why it belongs in discovery and not in a defect register afterwards.