Half a delimiter is a perfectly normal thing to receive
A parent using the AI chat saw a line of at-signs and capital letters in the middle of the conversation. Worse, the button offering to accept that reply as a lesson would have written those characters into a lesson body a child reads.
The chat lets a model mix two things in one reply: ordinary conversation, and an offered draft of a lesson. They are separated by markers — a distinctive opening string, the draft, a distinctive closing string. The extraction is a regular expression that finds text between the two markers.
A reply that hits the output token cap in the middle of the draft has the opening marker and no closing one. The expression matches nothing. The reply is therefore treated as ordinary chat, and ordinary chat is displayed verbatim.
Truncation is not an edge case
The first thing to say is that this is not an unlucky case. With a token-capped generator, truncation is a routine outcome, and it is more likely on exactly the requests where the content is longest and most valuable.
Any in-band delimiter protocol — markers, fenced blocks, XML-ish tags, a JSON object embedded in prose — has three possible received states and most implementations handle two of them:
- Neither marker. Plain content. Usually handled.
- Both markers. The happy path. Always handled.
- The opening marker alone. Truncated mid-payload. Almost never handled, because it does not occur while you are testing with short prompts.
The fourth state, a closing marker with no opening one, is rarer and worth handling too since it means something has mangled the front of the stream.
A protocol with an opening and closing token needs a defined behaviour for the case where the stream ends between them. Ours now reads a lone opening marker for what it is: everything after it is the draft, and the result is flagged as truncated so the interface can say so rather than pretending it has a complete lesson.
The fallback was the actually dangerous part
The display bug was ugly. The fallback was the real defect.
When extraction found no draft, the accept button fell back to using the whole reply as the lesson body. That is a reasonable-sounding safety net — if we could not find the draft, use everything — and it means unparsed model output goes directly into a field a child reads, with the protocol’s own control characters still in it.
Never let unparsed generator output fall through into a user-visible field as a fallback. A parse failure is a failure. The honest behaviour is to say the reply could not be read as a draft and offer the raw text for the parent to look at, not to quietly promote it.
The broader form of this rule is that fallbacks should degrade the feature, not the content. Losing the ability to offer a draft is a degraded feature. Publishing malformed text into a reading screen is degraded content, and content is what people see.
Then find why it truncated
Fixing the parser handles the symptom. The cause was more embarrassing and more instructive.
The chat’s output token cap was set to a smaller number than the “long” length preset the same product uses when a parent explicitly asks for a long lesson elsewhere. The chat is the one surface with no length control at all, so it is the surface most likely to be asked for something long, and it had the smallest budget in the system.
Three caps existed in three places, set at three different times, and nobody had ever put them next to each other. The cap was raised well above the longest preset, and the prompt now tells the model a budget exists so it can plan a shorter piece rather than being cut off mid-sentence.
Any system with several output-length limits needs them listed in one place and compared. They drift because each is locally reasonable, and the failure mode is always the same: the smallest one silently governs, in the place you least expect it to.
What to drop when you have to drop something
The same subsystem forced a related decision, and I like the answer enough to record it.
Conversation history is capped — a maximum number of messages and a maximum character budget. When a conversation exceeds it, something has to go.
Dropping the oldest messages is the obvious approach and it is wrong here, because the oldest message is the parent’s original instruction: what the lesson is about, who it is for, what tone. Drop that and the model gradually forgets the brief while remembering the last four rounds of “shorter please”.
So the rule is: keep the first message always, then keep as many recent turns as fit, and drop the middle. The intent and the immediate context survive; the negotiation in between does not need to.
That generalises past chat. When you have to discard context, work out which single item carries the purpose and exempt it, rather than applying a uniform recency rule to everything.
One privacy consequence worth stating, since this is a product used by families: the record kept of each generation stores counts, the model used and the provider, and never the message bodies. There is an operational cost to that — I cannot look at what someone asked when they report a bad result. That is the correct trade for a product where the messages are a parent writing about their child.
Where the whole class disappears
The last thing I would say is that in-band markers are a workaround, and they should be treated as one.
Where a provider supports structured output properly, the quiz generation path uses it, with the schema tightened mechanically rather than copied by hand, so the model cannot return unparseable output at all. There is no delimiter, therefore no truncated delimiter, therefore no fallback, and the entire family of bugs above does not exist on that path.
Markers are what you use when the provider gives you nothing better. If you have to use them, assume every message will one day arrive cut in half, because sooner or later one will.