Ordered delivery on top of an unordered world

10 November 2026 · ThaiPo

Someone types a long, careful paragraph. Then, a second later, "ok?". A translator that posts the translation of "ok?" first has not translated the conversation. It has rearranged it.

ThaiPo translates every message in a chat concurrently, and a two-word reply comes back from a language model long before the paragraph sent just ahead of it. LINE gives us no way to set a timestamp on a message we post, so the order people read is simply the order we push. Getting that right sits on top of a stack where nothing is ordered at all.

Speech bubbles of different sizes racing along separate tracks and then lining up in numbered order at a single gate before entering a chat window

Everything underneath is unordered

  • Webhooks. They can arrive out of order, and when an acknowledgement is slow LINE redelivers them, in our experience minutes later.
  • The log. Our source of truth is an object store. Keys are time-prefixed by whichever ingest machine received the event, using its own clock, so two machines can disagree about which came first. There is no total order to consult.
  • The work.Translation runs in a worker pool, against models with very different latencies. Finish order is roughly "shortest first".
  • The workers. They crash, get redeployed, and time out, mid-message.

A number at the door, a frontier at the exit

The mechanism is small. When an event is enqueued, it gets the next number in a per-chat counter. There are two enqueuers (a fast path straight from ingest, and a dispatcher that sweeps the log as a safety net), so the number is handed out under the same enqueue-once guard that stops them double-queueing: whoever wins the guard assigns the number, exactly once.

Translation then runs fully in parallel, unchanged. Only the final push is gated. Each chat has a frontier: the highest number such that it and everything before it is finished. A worker holding message 7 waits until the frontier reaches 6, then sends.

When a job reaches any terminal state, it marks its number done. If message 8 finishes while 7 is still thinking, 8 parks in a small "finished ahead" set. When 7 completes, the frontier advances across every contiguous finished number in one atomic script, and 8 is released. The paragraph lands, then "ok?".

Three details carry most of the weight:

  • Done means any ending. Replied, skipped (an emoji-only message produces no bubble) or failed: all of them advance the frontier. The completion runs even when the worker is shutting down, on a context that ignores cancellation, because a swallowed completion makes the next message wait for nothing.
  • The gate fails open. If the ordering state cannot be read, the message goes out immediately. The worst case is the old behaviour, never a dropped or frozen message.
  • It is per chat, and it forgets.The state for a chat expires after ten idle minutes and starts again from zero. Nothing links one chat's order to another's.

Every wait needs a timeout

A gate that waits for a predecessor is a gate that can wait forever. If message 7 is stuck on a model timeout, messages 8 through 40 must not sit behind it.

So the wait is bounded: four seconds by default. When it expires, the waiting worker forces the frontier past the gap and sends. The chat keeps flowing. The abandoned message is not cancelled; if it does finish, it sees the frontier is already past it and posts at once, late and out of order. That is a deliberate ranking: a correct order is better than a slightly late message, and a late message is better than a stalled chat.

The cost is real. A waiting worker holds a slot in the pool. The timeout is what keeps that cost a few seconds of one slot rather than an outage.

The bug the gate faithfully enforced

LINE sometimes batches several events into one webhook delivery. About 1% of our deliveries carry more than one, and when we measured, every such batch was messages from the same chat. Ingest stored the events of a delivery concurrently, and the sequence number is assigned at the end of storing. So within a batch, the numbers went to whichever storage write returned first.

The gate then did its job perfectly and delivered the two translations in exactly the wrong order. An ordering system is only as good as the moment you take the number. Ingest now fans out per chat, not per event: different chats still store in parallel, and same-chat events in a batch go in the order LINE listed them.

Late work: decide before you build retries

Retries, replays and catch-up sweeps all share a property: they deliver old work at a new time. We originally held that a late translation is still a wanted translation.

Then a voice note whose first processing had failed was released from the dead-letter queue three days later. The pipeline did what it was built to do and posted the transcript into the live group, which had long moved on. From inside the chat it looked like a message from nowhere. Every identifier was correct; the timing made it wrong.

Now there is a staleness gate. A message whose send time is more than a limit ago (an hour by default) is closed cleanly without being translated, and posts nothing. The age is measured from the platform's own send timestamp, which never changes, not from when we queued the job, which every replay resets. Commands get a tighter limit of ten minutes. Because a drop is invisible to the sender, it is never invisible to us: drops are counted, and a burst pages.

One honest consequence: requeueing an old dead-lettered item is now close to discarding it, and the admin console says so instead of reporting success.

Parked must mean parked

The last lesson arrived as a metronome. One event was being re-translated roughly every fifteen minutes and twenty seconds, for hours, after it had been parked in the dead-letter queue.

The rhythm was the clue. The dispatcher re-lists the log from its cursor minus a two-minute overlap, to absorb clock skew between machines. An enqueue guard stops it re-queueing the same event for fifteen minutes. And the thing that ends re-reads for good is a small "processed" marker written when work finishes. Parking wrote a dead-letter record but not that marker. The cursor happened to be pinned on that event's own entry, so every time the guard expired, the dispatcher found unfinished work and queued it again. The brake had become the clock.

The staleness gate should have ended it, but those events carried no send timestamp, and the gate fails open without one. And each attempt timed out before the model reported any usage, so the spend never showed in our per-chat cost records.

The fix was one write: parking now leaves the same terminal marker as success, tagged so an operator's explicit requeue can tell the two apart.

Takeaways

  • Order is a property of a conversation, not of the system. A global order would cost far more and buy nothing anyone can see.
  • Decide what happens to late work before you build retries. Otherwise the retry decides for you, three days later, in front of users.
  • Every wait needs a timeout, and every timeout needs a stated winner.
  • Terminal states need one flag that every reader checks. Two flags is a loop waiting for a cursor.
  • The queue is disposable and the log is not. All the ordering state can vanish and the worst outcome is a few messages out of order. That is why it was cheap to build.

ThaiPo is designed, built and run by one developer, Steven Acreman.

Try it in your own chats

ThaiPo lives inside LINE and translates every message in both directions, free forever. Get started free or add @thaipo.ai as a friend on LINE and it sends your signup link right in the chat.