Our whole backend is files in a bucket: a Kappa architecture on object storage

14 July 2026 · ThaiPo

ThaiPo translates LINE and WhatsApp chats in real time for people who share no language. Behind it there is no Kafka, no event bus, no message broker product at all. The entire source of truth is an S3-compatible object store: every inbound message event and every translation we produce is one immutable object in a bucket. Everything else in the system, the queues, the caches, even most of the database, is a disposable view that can be rebuilt from those objects.

This is a Kappa architecture, the "one log, many readers" idea usually built on Kafka, except the log is a bucket. For a small company we think it is dramatically underrated, so here is the shape, what it makes easy, and what it costs.

The shape

Log-first architecture diagram: events land in an append-only log on object storage, a fast path serves them via queues, a slow path compacts and learns, every downstream store is rebuildable

  • The log. Webhook events append to the bucket as small JSON objects, keyed by a shard, a conversation id, and the date. Translations are written back next to the events they answer. Objects are never mutated; corrections are new objects.
  • The fast path. Ingest verifies the webhook, appends to the log, and pushes a job onto a sharded Redis queue. A translator worker picks it up, translates with full per-chat context, replies, and archives the result. End to end this is seconds, and Redis is doing what Redis is great at: hot, small, ephemeral work.
  • The slow path. A separate learning plane walks the same log later, on its own schedule: compacting days into summaries, mining vocabulary a chat actually uses, running quality evals, aging out data past retention. It reads the identical objects the fast path wrote. That is the Kappa property: one log, readers at different speeds, no second pipeline to keep consistent.
  • Postgres holds only derived account state. Users, subscriptions, per-chat settings, usage counters, learned glossaries. It holds no message bodies at all. If you are looking for a message, it is in the bucket; if you are looking for a fact about an account, it is in Postgres.

What this buys a small team

  • Redis is allowed to die. Queues, caches, and leases are rebuildable from the log, so cache failure is an inconvenience, not an incident. Nothing precious lives anywhere hot.
  • Bugs become replays. When a translator bug mangles a day of output, the fix is: patch the code, replay that slice of the log through the worker, done. The log is the test fixture for free, and it is the real one.
  • Disaster recovery is a copy job. Sync the bucket (plus a Postgres dump for the account state) somewhere else and you hold the whole company. Ours syncs off-provider nightly; restoring is mechanical because nothing irreplaceable lives outside the log.
  • New features read history without asking permission.Per-chat memory, retrieval, analytics backfills, eval corpora: they all walk the same objects the product already writes. No "we did not log that" regrets, and retention is enforced in exactly one place.
  • It is absurdly cheap and boring. Object storage is pennies per gigabyte-month, there is no broker cluster to size or upgrade, and our two Go services ship as distroless containers with nothing in them but the binary.

The honest tradeoffs

  • It is not a low-latency bus. The bucket is the source of truth, not the dispatcher; that is why Redis queues sit in front for the seconds-scale hot path. If you need sub-millisecond fan-out, this is not that.
  • Consumers must be idempotent.Webhooks redeliver, replays reprocess, and at-least-once is the contract everywhere. Every consumer treats "seen it before" as normal.
  • Small objects have overhead. Millions of tiny JSON files cost more in requests than in bytes, and listing is the slow operation you design around (hence date-based keys, so every reader scans narrow prefixes). The slow path compacts where it matters.
  • Privacy work is real work. An append-only log of conversations means retention and deletion need to be first-class: media objects expire on short lifecycle rules, text follows documented retention, and deletion means deleting log prefixes, not flipping a flag.

When we would choose it again

Every time, at this scale. The pattern fits any product whose truth is "things that happened": chats, orders, telemetry, documents. If your event rate fits comfortably in "thousands per minute" rather than "millions per second", the boring version of Kappa, a bucket full of JSON plus a queue you are allowed to lose, gives you replay, DR, and an audit trail for roughly the price of hosting a static site.

ThaiPo itself is the demo: it remembers months of a chat's in-jokes and vocabulary because the log makes history a feature, not a liability. If you want to see what that feels like in a real chat, add it to a LINE group.

Try it in your own chats

ThaiPo lives inside LINE and translates every message in both directions, free forever. Get started free or add @thaipo.ai as a friend on LINE and it sends your signup link right in the chat.