← Journal Software · · 7 min

MQTT topic design for a growing device fleet

MQTT topic design decides routing, permissions and how painful firmware changes get. The tree we use for connected device fleets, and why.

Milo Perović
Founder & lead engineer

  • Architecture
  • Cloud
  • Fleet
  • Security
MQTT topic design for a growing device fleet

MQTT topic design is one of those decisions that takes ten minutes on day one and costs weeks to undo in year two. Most connected products start with whatever topics the first firmware build happened to publish, and that works fine for a handful of devices on a bench. It stops working when you add a second customer, a mobile app, per-device permissions and a fleet you cannot reflash in an afternoon.

This post is for teams about to commit to an MQTT broker, or who suspect their topic tree is quietly turning into a problem. Here is the structure we use and why each part is there.

Why MQTT topic design is an architecture decision

A topic string looks like a label, but in practice it carries three jobs at once. It is the routing key that decides which subscribers receive a message. It is the boundary your broker uses for access control. And it is a contract baked into firmware that may run unchanged for years.

That last point is what makes it expensive. You can migrate a database schema overnight. You cannot change the topic a device publishes to without shipping an update to every unit, and devices that miss the update will keep using the old topic until someone visits them. So the tree you choose before the first production batch is, for most fleets, the tree you live with.

A few rules from the MQTT specification shape everything else:

  • Levels are separated by /, and topics are case-sensitive UTF-8 strings.
  • + matches exactly one level; # matches any number of levels and must come last.
  • Topics starting with $ (such as $SYS) are reserved for the broker and are not matched by a subscription that begins with a wildcard.
  • A leading slash creates an empty first level, so /devices/42 and devices/42 are different topics. We avoid leading slashes entirely.

A structure that scales with the fleet

The pattern we reach for puts the most stable, most selective identifiers first and the message type last:

v1/{tenant}/{device_id}/{channel}[/{detail}]

In practice that gives topics such as v1/acme/dev-7f3a91/telemetry or v1/acme/dev-7f3a91/cmd/reboot. Each level earns its place:

  • Version prefix. It costs a few bytes and gives you a clean way to introduce a new payload format alongside the old one, rather than in place of it.
  • Tenant. Even if you only have one customer today, a tenant level means a backend service or a customer integration can subscribe to v1/acme/# and nothing else.
  • Device ID. Use an identifier that never changes and never leaks meaning: a serial or a random ID assigned at provisioning, not a MAC address, a site name or a customer-facing label.
  • Channel. A small, fixed vocabulary describing what kind of message this is. More on that below.

What we deliberately leave out of the topic: location, firmware version, sensor model and anything else that is an attribute of the device rather than its identity. Those belong in the payload or in the backend’s device registry. Putting them in the path means a device has to change topics when it is relocated or updated, which breaks every subscription and permission that referenced the old path.

Separate telemetry, state, events and commands

The single most useful thing a topic tree can do is make the direction and meaning of a message obvious from its path. We keep a short list of channels and resist adding new ones casually.

ChannelDirectionTypical QoSRetainedUsed for
telemetryDevice to cloud0 or 1NoPeriodic measurements
stateDevice to cloud1YesCurrent configuration and status
eventDevice to cloud1NoAlarms, faults, button presses
cmdCloud to device1NoActions the device should take
cmd/ackDevice to cloud1NoResult of a command

The split matters more than the exact names. Telemetry is high-volume and individually disposable, so QoS 0 is often fine and saves airtime on constrained links. Events are rare and important, so they get QoS 1 and their own topic, which lets an alerting service subscribe to v1/+/+/event without wading through every temperature reading.

Commands deserve particular care. QoS 1 means at-least-once delivery, so a device can receive the same command twice. Include a command ID in the payload, have the device acknowledge it on cmd/ack with that ID, and make handlers idempotent. With MQTT 5 you can use the response topic and correlation data properties for the same purpose, but the principle holds whichever version you run.

Retained messages, wills and online status

A broker keeps at most one retained message per topic and hands it to any new subscriber immediately. That makes retained messages ideal for state: a dashboard that connects at 09:00 sees each device’s last known configuration without waiting for the next report. It also makes them dangerous for anything else. A retained command will be re-delivered every time the device reconnects and resubscribes, which is how a device ends up rebooting in a loop.

Our rules are simple:

  1. Retain state and a dedicated status topic. Never retain commands or telemetry.
  2. Set a Last Will on connect that publishes offline as a retained message to status, and publish a retained online immediately after connecting. The broker sends the will if the connection drops without a clean disconnect.
  3. When a device is decommissioned, clear its retained messages by publishing an empty retained payload to each topic. Otherwise ghosts of old hardware appear in every new dashboard.

Bear in mind the will only fires once the broker notices the connection is gone, which depends on the keep-alive interval. On a battery device with a long keep-alive, “offline” may arrive minutes late; your dashboard should treat it as a hint rather than a precise timestamp.

Access control follows the topic tree

Most brokers, managed or self-hosted, express permissions as topic patterns. A good tree makes those rules short and obviously correct. With the structure above, a device’s policy reduces to “publish to v1/{tenant}/{own_id}/telemetry|state|event|status|cmd/ack, subscribe to v1/{tenant}/{own_id}/cmd/#“, with the tenant and ID substituted from the device’s certificate or credentials.

A common failure in inherited projects is a device credential allowed to subscribe to # because it was convenient during development. One extracted key then exposes every other customer’s data. If your topic tree cannot express “this device, only its own topics” in a single rule, that is a sign the tree needs work before the broker does. We covered how this fits with the rest of the platform in one IoT backend for the device, dashboard and mobile app.

Changing topics without breaking the fleet

Sooner or later you will need a new payload format or a new channel. The version prefix is what makes this survivable. The sequence we follow:

  1. Teach the backend to read both v1 and v2 topics and normalise them into the same internal model.
  2. Roll out firmware that publishes to v2, in stages, using your normal update process. Our notes on secure OTA updates for small fleets cover the staging side.
  3. Watch the count of devices still on v1 fall, and keep the old path supported until it reaches zero or the remaining units are formally retired.

What we would do before the first production batch

  • Write the topic tree down as a one-page document with every channel, its direction, QoS and retain flag.
  • Start with a version prefix, then tenant, then a stable opaque device ID.
  • Keep attributes such as location and firmware version out of the path.
  • Separate telemetry, state, events and commands, and give commands IDs and acknowledgements.
  • Retain state and status only, and set a Last Will on every connection.
  • Express each device’s permissions as one rule scoped to its own subtree, and test that it cannot read a neighbour’s topics.
  • Check your broker’s limits on topic length, depth and subscriptions per client; managed services typically impose them.

None of this is complicated, but all of it is cheaper on a whiteboard than in a fleet. An hour spent on the topic tree now is an hour you do not spend later writing migration code for devices you can no longer reach.

Working on something similar?Free scoping call →