Device shadow design: keeping apps and devices in sync
Practical device shadow design: desired vs reported state, versioning, conflicts and sleepy devices, so your app and hardware stop disagreeing.

A user taps “heating on” in the app, the device is asleep, and the toggle snaps back to off. Or it stays on, while the thermostat on the wall carries on doing something else. Most of these bugs are not radio problems. They are a missing model of state, and good device shadow design is how you fix them.
This post is for product teams building a connected device with a companion app or dashboard. We cover how we structure a shadow, how we handle versions and conflicts, and what we keep out of it.
What a device shadow actually is
A device shadow (Azure calls it a device twin) is a small document held in the cloud that describes a device’s state. It sits between the device and everything that wants to talk to it: the mobile app, the dashboard, automations and support tools. None of them talk to the device directly. They read and write the shadow.
The core idea is splitting state into two halves:
- Desired: what the user or the backend wants. Written by the app and the cloud.
- Reported: what the device says is true right now. Written only by the device.
The difference between the two is the delta. When a device connects, it fetches the delta, applies what it can, and reports its new state. The app shows reported state as fact and desired state as “pending” until they match. That single rule removes most of the toggles that flick back and forth.
AWS IoT Core and Azure IoT Hub both provide this out of the box. If you run your own broker and backend, a shadow is a table, a version column and a few MQTT topics. The pattern matters more than the platform.
Device shadow design: what belongs in the document
The most common mistake we see is treating the shadow as a dumping ground. It becomes a 6 KB blob of telemetry, logs and configuration, updated every few seconds, and every update wakes every subscriber. Keep it to state that has a meaning when the device is offline.
| Put in the shadow | Keep out of the shadow |
|---|---|
| Setpoints, modes, schedules | Sensor time series |
| User-facing configuration | Logs and crash dumps |
| Firmware version, hardware revision | Rapidly changing values (every few seconds) |
| Connectivity summary, last-seen battery level | Secrets, keys and credentials |
| Target firmware version for OTA | Large blobs such as images or certificates |
Telemetry goes on its own topics into a time-series store. The shadow can carry the latest summary value, say battery percentage at last report, so the app can render a device card without querying history. Keep documents small: a few hundred bytes to a couple of kilobytes is typical for a well-scoped device, and managed platforms impose hard size limits that you do not want to discover in the field.
Group fields by owner
We nest the document by who is allowed to write each part: config (user-editable), system (backend-only, e.g. target firmware) and status (device-only, reported). Access rules then become simple. The app can write desired.config and nothing else, which stops a buggy app build from scheduling a firmware update.
Versions, conflicts and ordering
Two writers, unreliable links and messages that arrive out of order: you need versioning from day one, not after the first support ticket.
- Version the whole document. Every accepted write increments a version number. Clients send the version they last saw; the backend rejects a write based on a stale version, and the client refetches and retries. This is optimistic locking and it is cheap.
- Ignore stale deltas on the device. The device stores the last version it applied and drops anything older. Without this, a delayed message after reconnection can roll a setpoint back.
- Merge, do not replace. Writes should be partial updates of the fields that changed. If the app sends the whole config to change one value, it will overwrite a schedule edited on the dashboard a second earlier.
- Clear desired once applied. When reported matches desired, the backend can null out that desired field. A permanent desired value means a factory reset on the device gets silently undone at next connect, which may or may not be what you want. Decide deliberately.
Timestamps alone are not a reliable ordering mechanism. Device clocks drift, RTCs reset on brown-out, and phone clocks are set by users. Use server-assigned versions for ordering and keep timestamps for display.
Sleepy and offline devices
Battery devices are where shadows earn their keep. A sensor that wakes every 15 minutes, or a LoRaWAN node with a tight downlink budget, cannot accept commands on demand. The shadow queues intent for it without the app needing to know.
On wake, the firmware sequence we use is short: connect, request the delta (or subscribe and receive it), apply changes in one pass, report the result including any field it rejected, then sleep. Keep the delta request in the same connection window as the telemetry upload so the radio is on once. On LoRaWAN, where downlinks are scarce, we send only a compact delta and often a single config version number the device compares against its own.
The app needs to say all of this honestly. Show “will apply when the device next connects, usually within 15 minutes” rather than a spinner that times out. This is a UX decision as much as a backend one; we covered the phone side in building a BLE companion app that survives real phones.
Validation and rejected changes
The device is the final authority on what it can do. A user may request 35 °C on a heater limited to 30 °C, or a schedule with more slots than the firmware supports. Validate in the app for fast feedback, validate again in the backend, and still let the device reject.
When the device rejects a value it should report what it actually applied and a short reason code, for example status.lastError: "setpoint_out_of_range". Without this, desired and reported never converge, the app shows “pending” forever and support has nothing to go on. Clamp silently only where the clamp is obvious to the user.
Topics, transport and the rest of the backend
On MQTT, a shadow typically uses a small set of topics per device: get, update, accepted, rejected and delta. Fit these into your wider topic scheme rather than bolting them on; our notes on MQTT topic design for a growing device fleet explain the structure we use. Use QoS 1 for shadow updates and make every handler idempotent, because QoS 1 means a message may arrive more than once.
Behind the broker, store the shadow in your main database, not only in the broker’s memory. You will want to query it: every device on firmware below a given version, every device whose desired and reported have differed for more than a day. That second query is one of the most useful support tools you can build.
What we would do on a new product
A short checklist before writing the first line of shadow code:
- List every field, who writes it and whether it makes sense while the device is offline.
- Split desired and reported, and let only the device write reported.
- Add a document version, optimistic locking and stale-delta rejection on the device.
- Send partial updates only, and decide when desired values are cleared.
- Define reason codes for rejected changes and show them in the app.
- Keep telemetry, logs and secrets out; keep the document small.
- Design the “pending” state in the app as a first-class screen, with an honest time estimate.
- Build the “desired ≠ reported for too long” query before launch, not after.
Get these right and the app, the dashboard and the device stop arguing. Most of the work is deciding ownership early; the code itself is small.