← Journal Testing · · 7 min

Hardware-in-the-loop testing for small firmware teams

Hardware-in-the-loop testing runs real firmware on real boards every night. Here is a rig a small team can build, and the tests that pay back first.

Milo Perović
Founder & lead engineer

  • OTA
  • Power
  • QA
  • Testing rigs
Hardware-in-the-loop testing for small firmware teams

Most firmware bugs that reach the field were never hidden from tests. They were hidden from the tests a team actually runs. Hardware-in-the-loop testing closes that gap by running your real firmware on the real board, automatically, while a test rig drives its inputs and checks its outputs.

This post is for small teams building connected devices who have unit tests on the desk, a pile of manual checks in someone’s head, and an uneasy feeling before every release. We will cover what a useful rig looks like, what it costs in effort, and where it pays back first.

What hardware-in-the-loop testing catches that unit tests miss

Unit tests on a host PC are fast and worth having. They prove that a parser parses and a state machine transitions. What they cannot prove is that the firmware behaves when the peripherals, interrupts, clocks and power rails are real.

The bugs we see slip past host-only testing tend to fall into a few families:

  • Timing and concurrency. A DMA transfer that completes before the task waiting on it starts listening, or an ISR that occasionally runs longer than its budget.
  • Peripheral configuration. A wrong I²C clock stretch setting, an ADC sampling time that is too short for a high-impedance divider, a UART that drops bytes at 921,600 baud.
  • Power and reset paths. Brown-out during a flash write, a watchdog that never fires because a low-priority task still kicks it, a wake-up source that works once and then not again.
  • Integration regressions. A change to the BLE stack configuration that silently breaks provisioning, or an RTOS heap that runs out only after the third OTA attempt.

None of these show up in a mocked environment, and most of them are intermittent. That is exactly why they need a machine running them hundreds of times rather than an engineer running them once.

A minimal rig on a bench budget

A HIL rig does not need a rack of instruments. For a typical battery-powered device built around an ESP32-S3, STM32L4 or nRF52840, we start with parts most labs already own:

  1. The device under test, ideally a production-intent board rather than a development kit, mounted on a fixture or held with pogo pins.
  2. A test controller. A Linux single-board computer or a spare PC that runs the test framework and talks to everything else.
  3. A debug probe such as a J-Link, ST-LINK or CMSIS-DAP probe, used to flash builds, reset the target and read memory or fault registers when a test fails.
  4. Controlled power. A bench supply with a remote interface, or a cheaper USB relay board that can cut and restore the supply. Being able to vary the voltage lets you test brown-out behaviour deliberately.
  5. Stimulus and observation. A microcontroller board acting as a fake peripheral, a USB GPIO adapter, and a low-cost logic analyser for capturing protocol traffic when something goes wrong.
  6. Current measurement. A dedicated power profiler or a shunt and amplifier, so sleep current becomes a test assertion rather than a one-off measurement.

The total hardware cost is usually modest compared with a single week of engineering time. The real investment is in the software that ties it together, and in keeping the rig as reliable as the product it tests.

Simulating the outside world convincingly

The hard part of hardware-in-the-loop testing is not flashing firmware. It is feeding the device inputs that look like the real world and measuring its response without the rig itself becoming the source of bugs.

Sensors and analogue inputs

For digital sensors, a second microcontroller can emulate the sensor as an I²C or SPI target, returning scripted values, delayed responses or deliberate NAKs. That lets you test out-of-range readings and bus errors that a real sensor on the bench will almost never produce. For analogue inputs, a DAC or a digital potentiometer driving the input through the same divider used in the product gives repeatable voltages.

Radios and the cloud

For BLE products, the test controller can act as the central with a USB dongle and run the same pairing, provisioning and data-transfer flows as the companion app. For Wi-Fi or cellular products, point the device at a local MQTT broker or a staging backend that the test can inspect. Keep a test that runs against a real phone and a real network too, but run it less often.

Faults on purpose

The most valuable tests are the unpleasant ones: pulling power during an OTA write, holding a sensor line low, flooding the UART, dropping the broker connection mid-message. If your firmware watchdog design is meant to recover from a hung task, the rig should hang that task and confirm the device comes back.

Writing HIL tests that are not flaky

A rig that fails randomly gets ignored within a month. Flakiness is the main reason HIL efforts die, so we treat test reliability as a design goal from the first day.

  • Start every test from a known state. Power-cycle or hard-reset the device, flash or verify the build, and clear persistent settings unless the test is about persistence.
  • Wait on events, not on sleeps. Poll for a log line, a GPIO edge or an MQTT message with a sensible timeout, instead of sleeping for a fixed number of seconds and hoping.
  • Expose a test interface in firmware. A small command shell over UART or RTT, compiled into test builds, lets the rig query state directly rather than inferring it from LEDs.
  • Capture everything on failure. Device logs, the probe’s view of fault registers, the logic analyser trace and the supply current. A failing test that leaves no evidence costs more time than it saves.
  • Quarantine, then fix. Move a flaky test out of the blocking set straight away, log it, and fix the cause within the week. Never just add retries.

It also helps to separate the rig’s own self-test from product tests. Before each run, confirm the controller can see the probe, the relays switch and the fake sensor answers. When those checks fail, report a rig fault rather than a product failure.

Running the rig in continuous integration

A HIL rig earns its keep when it runs without anyone asking. The usual pattern is a self-hosted CI runner on the test controller, triggered by the same pipeline that builds the firmware.

WhenWhat runsTypical duration
Every merge requestSmoke set: boot, connect, core features, sleep current5–15 minutes
NightlyFull functional set plus fault injectionOne to a few hours
Before a releaseSoak runs, repeated OTA cycles, power-cycle loopsOvernight to several days

Durations vary mostly with radio connection times and how long the device takes to reach a stable sleep current. Keep the merge set short enough that engineers wait for it, or they will learn to merge around it.

One rig per product is the minimum. If several engineers share it, add a simple booking or queue in CI so that two pipelines never flash the same board at once. Much of the fixture and harness code also carries over to end-of-line testing for production, so design the two with each other in mind.

What we would do on a new product

You do not need the full rig before the first prototype. We grow it alongside the firmware, and the order matters more than the completeness:

  1. Automate flash, reset and log capture first. Even with no assertions, this alone removes a lot of manual work.
  2. Add a boot-and-connect smoke test and run it on every merge.
  3. Turn sleep current into an assertion with a sensible threshold, because it regresses quietly and costs battery life.
  4. Add power-cycle and interrupted-OTA tests before the first field units ship.
  5. Emulate sensors and inject faults for the failure modes your requirements actually call out.
  6. Schedule a soak run before every release and read its results, not just its status.

Hardware-in-the-loop testing is not glamorous, and the first version of a rig always feels like a distraction from features. But the bugs it catches are the expensive ones: intermittent, power-related and invisible until a device is installed somewhere hard to reach. Build the rig early, keep it boring and reliable, and let it run every night.

Working on something similar?Free scoping call →