Series · Testing and building for the Internet of Things · Part 2 of 3

An end-to-end test harness

Simulated devices, a broker in the middle, and a framework that checks every hop: how to automate an IoT system’s tests.

2018 · 3 min read

Take a typical small IoT system. A sensor notices something in the physical world. A device reacts, perhaps a camera that captures an image. A service processes what arrived, maybe with machine learning, and decides. Something tells a person: a web dashboard, an API call, an email. In between, a broker such as MQTT carries the messages.

Testing it with conventional strategies is hard, for three reasons that show up almost every time.

  • Hardware and network. Real devices need a dedicated lab.
  • Mixed technologies. IoT, AI, web and email, each with its own testing habits.
  • Mixed protocols. The server may speak MQTT to devices and SMTP or POP3 to mail at the same time.

Plan by feature and technology

Start by naming every part of the system and the technology it uses, then choose a strategy per part. The result is a test plan that looks more like a map than a list.

The harness

Simulated devicesSensorCameraMeterMQTT brokerSystem under testservices, rules, AIOutputsAPI · web · emailpublishTest frameworkJava · TestNG · protocol librarysubscribe, assertverifymock dataHTML report · CI job
The harness sits around the broker. Simulated devices make the lab repeatable; the framework checks the data on every hop and the outputs at the end.
  • A dedicated lab that covers the whole workflow, end to end.
  • Simulated devices that publish and consume on the broker using MQTT and ordinary libraries (Java works well). They make the lab repeatable and let you create situations real devices rarely produce.
  • A modular, protocol-based IoT library, reusable for testing other IoT systems, not just this one.
  • API and web automation for the software around the hardware.
  • Generated test data for edge cases, especially for the AI features, whose failures live at the edges.
  • Mail libraries to automate the checks on email features.
  • Configuration, HTML reports and a CI job (TestNG and Jenkins, for example), so the suite runs on every change.

What each test checks

The tests follow the data along its path:

  1. the sensor is broadcasting;
  2. the right value arrives when an object is in place;
  3. the device acts when it should;
  4. the processing extracts what it should;
  5. the decision is correct for the input;
  6. the person is notified.

Each step is an assertion on a message the framework can see on the broker, or an output it can read.

What it buys

A full regression suite for every feature, high automated coverage, faster development and delivery, and a secure environment for testing systems that must stay confidential.

Where it goes next

The same harness grows with the system: smarter detection, retrying a capture when the first one is poor, configuration instead of code, moving heavier processing onto the IoT network, identity checks that double as login, and, always, more security.

Next: an edge AI system.

Drafted in 2018
Updated for site in 2026