The Internet of Things is, put simply, devices (things) connected over the internet. More formally, it is the network of physical objects (devices, vehicles, buildings and more) embedded with electronics, software, sensors and connectivity, so that they can collect and exchange data.
That exchange lets businesses solve problems at a far larger scale, through observation and action. Imagine a municipal corporation managing a city: traffic lights and street lights, sanitation, water, electricity. Instead of manual checks at fixed intervals, it can watch live data from water tanks, street lights and the rest, and manage risk, repairs and resources far better. Agriculture, healthcare, rail networks and astronomy can use the same idea.
As IoT grows, the question of how to assure its quality grows with it.
The challenges
At first sight, testing an IoT application outside the lab looks impossible, and automating it even inside the lab looks like science fiction. The reasons:
- Scale. What makes IoT valuable to the business is what makes it hard for QA, especially when assuring real environments. Simulated environments help in the early stages.
- Variety. One application can involve many kinds of device, hardware and software. QA engineers need expertise at several levels at once.
- The internet itself, which is
- lossy: few guarantees on data consistency or staying connected; expect corrupted data and dropped connections;
- open: your data travels where everybody is, so security matters most and is hardest at this scale and variety;
- latent: more connections over longer distances hurt real-time performance, and testing over the internet raises false positives.
In the city example, manual QA would waste people, money and time, and still be inaccurate. Street lights, traffic lights, water, sewage, power and weather systems all work differently and rarely speak to each other. Most of their data is needed in real time. And none of it should reach anyone unauthorised.
The approach
- Automation is a necessity. Nothing manual can keep up with the scale and variety. And because of what IoT systems are, the automation has to be intelligent, which makes a strong case for an AI-based framework.
- The framework must speak IoT: its protocols, its rules and its failure modes.
- Simulate early. Simulated environments find major defects early, though building them is work.
- Domain knowledge is a must. Test scenarios have to come from the business domain, and so do the rules of any simulation.
- Functional testing, as for any application.
- Performance testing, because real-time exchange at scale, over a latent network, is the requirement in critical domains like transport or healthcare.
- Security testing, defined deliberately and kept current, because every exchange crosses the internet.
- Compatibility testing, early in design and planning, so sub-systems, environments and protocols fit before effort is wasted.
- Exploratory testing, because systems this vast hide loopholes, and an AI-based framework can keep watching the system's health where people cannot.
For the city, that means an intelligent automation system built with domain experts, developers and automation engineers, which keeps the system secure, functionally sound and within its performance targets, with compatibility and exploratory testing built into the flow.
The future of IoT is bright, and its applications will be complex: physical scale, many kinds of objects, strict security and performance. That makes QA both essential and hard, and the way through is automation and AI used well.
Next: an end-to-end test harness.
Drafted in 2018
Updated for site in 2026