IoT hack is not fearmongering...

In its 2023 complaint against Ring, the FTC alleged that security failures let attackers access customers’ accounts, private videos, and camera feeds, and use cameras to harass people in their homes.

Working on device-use agents made me wonder: once access has been obtained, could agents make this kind of misuse easier to repeat or scale? The Ring case doesn’t answer that question, but it shows why the consequences of compromised connected devices deserve attention.

My work has involved multimodal device interaction, execution architecture, and evaluation for on-device testing. A recurring question is whether the system can connect what it observes with the action it takes and the result it checks afterward. An interface can change between observation and execution. An action can run without achieving what was intended. Understanding those differences matters when trying to make an agent dependable.

An interface gives a model a defined set of actions it can request, together with feedback about what happened. With that feedback, it can adjust its next action instead of waiting for someone to guide every step. Operating a device through an authorized interface and breaking into one still require different capabilities. Access, permissions, and the particular device remain important. The concern is what an agent can do once those conditions give it a route to act.

I also don’t think this concern requires the strongest general-purpose intelligence. A system may only need to be good enough at a narrower set of tasks. Knowledge distillation offers one way to transfer learned behavior into a more compact model, though its performance on a particular task still needs to be tested. I would pay attention to what becomes good enough and affordable for device operation, not only the most powerful model.

Dario Amodei has warned about a future persistent botnet at internet scale. Anthropic’s IoT research makes some of the underlying questions more concrete. Researchers ran distributed inference across 15 purchased TV boxes. In a separate replication experiment, agents worked across a firewalled network of simulated Android devices with unauthenticated access. The model ran on a separate server, and the researchers’ setup started the next agent copy after checking the target’s inference server. These experiments explore whether models can run on limited hardware and carry out parts of a replication sequence. The simplified access and orchestration leave a gap between those results and an autonomous IoT worm spreading in the wild.

Deliberate misuse is one route to harm. An agent carrying out an ordinary request could also be steered by untrusted content. NIST’s analysis of an agent red-teaming competition examines this problem through indirect prompt injection: malicious instructions embedded in material an agent reads can redirect its actions. For an agent with access to a device, that makes the boundary between external content and authorized instructions especially important. Anthropic’s computer-use documentation likewise recommends isolating agents from sensitive data and actions.

For me, this brings the question back to the boundary around the agent: what it can reach, what it is allowed to change, and how we know what it actually did. I’m still learning how to think about those boundaries. But after working on the connection between reasoning and device operation, I don’t find the possibility of harm far-fetched. It deserves careful engineering rather than either panic or dismissal.