I've been hearing the phrase "human in the loop" more and more recently, and it's quite easy to swallow. But having worked in operational technology and defence for years, it's something I struggle to reconcile, because I'm aware of plenty of fully autonomous systems already in service, weapon systems included. And when someone says "human in the loop" like it settles the question, my honest reaction isn't relief but a question: which loop, and where in it?
What's odd is the timing. This phrase hasn't been steady background noise for decades. It has surged in the last year or two, precisely alongside the current wave of AI: generative systems, autonomous agents, defence AI programmes, all of it. My suspicion is that a lot of that surge is doing emotional work rather than descriptive work. It's shorthand for "don't worry, we're not building the next Terminator", said to produce a feeling that someone responsible is still deciding, rather than to commit to anything specific about how a system behaves.
To be fair to the phrase, it does have a proper home. In machine learning, "human in the loop" means something concrete and slightly unglamorous. A person labels the training data. A person reviews the edge cases the model gets wrong. A person signs off outputs before they go live, or corrects them when they don't. That's real engineering practice, not branding, and I'm not attacking it. It's exactly because that meaning is precise and earned that the phrase carries any weight at all.
The problem is what happens when the phrase travels. It was built to describe a person supervising a model's outputs inside a bounded workflow. Over the last year or two it has been exported onto much higher-stakes territory: autonomy, weapons, life-or-death decisions. It kept its warm, competent connotation and lost the specifics that gave it meaning in the first place. It borrows credibility from a real discipline to reassure people about something that discipline never covered.
You can see how far the borrowing goes by looking at what's already been fielded for decades without anyone using the phrase at all. The US Navy's Phalanx close-in weapon system has been at sea since 1980, first fitted to USS Coral Sea, and it's now on ships across dozens of allied navies. In auto mode it searches, detects, evaluates, tracks, engages and assesses the kill with no human input at any stage, including against manned aircraft that stray into its engagement envelope. The land version, C-RAM, does the same job defending forward operating bases. Aegis has an auto-special mode built for exactly this. Iron Dome, Goalkeeper, the German Army's NBS Mantis, all of them engage without a person approving each individual shot, because nobody can react in the two or three seconds a sea-skimming missile allows.
If "human in the loop" were ever a universal safeguard, it would have had to reckon with these systems a long time ago. It didn't, because until recently nobody needed the phrase for a missile defence gun. It only became ubiquitous once AI came along and people needed reassuring about it.
Part of the issue is that one phrase is being used for at least four different arrangements. Human in the loop, properly speaking: a person approves each individual engagement before it happens. Human on the loop: the system acts on its own, and a person supervises with the real ability to abort. Human out of the loop: nobody is positioned to intervene in time to matter. And the ML-workflow sense, where the human's role is earlier still, shaping the data and correcting the model before it's ever switched on. Most fielded autonomy sits in the second or third category. Used unqualified, the phrase gets stretched to cover all four, which is exactly why it reassures and exactly why it needs pairing with the specifics before it means anything: who has the authority to decide, whether there's time to intervene, whether an override actually exists, what class of target is in scope, materiel or people, and how tightly the operating envelope is bounded.
Phalanx meets a tight version of all of that. It's defensive, not offensive. It targets missiles and shells, not people. Its autonomy is a necessity, not a preference, because no human can react in the time available. And it's closely bounded, switched on inside a narrow envelope, switched off by a person the moment it isn't needed.
Gui Wainwright's work shows what happens when those specifics change. He built the software behind his company, Occam, on paternity leave, in his garden shed, with his daughter in a papoose. It makes drones Ukraine already has autonomous, hardware-agnostic and unjammable, because a system that never receives a signal can't be jammed. The drones wait in cover inside a defined kill box and strike what advances. Wainwright's own description is a sophisticated minefield. The press coverage said it kills with no human interaction. His more careful account is that the human is front-loaded: you set how the system should behave, then you let it run. Occam is offensive, not defensive. It targets people, not materiel. Its human role is design, done once, not supervision, done continuously.
It is far from the first autonomous weapon. MBDA's Brimstone, already in the UK inventory, is salvo-fired into a defined target box and picks its own targets inside it, and the landmine, the oldest autonomous weapon there is, is the comparison Wainwright makes himself. Widen it to a swarm, many platforms coordinating at machine speed under one mission plan, and approving the plan before it runs is a genuinely different act from choosing each target as it appears. Say "human in the loop" about any of that and you've specified almost nothing.
Policy has already worked this out. The US Department of Defense's Directive 3000.09, updated in January 2023, doesn't use the slogan at all. It requires "appropriate levels of human judgment over the use of force" and says plainly that what counts as appropriate depends on context, on the target, the timing, the envelope. That's the fix, written into doctrine: specify, don't just reassure.
So what does the phrase mean? Not nothing, and not always vibes. It means very different things depending on where the human actually sits and how much real power they hold when it matters, and the ML meaning is the only version that comes with that detail built in. Used unqualified about lethal autonomy, it lets everyone skip saying which version they mean, and it still makes us feel better about AI. The question isn't whether there's a human in the loop but which loop, what authority they hold, and whether that's enough for you to feel the reassurance the phrase is meant to give.
Sources
- Department of Defense Directive 3000.09, Wikipedia (the "appropriate levels of human judgment over the use of force" requirement and the January 2023 update). The directive itself: DoD Directive 3000.09, Autonomy in Weapon Systems.
- Phalanx CIWS, Wikipedia (first fully fitted on USS Coral Sea in 1980; searches, detects, tracks, engages and confirms kills automatically).
- Brimstone (missile), Wikipedia (finds targets within a designated area).
- British coder who revolutionised Ukraine's drone army on paternity leave, The Times, July 2026, on Gui Wainwright and Occam.
- Joy Laoun, London-based defense tech startup Occam closes €3M pre-seed round, Vestbee, 11 February 2026. Most coverage of Wainwright's minefield comparison is behind national newspaper paywalls.