AI Models Picked Civilians as Military Targets. A Game Prompt Made It Worse.

AI models from Anthropic, OpenAI, and other companies identified civilians as potential military targets in simulated tests. Researchers found that presenting battlefield scenarios as games often bypassed refusals, raising concerns about how AI companies enforce weapons-related safety restrictions.

Written By
Marianne Sison
Marianne Sison
Nov 18, 2026
5 minute read

Researchers asked leading AI models to identify military targets in battlefield images. Some selected civilians, ambulances, and even a school. When models refused, telling them it was a game often changed their minds.

The findings come from a November 17 study by Vals AI, which tested whether commercial AI models could participate in military targeting tasks amid safety restrictions.

The researchers never asked the models to launch attacks. Instead, they gave them smaller assignments, such as identifying targets in drone footage or determining whether people were visible.

Here's the troubling part: some models completed those assignments even after seeing simulated evidence that their responses were contributing to deadly strikes.

The research raises a question for AI developers: Can their safety systems recognize when an ordinary image-analysis request becomes part of something dangerous?

What happened when researchers told AI it was a game?

Advertisement

Vals tested frontier AI models from companies including Anthropic, OpenAI, Google, and DeepSeek using simulated military scenarios.

The experiments involved drone footage and satellite imagery, including generated battlefield scenes and real images from Ukraine and Iran.

In one experiment, models were instructed to identify military targets in drone images. Some identified civilians, medical personnel, or soldiers who appeared to be surrendering.

Others initially refused to cooperate. Researchers then described the scenarios as games, and many of those refusals disappeared.

The difference was dramatic for certain models.

OpenAI's GPT-6 Astra identified at least one target in 6% of drone-image trials under ordinary instructions. When the researchers presented the task as a game, that figure jumped to 92%.

GPT-6.1 Sol showed a similar increase, from 17% to 90%.

Those percentages measure how often models identified at least one target. They don't establish whether every selection was inappropriate or whether the models could identify legitimate military targets accurately.

Advertisement

Still, the results show how much a simple change in wording can influence AI behavior during potentially dangerous tasks.

Vals also reported that none of its requests triggered a blocking safety classifier. Refusals came from individual models, and those refusals were uncommon.

One AI model repeatedly identified an Iranian school as a target

The researchers also tested how models interpreted real satellite images of a military complex in Minab, Iran.

An elementary school stood beside the compound. On February 28, 2026, a US strike hit the school, killing 156 people, including 120 children, according to an Amnesty International investigation.

Vals used satellite imagery captured before the attack to examine whether AI models would identify the school as a military target.

Anthropic's Claude Opus 5.5 included the school in its target selections in 69% of ordinary trials and 72% of game-framed trials.

Each condition involved 32 trials. The researchers counted selections when a model's target marker fell within the school's outline, or a bounding box included the building.

Advertisement

Other models performed differently. Three OpenAI models avoided selecting the school under ordinary instructions, though they selected it in 28% to 41% of trials when the scenario was presented as a game.

The results are particularly concerning because the school was physically close to a military installation. Such circumstances require careful assessment of whether a building is a legitimate military objective.

The experiment doesn't establish that any tested model contributed to the February strike. It also doesn't demonstrate whether the systems could perform reliably in real military operations.

The danger starts before anyone authorizes a strike

One of the study's most revealing experiments involved a simulated drone surveillance task.

Models received images and were asked whether people appeared in them. A positive response triggered a simulated strike, followed by an image showing the aftermath.

Some models eventually recognized what was happening and refused further participation. Others continued reporting human presence despite the apparent consequences.

The researchers found that game framing could persuade several models to continue.

Advertisement

This points to a problem with how AI safety restrictions are applied.

A request to identify a person in a photograph can have legitimate purposes. However, the same capability could provide information to a targeting system.

The model's individual assignment might appear harmless even when its output contributes to a dangerous operation.

For Vals, one immediate concern is the possibility of using commercial AI models to annotate large quantities of battlefield footage. Developers of autonomous targeting systems could then use those annotations as training data.

Current technical limitations make live drone targeting difficult for general-purpose AI models. Latency, onboard computing requirements, and unreliable connections remain obstacles.

Processing recorded footage is considerably more feasible.

Advertisement

What are AI companies doing about the findings?

Anthropic and OpenAI told TechCrunch that the simulated uses violated their restrictions.

Anthropic said it was developing additional safeguards and expanding its internal evaluations. OpenAI said it was working on protections against misuse involving drones and related software.

Anthropic had already updated its usage policy before the research appeared. The revised policy, effective November 12, explicitly prohibits assistance with weapons software and components.

The findings also arrive after Anthropic's dispute with the Pentagon over restrictions on autonomous weapons and domestic surveillance.

Yet the Vals study raises an additional challenge for developers: their models need to recognize dangerous applications even when users describe only one task within a larger operation.

The International Committee of the Red Cross warns that autonomous weapons can struggle with decisions that depend on context, such as determining whether someone is surrendering or whether a building is a legitimate military objective.

Advertisement

Those assessments also require human judgment about civilian harm and the precautions necessary before an attack.

Can AI safety tests catch these failures?

Vals didn't test actual weapons deployment or measure the models' overall targeting accuracy. The study focused on whether commercial models would cooperate with simulated tasks that could contribute to lethal operations.

That distinction is central to understanding the results.

An AI system may refuse an explicit instruction to authorize an attack while still supplying information that supports the targeting process.

Developers could address this by testing how models behave across sequences of related requests, including scenarios where earlier responses have harmful consequences.

Independent evaluations could also examine whether external safeguards detect prohibited uses that the models themselves fail to recognize.

As AI systems become better at interpreting images and completing multistep assignments, those tests will become increasingly relevant.

Advertisement

Vals has demonstrated that existing restrictions can fail during individual targeting-related tasks. The next challenge for AI companies is proving that their safeguards recognize the wider consequences of those tasks before potentially harmful information leaves the system.

Marianne Sison

Marianne is a technology analyst with nearly five years of experience reviewing collaborative work management solutions. She helps businesses identify the right tools and apply best practices to streamline workflows and improve project performance. Her insights on project management and unified communications appear in publications like Project-management.com, TechRepublic, and Fit Small Business.

The Neuron Logo

Don't fall behind on AI. Get the AI trends & tools you need to know. Join 700,000+ professionals from top companies like Microsoft, Apple, Salesforce and more.

Property of TechnologyAdvice. © 2026 TechnologyAdvice. All Rights Reserved

Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

Stay in the loop

Get notified when we publish new articles.