The world banned landmines and cluster munitions. Can it stop killer AI before it is too late?

Throughout the history of war, the machinery of killing has always left room for a human to intervene.

A soldier could see the person in front of them and choose to maim rather than kill; a pilot could identify the target, question the intelligence and decide not to strike.

Technology has been steadily changing that. Autonomous weapons systems now threaten to remove any meaningful human control over the use of force in conflict.

AI-assisted targeting was first deployed by Israel in its genocidal war on Gaza and thenin Iran, with strikes happening at unprecedented speed – over 1,000 hits in the first 24 hours of the US-Israeli war, suggesting the use of automated systems.

For more than 13 years, the global coalition Stop Killer Robots has been fighting to prevent that future, pushing governments towards a ban on machines capable of selecting and engaging targets without oversight.

New MEE newsletter: Jerusalem Dispatch

“The systems are designed to kill, so there are no checks and balances anymore,” executive director Nicole van Rooijen told Middle East Eye.

“If a system is designed to kill, it will kill, right? And that in itself is problematic as it is very much contingent on the type of information that is fed into the machine.”

Van Rooijen pointed to the US air strike on Shajareh Tayyebeh Primary School in the Iranian city of Minab on 28 February, in which 156 people, including 120 children, were killed, as an example.

Evidence indicates that US forces hit the building, located next to an Islamic Revolutionary Guard Corps (IRGC) compound. To this day, that attack is the deadliest throughout the US-Israeli war on Iran.

Inaccuracy that kills

“Preliminary information is that it is a US strike, but also that it is due to outdated intelligence fed into the machine,” Van Roijen said.

“If the information is not completely accurate or is outdated, it has devastating consequences on the ground, most likely for innocent civilians.

‘It is false promises of being more accurate, more precise, yet the results are worse, not for their militaries but for civilians on the ground. They pay the brunt of it’

– Nicole van Roijen, executive director of Stop Killer Robots

“This is especially true in a volatile environment when you are conducting hostilities but you do not have boots on the ground to confirm what the machine wants to target.”

Another major issue is accuracy. Specialists working on AI and integration into the military kill chain have estimated reliability at between 25 and 50 percent, Van Rooijen explained.

“And that is the best-case scenario,” she added. “The analogy is carpet bombing.”

Stop Killer Robots have also sounded the alarm that autonomous weapons systems “drastically” lower the threshold for going to war as it does not require deploying a military force.

“No one wants to see their soldiers die on foreign land, so that calls for some restraint. You think twice about sending your own people,” Van Rooijen.

“If it is all automated, you can just send machines. Israel and the US have been threatening Iran for decades, but never actually went through with it, and I do not think it is a coincidence… it gives you a false sense that you can do it quickly… and come back a winner.

“But it is false promises of being more accurate, more precise, yet the results are worse, not for their militaries but for civilians on the ground. They pay the brunt of it.”

No guardrail

Currently, there is no common, agreed upon definition of what an autonomous system is.

There is no treaty that regulates how they are being developed, tested, sold or deployed, setting up the conditions for a “free for all” arms race for tech giants to make a fortune “out of the misery of people”, Van Roijen said.

She finds the absence of protections particularly difficult to reconcile.

‘An autonomous weapon system is a machine and a machine cannot truly understand the value of a human life’

– Bonnie Docherty, senior arms advisor at HRW

“There are more guardrails in place if you test a new airplane, or a new car, than integrating AI into the kill chain,” she noted.

Looking back in history, there is a precedent for stopping a weapon before governments can no longer imagine warfare without it.

Among those campaigning for a treaty governing autonomous weapons systems is senior arms advisor at Human Rights Watch (HRW), Bonnie Docherty, who had a central role in the negotiations that eventually led to the 2008 Convention on Cluster Munitions, which prohibited their use.

She told Middle East Eye that the fight over autonomous weapons systems follows the same principles of humanitarian disarmament, striving to prevent arms-inflicted human sufferings.

For Docherty, boiling down individuals to numbers, and “delegating life and death decisions” to machines, is a “great concern”.

“An autonomous weapon system is a machine and a machine cannot truly understand the value of a human life,” she explained.

“To give that machine the power to kill… is ethically inappropriate. Whoever is taking a life should be able to understand the significance of it.”

Deja vu

Evidence gathered by HRW additionally shows that systems can reproduce or amplify algorithm biases, which can be lethal.

“We looked at the discrimination front and particularly at race, gender and disability,” she said. “For example, on the disability front, artificial intelligence may have trouble distinguishing a cane from a gun.”

Israel using AI weapons system co-produced with Indian firm in war on Gaza
Read More »

Yet states like the US, Israel, Russia and China are reluctant to surrender weapons that could give them an advantage, a hackneyed argument that Docherty knows too well. Behind them is also a tech industry peddling its own commercial incentives.

She has heard it before, in the negotiations over the cluster munitions treaty. Governments were greedy to retain technologies they were developing, arguing that existing international humanitarian law is sufficient.

What broke the camel’s back at the time, as Doherty put it, was Israel’s blanket deployment of cluster munition strikes in southern Lebanon in 2006. Many submunitions remain scattered across the territory to this day, resulting in hundreds of deaths and injuries.

“I feel like I am having a deja vu,” Docherty said. “A lot of these arguments, you could just plug in the same weapons system.”

Last week, the United Nations secretary-general and the president of the International Committee of the Red Cross (ICRC) renewed their call for a treaty governing autonomous weapons systems.

Three years earlier, they had urged states to establish rules by 2026. Now that the deadline has become “untenable”, they warned that the world is fast approaching a “moral red line”.

The renewed appeal also comes as the ICRC suspects that increasingly sophisticated forms of autonomy are being integrated into weapons systems across a range of conflicts, not only into defensive but also offensive operations, including in populated areas.

Legal adviser with the ICRC, Georgia Hinds, told Middle East Eye: “While it is very hard for us to verify… we are seeing this increasing integration of autonomy into weapons in many conflicts, from sophisticated military capabilities down to more commercial off-the-shelf systems that are being adapted.”

‘A big risk’

What the organisation is insistent about is that there is still hope in the form of a diplomatic route to negotiate the rules around the systems.

In November, states will gather in Geneva for the Seventh Review Conference of the Convention on Certain Conventional Weapons (CCW), which takes place every five years.

‘It is also likely to be used by non-state actors who fall outside of those more regulated investigation mechanisms’

– Georgia Hinds, legal adviser at the ICRC

It could be a watershed moment for the campaign as the Group of Governmental Experts on Lethal Autonomous Weapons Systems, the primary UN intergovernmental forum addressing emerging technologies related to autonomous weapons, has been developing a draft text, which includes prohibitions on some systems and restrictions on others.

The final session of the group takes place this week, and its work will then feed into the upcoming conference.

Campaigners are hoping that it could be turned into formal negotiations for a legally binding treaty – at last.

“It really would not take much now to convert that into a binding instrument,” Hinds said. “In November, the decision needs to be taken to move to formal negotiations.

Rights groups call for a halt of AI tech use in the military
Read More »

“Rolling over discussions that have now been ongoing for nearly 13 years in the group of governmental experts is not acceptable. There is really only one option in November for us.”

If states do agree, a treaty could be only months away. The preferred – and shortest – route would be to masterfully negotiate an additional protocol under the CCW.

However, experts like Hinds are doubtful as to how realistic this is. She said that the conference does not offer a simple yes-or-no ending, and that there are several ways to progress with the campaign.

The overall goal is to get ahead of the technology, she added, warning that as autonomous weapons spread, it becomes easier for anyone – even armed groups outside any legal oversight – to turn cheap commercial tech into deadly autonomous weapons.

“The [more] opportunity for these systems to proliferate, the easier it becomes to adapt a commercially available system… it is also likely to be used by non-state actors who fall outside of those more regulated investigation mechanisms,” Hinds said.

“All of these things again increase accountability concerns for me. So yes, I think it’s a big risk.”

Middle East Eye delivers independent and unrivalled coverage and analysis of the Middle East, North Africa and beyond. To learn more about republishing this content and the associated fees, please fill out this form. More about MEE can be found here.

Source: www.middleeasteye.net