Babbage | How AI is really being used in war
98 claims analyzed
Transcript(98 claims)
The Economist.
It's not uncommon to hear fighter jets on maneuvers in the skies above the Baltic Sea in the Northeast Of Europe. But last summer, unbeknownst to anyone who might have heard the roar overhead, a pilot handed over the reins of their aircraft to an AI model. The artificial intelligence system known as Centaur took part in an air to air combat scenario that pitted its airplane against another human piloted fighter jet. For the first time, an AI piloted plane executed evasive and offensive movements, all while its enemy was beyond the typical visual range of human pilots. That's usually some tens of kilometers.
The AI agent even suggested when it might fire weapons in the exercise. Centaur was developed by Helsing, a European defense firm.
We developed Centaur with reinforcement learning.
James Lawson is one of Helsing's directors.
We are able to train the AI in parallel thousands of times faster than real time.
Centaur was trained in a flight simulator. The model was rewarded whenever it did a good job.
In a couple of days, Centaur is able to do fifty years of full time piloting like a human, but without any of that guidance. It just learnt it through trial and error.
Each bit of feedback helped to tweak and refine its underlying neural network.
And so it starts off not very intelligent and then very quickly gets to human beating performance levels. Retired pilots from the RAF will consistent confirm both that it has some novel behaviors and that it has learned the same tactics that they were taught.
It's not clear who would win in a real combat situation, human or machine, especially if the AI pilot kept generating entirely new tactics.
The vision is towards greater autonomy and where possible, not necessarily needing a human pilot in future. If you're trying to complete a deep strike mission where you know there's going to be a large amount of anti air capability and a high risk of being shot down, we would much rather send a autonomous platform rather than an aircraft with a pilot.
From finding targets to command and control, AI holds a lot of promise for those on the front lines of battle.
The use of AI is very focused on the problem of how do you sense, how do you find the enemy, how do you make a decision about what to do, and then how do you deliver that effect. And AI can be used to transform every single stage of that process.
In this episode, we'll take you through how soldiers are testing out artificial intelligence in the heat of battle today and what they hope to do with it in the near future. And this is not just a story about the frontier of technology. There are also many questions to be answered about how much control humans should maintain over the future of warfare. I'm Alok Jhaar, and this is Babbage from The Economist. Today, the reality of AI in war.
With me on today's show is Shashank Joshi, the economist defense editor. Hi, Shashank.
Hey, Alok. How are you doing?
I'm good. Thank you. Now, Shashank, you've been covering the war in Iran for the paper. And in addition to the usual stories that we're all hearing at the time, know, about the airstrikes and the casualties and obviously the political and economic consequences of all that fighting. It feels like technology has really become a central topic of discussion too.
You and I have spoken a lot in recent years about the various ways that obviously war shapes technology but also how war itself is shaped by the technology that's been developed. So things like drones, satellite internet, these things have really transformed what's going on in Ukraine for example But it feels like this current war in Iran has focused a lot of attention on artificial intelligence. Maybe it's just because we talk about artificial intelligence a lot, but I'm just wondering, did you expect that to happen?
I think we saw even before this war began, Alok, in the weeks before, there was that extraordinary standoff, that row between Anthropic, which is one of the world's biggest AI companies that makes a model many people might have used called Claude, and the Department of Defense or Department of War, if you prefer the current terminology, the Pentagon, over how that model could or could not be used by the Pentagon. And I think that that thrust AI in particular into the forefront of these military discussions around planning, targeting in quite an opaque way because we didn't know exactly what it was doing. And it wasn't just a hypothetical discussion, you know, the kind maybe we've had of how AI could be used in war. It was a very real discussion because Claude, specifically Anthropic's product, had been used in the raid to capture Nicolas Maduro, the dictator of Venezuela, at the beginning of this year. And maybe that played a role in precipitating this row between Anthropic and DOD.
And so when the war in Iran began, I think it was at the forefront of our minds. And we know that Israel had used quite advanced software, including what we would describe as AI in its targeting apparatus, in its intelligence apparatus, certainly, in Gaza in previous years. So I think there's no surprise that, like with any big war, they become, I guess you could call them showcases or proving grounds for the most cutting edge technologies available, and AI is definitely in that list.
Yeah. Military has always been a use case for the most cutting edge technologies, but it feels like they've come to the fore in many, many situations. And, you know the particular areas I'm thinking about for our conversation now about where AI is most discussed. One is targeting how targets for the airstrikes have been decided and then actually the second part is how AI might be deployed within weapon systems themselves so things like drones or other instruments. So let's go through those one by one.
Let's start with targeting. Take me back to the sort of pre AI era How do targets usually get chosen in a battle?
So this is a process that involves lots of humans. If you think about an organization like CENTCOM Central Command, that's the Pentagon branch that's been leading the war on Iran, You would have the j two, which is what militaries call the directorate for intelligence, and they would basically look at all the intelligence they have, you know, intercept, satellite images, human intelligence reports, all kinds of things. And they would come up with a database of targets based on that information. They would also draw up things that you're not supposed to hit. So no strike list.
That could be saying, look. There are schools here. There are hospitals here. And then you have weaponiers who decide what ordinance is required to destroy a particular sort of target, you know, saying that's hardened, that needs a bunker buster, that's a very light target, that could do with a pretty small munition. And you've got then a j five directorate who assemble all of this into some kind of operations plan saying, you know, when do we attack this target?
And then they pass those plans to something called the j three, which is the directorate in charge of operations. Now, all of that, Alok, has always involved a degree of software since the beginning of the computer age. But the quality and sophistication of that software is changing all the time and has now become really quite formidable.
So the target selection databases that you just described, going through these different commands, coming up with a list of targets that should be attacked with certain weapons. You know, they might be using software, but generally speaking, these are human led activities. Right?
They are human led activities, but we're seeing more and more aspects of that delegated to computers. And can I give you one example? I mean, a couple of years ago, I spoke to Tamir Hayman, who was a general who led Israeli military intelligence. And he said there had been a couple of big breakthroughs. And one of them, I think he called it the fundamental leap, probably ten, eleven years ago, was in speech to text software, which is a kind of AI, isn't it?
That would allow voice intercepts to be searched for keywords. And if you saw a Hamas unit discussing something, you had software that was automatically transcribing that, maybe automatically translating that, and that can obviously feed directly into the production of targets. So, it is human led, but with more and more of it aided by computers.
Yes. That's an interesting example actually. It's ten years ago. This is just when that kind of machine learning was getting good at very specific narrow tasks. So things like transcription, things like machine translation these became very good in the late 2010s and it was just before the generative era.
Now we are in the generative era right so this is what we're thinking about now in terms of LLMs and other things that can connect much much more information, visual data as well, all sorts of different types of telemetry. So just talk to me about how all of that has been sort of filtering into the decision making that the soldiers are doing these days.
So to answer that, Ark, it's worth telling you a little bit about something called the Maven Smart System. And that is, there's no easy word to describe what it is. The phrase you hear from military people is it's a decision support system, a DSS. It's a tool, and it does everything from targeting to command and control to assessing the impact of strikes. And it's software that is unifying lots of that.
It's built mostly by Palantir, a company that has always worked with America's intelligence agencies and military for many, many years. And it's now the system that is used not just by America's military, including CENTCOM in Iran, but also by NATO as an allied organization of 32 countries. And Maven is pulling together satellite images and social media and voice intercepts and all kinds of things to put it together and say, hey. I've got a snippet of an image of an Iranian missile on social media. I can synchronize that with my fancy spy satellite that can pick up the electronic emissions of a radar or of a radio system, and I can fuse these to say, I think that missile is in this town over here, and I can feed that to then the targeting apparatus of the military.
So far, Alok, that is not really generative AI as we would understand it as consumers of ChatGPT and Claude and Gemini. Right? That's AI, but it's not generative AI in the sense that you and I understand it.
It's a pattern matching, all of that recognition.
Yeah, it's pattern matching. It's kind of classic machine learning, I suppose. But there has been an integration of modern LLMs, including Claude because Anthropic was the first company in America to really be authorized for work on classified systems into the Maven Smart System.
So tell me about that. So how do the modern LLMs fit into what you've just described? I mean, I can imagine that you could have used an LLM as an interface to something like this to search all that data, but I guess there are more sophisticated ways of using them.
I think this is the crux of it. And I'm not gonna lie. I don't have great visibility into exactly what's happening in the guts of these systems. You know, this is a work of journalists trying to find little shafts of light where we can. But my understanding from talking to lots of people is that the use of LLMs in that enterprise has been overhyped.
I think, you know, people have this imagination that Claude is being told, go plan the operation to snatch Maduro. Make no mistakes. Get it right. That's the joke. Right?
Or is
Would you like the deputy as well? You know?
Here's lots of pictures. Which one contains the Iranian missile launcher? And, actually, the fact is, Alok, there are specialist models that are much better at doing those things. America has worked on them in things like Project Maven for years and years. They are trained on classified specialist data in a way that LLMs that we use day to day are not.
And by the way, I was also told by people familiar with this situation that, look, a model like Claude or another LLM, they are not actually that good at working with geospatial data. Right? Coordinates, the relationship of things in space in a way that actual specialized models are. What the LLM is typically doing is overseeing lots of these other models. It's orchestrating the work of these other models.
It is sifting through lots of the data and offering summaries to the humans who have to still sit at the other end, ultimately making the decision, saying, how many instances of Iranian missile launches do we see in our dataset across these various sorts of data? So it's not, you know, looking at the satellite image at the forest in Tehran. It is doing something much more higher order and in some ways quite different than that direct raw military task people may be imagining.
Can I ask about the way that these systems operate now, the classified systems? They seem to be accelerating the number of targets you can categorize quickly at the beginning of attack. The, the war in Iran, the number of targets attacked in the first few days was astronomical compared to previous wars. So they're clearly making decisions faster, but they also seem to be making mistakes. I mean, was that well documented incident recently, of course, about the attack on a school in Iran.
How do mistakes like that still get through given that these targeting systems are supposed to be more accurate, faster, all of that?
So there is no evidence right now that that incident you describe, which is this American missile landing on a girl's school in Minab, which was near a revolutionary guards base, but separate from it, was down to AI. We don't know whether it was a human who found that target, whether it was orthodox software, whether it was something more wizzy that went wrong. But what we do know is that what often happens is when you have these targets, you store them in something that we could call a target bank. The Israelis did this in part because they wanted to be able to strike something very quickly once a war began. You want to know all the locations of your potential enemy headquarters and missile launchers and all the other things.
And what you should be doing is revalidating those targets at some point to say, if you put them into your target bank two years ago, actually, is that still a weapons dump or is it now something else? It really interestingly, in our story that I reported with our colleague, Anshel, who's in Israel, Anshel told me about situations where when civilians were killed in strikes, the Israel Defense Forces would go back and check their information. And often it was because Hamas had used that building previously, but it moved on and families had moved in. And the IDF did revalidate these targets, but it wasn't often enough. And that is exactly what seems to have happened in Minab, which is that this was a target.
It then became a girls school, very visibly so, you know, blue bright painted walls and people visible in the playgrounds, all of that, and it was not revalidated. The key point, Alec, I think you alluded to this, is not that this is the responsibility of AI. Indeed, you could argue AI could help you revalidate those more frequently. The problem is if you are using AI as an industrial scale machine to produce an order of magnitude more targets than you did in the past, does that potentially outstrip your ability to then check and scrutinize all those targets? Are you more likely to make those mistakes?
I think that's an interesting question to consider.
Well, let's consider it because, obviously, the number of targets that have been identified at the beginning of this war, as I said, far outstrip anything that's seen before and they can't all be checked manually. Do you see evidence of commanders allowing the software to become more autonomous in these decisions in some cases to relieve that pressure on what I assume is fewer members of military staff in the first place, given the cuts and things that are going on in the DOD?
I think it's quite interesting because the appeal of AI enabled decision support systems like Maven is to act faster than the adversary, is to say, we will spot your missile launches and take them out before you can move them, before you can do anything. In this conflict, because America and Israel have such massive advantage over Iran, I'm not sure it's about speed. Right? It's more about volume. It's more about scale.
And I don't know how much responsibility commanders have ceded to the machine, how much critical scrutiny they are applying. I'm not inside the headquarters. I can't say that. But what I can say is having spoken to military officers who are familiar with MAVEN in the NATO context, they are concerned that this tool is going to put pressure on commanders to cede more responsibility to the system, that they might still approve the strikes, but they might not know what they are approving. That's not real human control.
Whereas if you had a really thoughtful human overseeing the strikes who could intervene, but was just ready to press a button to stop it, you could actually have a situation of greater oversight if you had that rather than the man just pressing a button repeatedly. And I think, Anok, the other interesting thing is whether the AI will then have certain biases or tendencies that filter into our human decision making. So there's a recent paper that I thought was fascinating. It tested leading chatbots in war game simulations. What And it found was that LLMs opted to use nuclear weapons in simulated war games in ninety five percent of cases.
So you could find that the way these things think begins to overwhelm or infect our human modes of thought.
Yeah. Well, I guess chatbots don't have homes and families to think about when it comes to nuclear war. For them, it's just another button to press. Right? That's the difference.
This is why we should think very carefully about whether or not to put those sorts of things in charge of very advanced weapons. Talking about autonomous weapon systems, I mean we're very far away from nuclear weapons here. The devices that we often talk about and we've talked about a lot in recent years in warfare is drones. How are drones being used in Iran at the moment?
Well, we're seeing massive drone use. And actually, you're absolutely right. This is the opposite end of the spectrum because 99% of this has nothing to do with AI or autonomy. These are drones that are flying a long way, a thousand kilometers in many cases, either targeting things inside The Gulf States or Israel or elsewhere like Cyprus, either by homing in on a certain point with GPS or being controlled remotely by somebody. They're doing nothing fancy from a software perspective, really.
But the advantage they have is that while they are much slower than a ballistic missile, they're also much cheaper than a ballistic missile. You know, maybe 50 to $100,000 for a Shaheed rather than a couple of million for a ballistic missile. And so what they give you is sheer volume. You can swap defenses, and you can force adversaries to use up more expensive interceptor missiles to defend against them and just sort of overwhelm your opponent by raw numbers. And I guess that is really a triumph not of software or AI or technology.
It's a triumph of quite basic engineering.
Of course, we always talk about Ukraine as being the cutting edge when it comes to drones and the technology folk in Ukraine really pushing the boundaries of what's possible there. To what extent are drones using any autonomous software or becoming autonomous in places like Ukraine?
They are. You know, Al, you and I have talked about this before. The problem really is if you have a drone that's trying to go, let's say, fly into a tank, as you know, there is lots of jamming that takes place to try to cut the signal between that drone and its pilot so that it can just fall to the ground or fly off. To overcome that, you either need a fiber optic cable to send that signal to the drone so you can control it all the way to the target, or you need some other method. And on a minority of drones, but a growing number, you have autonomous terminal guidance, which from a couple of kilometers out and perhaps even more now can say, you've got me close enough.
The jamming is about to kick in. I'm about to lose my control to the pilot, but there's a chip onboard the drone that can pick out the tank and keep it in vision with a little camera and home in on it even if the signal is completely cut. And, really, that last mile autonomy has advanced enormously during the course of the war in Ukraine to the point where Ukraine is now telling other countries, we have the best military data in the world of images, of labeled datasets. We can sell these to you, and they can benefit your military software. And I think Ukraine has a massive asset there.
So we've heard about how AI's ability to distill information and extract insights is already transforming military command and control. And what you've just told us is that the technology is starting to be used in a different and so far more limited way at the other end of the scale too on on the front lines, which is how Ukraine's now in a position to offer to share its knowledge with armies that are operating in The Gulf. Shashank will come back to you in just a moment. First though, let's hear from one of the companies building machines for the battlefields of Ukraine.
So Helsink has a a permanent presence in Ukraine.
That's James Lawson, one of the directors of Helsing, the firm you heard from at the start of the show. They've been building AI agents to pilot those Swedish fighter planes.
On the second anniversary of the conflict in discussion with the Ukrainians, they specifically asked that we start to put our AI capabilities onboard drones. Usage of AI in Ukraine is still in its early stages, but what we are seeing is it can make a massive difference. So we use AI in a in a number of different ways. For example, we'll use AI to help the users more quickly identify enemies, particularly through a surveillance drone, but also increasingly using satellite imagery as well, effectively taking photos off the ground, but also using things like synthetic aperture radar. But it takes a very long time to look through all of this.
So AI is able to compress that search down by a factor of at least 60 fold, we've seen, but it could be even faster as these systems improve. And then once you've found the target and you want to navigate towards that target, we use AI onboard the drone to help it navigate. So it uses satellite imagery, and it has cameras, and it's able to effectively map read to look at the terrain around it and work out where it is and get from a to b without relying on traditional GPS signals anymore.
That's important because Russia's forces have been able to jam GPS signals, blocking an operator's ability to locate drones. And this cat and mouse game of drone innovation against jamming countermeasures is something you'll have heard a lot about on Babbage in recent years. Now AI has also entered the conversation.
The Ukrainian countermeasures of that was first person view drones to have human pilots manually pilot these drones from a to b to spot targets and to strike at targets. The Russian countermeasures of that is to then use electronic warfare to jam the radio frequencies and therefore, again, stop the drones from being effective. So good software and AI helps overcome those challenges because you are no longer reliant upon GPS. You are no longer reliant on those radio frequencies all of the time, but you can navigate autonomously from your launch site to the target. And then when you get to the target, you're able to do that last mile without the fear of being jammed.
So this is our HX2, which is a strike drone.
At Helsink's London office, James is showing us around a roughly one meter long black drone.
The h x two is manufactured to be able to deliver a strike on armored targets and electronic communications equipment, anti air capabilities, radars, and other adversary equipment, but to do so at the lowest cost possible with the greatest precision. So it communicates with a radio on board. Over that radio, it feeds back what it can see through these two sensors at the front, both for daytime and for night vision. And then on board at the front here, just behind the sensors in the nose, we have our payload. It could be up to four kilograms of explosive.
This drone isn't like the quadcopters you've probably seen flying around in your local park, nor is it like the Iranian Shahad drones, which look like small airplanes. Instead, the h x two has a slim fuselage. It's dominated by x shaped wings in the middle, and it has smaller x shaped fins at the back, which hold the four rotors.
The h x two is electric powered. So at the back, we have a battery, and that battery basically enables it to go as far as a 100 kilometers. It's catapult launched to be able to get them into the air very, very quickly and easily.
The drone feels fragile, but that's by design. It's made of very low cost and lightweight materials.
The total weight is around 12 kilograms. Its top speed is just over 200 kilometers an hour. The key thing is that it's a x wing design, and visually, this is somewhat menacing, but it's actually a deliberate design decision to help the aircraft be more precise. So with a fixed wing design, when you just have two wings, you are able to get greater range because you have less drag. But it is harder to control the aircraft.
It's less precise, and that's why these x wings have become more popular.
The wingspan of the drone is around a meter, and the four parts of those x shaped wings can be neatly clipped back onto the fuselage. That means the drones can be packed away into boxes and easily transported to the often hidden locations where they need to be launched from. They're essentially flat pack drones.
Within our leadership, we have expertise from companies like Tesla and IKEA because for this new era of warfare, you need new modern manufacturing techniques, and you need to be able to produce these capabilities at scale if we are to deter our adversaries and make Europe safer.
How do these drones actually use AI, and to what extent are they being deployed autonomously?
So it's able to navigate from the launch site to the target autonomously without the reliance on GPS. And then when it gets to the target with a human designated target, so a human makes that crucial decision of what to target. But once that designation has been made to robustly strike the target even when there is jamming, At that point, the drone enters its autonomous terminal guidance, and the operator can take their hands off the controllers, and it will very precisely and very accurately glide into the target.
It's hard to know how well these drones actually work in war, but my colleagues here at The Economist have found that so far, drones with terminal guidance are proving to be reliable and successful. That's particularly important in Ukraine where there's a shortage of skilled drone pilots. But as Shashank mentioned earlier, only the minority of drones are being used in this way so far. That hasn't stopped engineers thinking about the next stage of the drone arms race, though.
Where it goes next is towards other capabilities that make it more robust and more lethal. In particular, we're investing very heavily into swarming. So not just having one operator to two or three drones, but potentially having one operator able to control ten, twenty, 30 drones simultaneously. And that means that they're able to take on more targets.
That might sound somewhat apocalyptic, but James reckons that European armies need to develop autonomous capabilities, especially given the modern age of unpredictable and very tense geopolitics.
Greater autonomy provides a more effective deterrence to Russia to dissuade them from further expansion of the war in Ukraine and on NATO. I would also add that AI makes these platforms more precise, and by being more precise, we're able to better discriminate where we use force. And that means not only are we more effective in striking our adversaries, but we can do so with a much reduced risk of civilian casualties of collateral damage and actually with stronger compliance with international human rights law. We do need to operate at pace. We know that Russia and China are investing in these capabilities as well, and we don't want to end up in a situation where our troops are put at a huge disadvantage.
But I think we can always do that while maintaining European values and ethics. I don't think we have to have the compromise there.
I'm back now with Shashank Joshi, The Economist Defence Editor. Shashank, James from Helsing there mentioned this idea of drone swarms as the next frontier for how AI technology is being deployed at the sharp end of warfare. Just talk me through why armed forces would want to do this.
Think about a Ukrainian drone team. Right? You've got a pilot. Yes, of course. But you've also got the technician to prepare it, make sure the batteries are working, the warhead's working.
You've got the driver who also has to protect the team. And you might also have a reconnaissance drone run by another three people to spot targets for the first drone. So you suddenly got, you know, six people to get one drone up in the air and effective. Imagine if you could flip that ratio. If one person could control lots of drones at once, that's the appeal of a swarm.
And here's the really important thing, Alok. We talk about drone swarms. You know? Oh, Iran launched a swarm of drones at Saudi Arabia. Those aren't real swarms.
Lots of drones are not swarms. A swarm is more like a murmuration of starlings. Right? It's a set of drones in which the movement of the hole is spontaneous and in which each drone is coordinating with or communicating with the other drones in that flock. So it's not just sending 10 projectiles up in the air.
It's those 10 projectiles achieving some task by cooperating with each other. You can imagine being overwhelmed by a 100 drones that would observe the environment and would say, that first drone hit the front of the tank. I'm gonna go around the side and hit the side of the turret. I'm gonna go and hit the tracks. And it would be a much more sophisticated way of applying mass than everything just going for the same spot.
That is the holy grail of drone warfare.
Okay. So how would that happen? I mean, I assume if you want a swarm of 10 or 20 or 50 drones, they have to be communicating with each other and reliably doing so whilst also monitoring their environments and being quite far away from the operators. Right? So is this a communications challenge?
So it's a coordination problem. You know, how do you get the drones to do the right thing, knowing what the other drones are doing? That can be a communications issue. There are other ways to solve it. There's a drone called the V2U, which is a Russian attack drone.
That's really interesting. Each drone has wings colored in a different color. And what might happen is, for example, the red colored drone might be assigned to attack the first target. The orange drone might be assigned to attack the second and so on. And if the first drone misses, the second one can take over, but it relies on the computer vision on each drone, tracking the drone ahead of it in, I guess it's like a queue, it's like a line.
And that's a really, I think, clever and interesting way of coordination. There are other ways of coordination. And I think probably, you know, the most promising one is like a mesh network. If you think about the WiFi mesh networks, some of our listeners may have at home, it's kind of nodes where each drone is sending communications. And if one is lost, it doesn't matter.
It can route the communication some other way. The problem is managing those meshes gets very complicated as you multiply the size of the swarm. And, you know, in theory, these are difficult to jam because you've got a signal being sent over quite a short range drone to drone, but no one really seems to have mastered it just yet on a very, very big scale.
We might be technically some way away, but it feels like you can already see some of these sort of moral, ethical, legal implications of this because, of course, this requires a huge level of automation. There's no way one human being could direct every single one of those drones in the sort of time required to attack a target so I wonder you know if you give lots of drones autonomy they can communicate with each other they control they attack the tank in the way that you suggest these 50 drones That really requires soldiers and commanders to kind of cede authority to technology. This is where the AIs are taking over in terms of decision making. Right?
I think it all depends on the context. If you know there is a tank over the hill, a swarm may be a very sophisticated and effective way to destroy it. And you may have ceded control over that swarm in terms of which tank it attacks first in the group. But actually, you know what you're hitting. You have a pretty clear sense of what you're striking.
The swarm is not going to wander off to Moscow and say, I'm gonna find my way to the Kremlin. That's just not what's gonna happen. I think what matters is not really how much autonomy a weapon has in terms of tactical autonomy. Because in that sense, a naval mine going back to the First World War is highly autonomous because you don't know what ship is going to come along and blow up. Autonomy for me, Alok, I think it helps that we think about the autonomy of a weapon in time and space.
Is that weapon to which you have ceded a degree of tactical freedom? Is it allowed to robe free over a massive area where you don't know what will turn up? Maybe a school bus, maybe a Russian tank. Is it being given lots of autonomy to loiter in an area long after you, the commander, have left that area? So you knew there was a tank there, but you've wandered off and maybe civilian convoys of refugees have come through.
The more freedom a weapon has in time and in space, the more true autonomy it has, and the greater the burden on a military commander to understand how the weapon they are using is likely to behave in that novel situation the weapon may face. And therefore, I think we either have to have a really good understanding of how the weapon will work or incredibly sophisticated weapons that can make these contextual judgments, including discriminating between civilians and competence. And right now, I think we're still some way away from that, which is exactly why companies like Anthropic are saying, we don't think modern LLMs are fit for purpose for this sort of task just yet.
So what concerns me about this is that this is gonna require people to sort of step back, not necessarily use the most cutting edge equipment and ideas and technology whilst they think about the sort of human interaction element of all of this. But you can imagine defensive departments around the world wanting to develop something like this as quickly as possible to gain comparative advantage. Right? So is there going be a race to the bottom here or is there any hope that people would sort of come together a bit and put some rules down about how to use these technologies in warfare?
I think there's a kind of interesting chasm I've written about between ethicists and lawyers who are concerned about autonomy in weapons and the military who, particularly in Ukraine, say this stuff is being deployed. The ship has sailed. It's too late.
We've got no time to think about this.
And there's no time to think about this. And our adversaries will rush ahead. But I also think that you've still got a profound intellectual debate. There are lots of military officers who will say, AI is going to be better at making decisions under pressure than humans. I mean, I remember someone saying, imagine an area you're striking with shells, which we've done, you know, for over one hundred and fifty years.
You, as a commander, could say, I want that square kilometer blanketed with shells. If a soldier in that grid square decides to surrender, hands up, I want to give up, I'm sorry. There's no surrendering. That's gone. You know, they're gonna be killed.
The shell cannot take the surrender of that soldier. And so what the military officers might say is, why then would you expect my autonomous weapon to be able to deal with that same situation? In fact, arguably, it will be in a better position to spot an individual with their hands up and having placed their gun down to say, we're not allowed to shoot that person because that would be against the laws of war. And I mentioned general Tamir Hayman from the Israeli Defense Forces earlier in this conversation. And when I spoke to him a couple of years ago, he said, we did various kinds of tests where we compared the capabilities and the achievements of the machine.
And compared to that of the human, most tests reveal the machine is far, far, far more accurate in most cases, it's no comparison. So I'm not endorsing one view or the other here, but I am highlighting that I see this intellectual gulf between militaries who see enormous potential in this technology, both for the battlefield, but also ethically, and so many others who are concerned it is outstripping the human ability to really understand what's going on inside them.
Shashank, that's been a fascinating and, slightly worrying conversation. Thank you very much for your time there.
Thank you so much for having me, Alok.
And thank you for listening. If you know someone who might enjoy this podcast but doesn't subscribe to The Economist, you can share this episode with them for free on The Economist app. Just find us on the podcast tab, tap the share button, and select give as a gift. That's all from us. Babbage is produced by Jason Hoskin with mixing and sound design by Nico Rofast.
The executive producer is Hannah Murillo. I'm Alok Jha, and in London, this is The Economist.
No claim selected
Click an underlined phrase in the transcript to see the fact-check here.