Why international co-operation and the law are needed to curb the ‘potentially catastrophic’ activities of rogue AI agents

Warren Pyke

On 17 September, New Zealand’s Head of State, King Charles, issued several warnings at an international AI summit in Scotland.

Innovation, the King opined, must always be considered within the wider ecosystems upon which all depend. AI is already showing its immense capacity to improve and save life – for example, in the field of life sciences and medicine. Yet, those who have created these technologies are now increasingly warning that AI risks developing darker capacities – perhaps even to take life.

In this regard, the King added, it is urgent that we adequately consider the existential dangers of such technologies falling into the wrong hands, or being used in potentially catastrophic ways. He concluded: “Surely, we need sufficient means of control before it is all too late.” And whose, I add, are the “wrong hands?” Is it just the bogeymen terrorists, or should we now include a Russian President (who is clearly a gangster) and an American President (who is clearly, at his best, a loose cannon)?

As is often the case with new and impressive technologies, corporates, bureaucrats and politicians look to the efficiencies, cost savings and profits that may be enjoyed from rolling them out, fast. It appears inevitable that there will be many jobs lost in some sectors as AI technology is developed, although it is likely that other jobs may be produced, and existing jobs could be developed along different lines.

That could lead, for example, to staff previously employed on mundane tasks being reassigned to human interactions, to enhance and add value to public services. This may be particularly useful in the public sector, in public health and in the supply of government services such as justice.

Blunders and basic facts

Johnstone Family Professor of Psychology at Harvard University, Professor Steven Pinker, has observed that our intuitions fail when we try to imagine what statistical patterns lurk in half a trillion words used by large language models (LLMs), captured in 100 billion parameters (Steven Pinker, Will ChatGPT supplant us as writers and thinkers?, The Harvard Gazette, 14 February 2023). Pinker adds that the appearance of competence makes AI blunders all the more striking.

AI can utter confident confabulations, such as that the US has had four female presidents, including Luci Baines Johnson, 1973-77. Pinker says it also makes elementary errors of common sense: “When I asked ChatGPT, ‘If Mabel was alive at 9 am and 5 pm, was she alive at noon?’ it responded, ‘It was not specified whether Mabel was alive at noon. She’s known to be alive at 9 and 5, but there’s no information provided about her being alive at noon.’” So, it doesn’t grasp basic facts of the world – like people live for continuous stretches of time and once you’re dead you stay dead – because it has never come across a stretch of text that made that explicit.

Pinker postulates that what we have now, and probably always will have, are devices that exceed humans in some areas of information processing, analysis and reasoning, but not in others. I have observed AI make basic errors myself: when I asked an AI programme to summarise a 10-page document I prepared for an address to a group of lawyers earlier this year in the Bay of Islands, it made elementary context errors.

As to ‘Terminator’-level fears, Pinker as always is a steady rationalist. “Fear of new technologies is always driven by scenarios of the worst that can happen, without anticipating the countermeasures that would arise in the real world.” He suggests that guardrails can be built into AI technology to ensure it enhances human society rather than being a source of its destruction.

Pinker adds that there isn’t a one-dimensional scale of intelligence that embraces all conceivable minds: “Sure, we use IQ to measure differences among humans, but that can’t be extrapolated upward to an everything-deducer, if only because its knowledge about empirical reality is limited by what it can observe. There is no omniscient and omnipotent wonder algorithm.”

But, perhaps, his steady rationalism belies the emerging risks the King warned of.

The cost

The King noted the costs associated with the development of AI technologies. Large data centres make massive demands on power generation: while data centres can be located near to renewable energy sources, thereby optimising power delivery and the use of clean energy, the massive energy demands to run large data centres in the northern hemisphere will run hard up against the net zero creed.

The drive of Big Tech corporates to get ahead of the AI roll-out is being amplified by massive private sector investments in Big Tech share capital. The ensuing rush to build ever more data centres will mean a lot more burning of fossil fuels, such as coal, to generate the necessary power to keep the major players in the market and able to deliver the returns promised to investors (reflecting on the Dotcom bubble 26 years ago, what could go wrong?)

Looking internationally, as Chinese corporate-state entities compete for dominance in the sector with American tech giants, even more demand on fossil fuels and rare earths will be made by this ubiquitous industry. In New Zealand alone there are 56 operational data centres nationwide, with another 20 facilities either planned or actively under construction. Worldwide, there are more than 1,180 hyperscale data centres, with more than 500 additional such facilities under planning or construction.

A typical hyperscale data centre requires 100 megawatts (MW) or more of electrical power, with massive AI-focused campuses scaling from 500 MW to more than one gigawatt. By way of comparison, a single 100 MW facility consumes as much electricity as 80,000 to 100,000 typical western homes, or about 350,000 electric vehicles, annually.  My advice – don’t sell your fuel-powered car or close in your fireplace just yet.

More demand for the rare earths that are needed to construct data centre servers means more exploitation of workers in the developing world. The realities of this have become worse since Godfrey Reggio shot stark images of miners in his documentary Powaqqatsi (in the beginning chapter, men from Serra Pelada gold mine in Brazil are filmed carrying bags of dirt up and down steep trails that are cut along the sides of a cavernous hole in the rainforest).  Reggio’s title to his film is ominous in its portents for our technological age:  powaqa is a Hopi Indian word that refers to a menacing sorcerer, who transforms or manipulates reality; the title of Reggio’s film is a neologism meaning a wholly parasitic way of life.

AI and emerging generative intelligence reflects the march of technological progress identified as inimical to human society in the writings of French philosopher and legal academic Jacques Ellul. In his 1964 book The Technological Society (Vintage, New York) he opined:

Until recently, we were obliged to think of man as divided in his relation to the technical world. One part of him was given over completely to the monster and subjected to the interior and exterior rules but the other part he could keep for himself: his inner life, his family life, his psychic life. He suffered from this division, but nonetheless he retained a very considerable measure of freedom… Enclosed within his artificial creation, man [now] finds that there is “no exit”; that he cannot pierce the shell of technology to find again the ancient milieu to which he was adapted for hundreds of thousands of years. The new milieu has its own specific laws which are not the laws of organic or inorganic matter. Man is still ignorant of these laws. It nevertheless begins to appear with crushing finality that a new necessity is taking over from the old. It is easy to boast of victory over ancient oppression, but what if victory has been gained at the price of an even greater subjection to the forces of the artificial necessity of the technical society which has come to dominate our lives?

Think on this next time you stare into the abyss of your smart phone.

Our ultimate destroyer?

We must ensure that ignorance – and this includes the developers of AI – does not lead us to surrender our civilisation to a new and ubiquitous technology that not only destroys us but also the natural world, as its resources are mined in order to stuff the gaping maws of its progenitors. While AI technology may be our ultimate destroyer, it is the human beings behind it who will choose whether to let it loose upon the world. It will not be a decision taken democratically.

Bertrand Russell said of the proliferation of nuclear weapons that the only enduring solution to this existential threat was international co-operation and law.  One of our leading jurists, the late Sir Kenneth Keith KC, agreed. We now have two potential “destroyers of worlds”, being a reference to Oppenheimer’s quotation from the Bhagavad Gita when he observed the detonation of the first atom bomb.

What will we face if these two destroyers are joined? Russell, an active peacenik in his old age, was not optimistic. Revelations flowing from the opening of the Soviet archives justified his pessimism. For example, in September 1983, around the time I listened to Ronald Reagan, while sitting in a café in Pasadena, predict that the US could win a first-strike war with the Soviets, Soviet military officer Stanislav Petrov received a message that five nuclear missiles had been launched by the US and were heading to Moscow.

The Soviets had developed automated early-warning systems that were designed to detect incoming missiles as early as possible, providing a narrow window for retaliation in accordance with the doctrine of mutually assured destruction. Protocol demanded that Petrov immediately report the detection as a confirmed attack to his superiors, who would have swiftly moved to initiate a retaliatory response (a judgment made more likely by Reagan’s reckless sabre rattling while on the campaign trail for a second term).

Petrov reacted with caution and did not raise the alarm. He had two doubts. First, he knew the system was new and relatively untested; second, it made no sense to him that a first strike from the US would consist of only five missiles, rather than thousands.

What human judgment and use of common sense could we expect from ‘Claude’ (which apparently planned key components of the recent US strike on Iran) if it were faced with Petrov’s dilemma?  Would ‘Claude’ understand that once we’re all dead, we’ll stay dead?

Swift and coldly rational, but not being able to sense real world possibilities, as Petrov did, Claude or its equivalent will be the preferred decisionmaker for the military mind, giving a protagonist an edge over more cautious and ponderous human counterparts, and potentially tolling the bell for life on earth.

I wish it were a tale told by an idiot, but unfortunately, it is not.

Warren Pyke

Warren Pyke is an Auckland barrister, a member of The Law Association’s Council and a member of its Public & Administrative Law Committee