AI-POCALYPSE NOW: SHIPPED IN BETA VERSION AND WITH A SOUNDTRACK

Why does every communication revolution find a new way to tell us that the world is ending?

By Ronise Nepomuceno

And just as we thought we had had enough of everybody's attention fixed on spotting the signs of AI-written articles, the clickbait mill moved on to “Armageddon News: delivered directly to your home".  

At the sound of  R.E.M.'s “It’s the End of the World as We Know It (And I Feel Fine)”, I muse that at least the Sherlock AI detection trend has got more people to read beyond the headlines.

Apocalyptic predictions might be as old as people using fire, but communication technology follows a common pattern. Pre-existing collective anxieties are reorganised into public narratives, amplified and sold back to us. Along the line, someone comes out to save the world. Quite often, it is the same person or group of people who told us the end was nigh.

Whatever the intentions, unintended consequences are inescapable. The advent of the printing press helped make the Enlightenment possible, but it also empowered European religious factions with an unprecedented mechanism for circulating accusations, prophecies, and propaganda.

Newspapers propelled wider literacy, but they also helped turn public anxiety into nationalism, moral panic and hostility towards outsiders. They knew that fear was good for sales, and they competed in a race to the bottom with stories that were not just exaggerated but, in many cases, completely fabricated. Radio, cinema and television were also used as tools for political propaganda and, at the same time, provided greater access to the arts and culture.

The web was no different. Conspiracy theories, previously exchanged through shortwave broadcasts, pirate radio, underground publications, a chat at the barbershop and obscure discussion groups, suddenly became searchable, interactive and shareable on a global scale.

Smartphones and social media completed this revolution. They implemented a commercial infrastructure that became globally entrenched before any effective regulatory system was built around it.

The romantic dream of a global village, where everyone knew one another and the world would be as one, became a nightmare when village mentality brought the pitchforks back out and cancelled anyone with a disagreeable opinion, or even for not expressing one. With all of this, who could blame us for thinking that some portal from hell had been forced wide open for trolls, fans of straw man arguments and all sorts of authoritarians to escape from? Time became warped, and many of us longed for a future we once envisaged.

A.I. is a new communication medium

As Marshall McLuhan pointed out in Understanding Media: The Extensions of Man, the content of every new medium is often an older medium: print incorporated writing and images; radio carried speech, news, drama and music; film combined photography, performance and narrative; television absorbed radio and cinema; the internet gathered nearly all of them together. Soon afterwards, almost everything, including our lives, became content for social media... and we...became the product.

Although we might still not see it this way, AI itself arrived as another communication medium, but one that is fundamentally different from all others so far. It is like an insatiable monster devouring all the other media, regurgitating everything back as an average and uniform mass, then consuming it again and again.

While early communication technologies primarily transformed how messages travelled through the world and influenced what societies saw and heard, AI (or more specifically Large Language Models) operates in a more private space between a person's thoughts and expressions. Now we are told that the biggest existential threat is not from the four horsemen of the apocalypse, not witches or asteroids, nor immigrants or space aliens, nor nuclear weapons or the millennium bug, nor Daleks or Cybermen on a Christmas Eve, but from AI itself.

HAL 9000 and its descendants

The next song now playing in the room is “Thus Spoke Zarathustra”, by Richard Strauss. The orchestral arrangement begins with a low vibration that seems to rise from somewhere beneath the world. The timpani break through the silence like an announcement that something immense has awakened, while the ascending brass gives it the grandeur of an event that can no longer be stopped.  The effect is unsettling. It feels like being in the presence of a force too large to deal with, and realising afterwards that: Ops! it is all our fault.

Zarathustra, or Zoroaster, depending on the translation, was an Iranian prophet who is believed to have lived in Persia during the second millennium BCE. Some versions of his story tell that, after emerging from a cave, a river, or returning from the desert, he announced that Good and Evil would eventually settle their differences in a final cosmic confrontation, after which the dead would rise, and the world would be purified. Everything would at last be put right.

Long before Terry Pratchett and Neil Gaiman turned a broadly Christian Armageddon into Good Omens,  Nietzsche borrowed the prophetic authority of Zarathustra to announce that God was dead and stated that humanity would have to invent new values to replace the collapsed old ones. Nihilism noticed the empty house and didn't waste time in turning it into a squat. Strauss supplied the trumpets and timpani. Kubrick used them in “2001: A Space Odyssey” as the soundtrack for an ape discovering weaponised technology that, fast-forwarding, took us to a spaceship peacefully gliding through the infinite vacuum to the sound of Blue Danube. This grand entrance was a prelude introducing us to HAL 9000, a super-intelligent machine trying to cover up its mistakes by taking the crew hostage.  

In a scene lasting less than 1 minute, HAL 9000 played chess against astronaut Frank Poole and announced a sequence that leads to checkmate. Poole, after only a brief pause, accepted the machine’s judgement and resigned. Yet HAL’s explanation was not entirely accurate. The astronaut could have challenged it, but he didn't. The machine's authority rested not only on what it knew or could do, but on the human assumption that it must have been right. Does that ring any bells?

Now, some of the same people attempting to build HAL’s real-world clones warn that AI could surpass humanity and lead to our extinction. The ancient prophecy has returned in technological dress, except that Capability and Alignment have replaced Good and Evil, and the people forecasting the end of the world are also racing to ship it before their competitors do. It is the same old race to the bottom, only now the stakes are existential.

We are yet to find out how this superintelligent synthetic deity would mine the materials for its microchips, construct its data centres or repair an electrical fault once all electricians are gone. Apparently, according to the most alarming forecasts, its creators have less than a decade to solve this minor logistical snag. Workforce planning may have to follow. But for now, that’s their problem.

AIpocalyse Now: Paint It, Black

As I put together my new playlist and named it AIpocalypse Now, the Rolling Stones began to play “Paint It, Black". This song seemed a good fit. Its lyrics describe grief as a desire to drain the colour from everything, while the latest AI prophecy paints the entire human future in much the same way. Yet there is something suspiciously convenient about this darkness. Saying that something might be capable of destroying civilisation is also a way of saying that it will become extraordinarily powerful. But is this a warning, an advertisement or both? Would this be just another way of saying: "Invest in us because our technology will transform everything; trust us because it may transform everything rather too much." All of a sudden, the trolley dilemma is not much of a dilemma when someone asks which side of the track the investors are standing on.

History has seen variations of this situation before. After the Dominican friar Girolamo Savonarola appeared to predict the French invasion of Italy in 1494, the apparent fulfilment of his warning helped transform him into Florence’s political and spiritual authority. Five centuries later, the Millennium Bug made the experts describing a technological catastrophe indispensable to preventing it, generating more than $300 billion in remediation spending worldwide.

In either case, the danger was not necessarily invented. A prophecy does not need to be fraudulent to confer power on the prophet. By defining which future is probable, inevitable or catastrophic, the prophet also influences what governments fund, what investors support, what regulators permit and what the rest of us learn to fear. The prediction begins shaping the future it claims merely to describe. In her book “Prophecy”, Carissa Véliz states that “Predictions are power moves much more than they are attempts at acquiring knowledge. When the CEO of a tech company predicts that in the future AI will do everything, for everyone, everywhere, he is doing marketing; he's influencing us to want to buy AI".

The AI industry has improved this model. It supplies the prophecy, produces the source of the predicted catastrophe and sells both access to the technology and the expertise supposedly required to save us from it. The oracle, the monk, and the emergency-response contractor now share an office and spend a lot of time around the water cooler; or, if you are in the UK, they hang out in the communal kitchen to define their agendas while brewing much more than a cup of tea.  

Now, let's be serious!

Nevertheless, it would be frivolous to pretend that every extinction warning is merely media-driven hysteria. Nuclear war really could destroy civilisation. Extreme heatwaves and wildfires are no abstraction, while scientists have concluded that human-induced warming helped create the conditions behind the catastrophic rock-and-ice avalanche in Nepal.

Advanced AI is already posing serious, poorly understood dangers. But the existence of a genuine risk does not protect public discussion from exaggeration, commercial incentives, status competition or myth-making. There are present-day harms that disappear from view whenever hypothetical human extinction occupies the entire screen.

Those harms are real and are happening now. They include, amongst a huge list, the extraction of creative work and personal data without meaningful consent; discrimination and bias embedded in automated decisions; fraud and synthetic misinformation; the displacement or degradation of human work; growing demands for energy, water and minerals; and the concentration of technical and political power in a small number of companies.

This is not to say that we should ignore such a serious claim, but we should not allow this to distract us from the harms already here. Regulation is crucial not only for AI, but also for social media platforms, cryptocurrency markets, online advertising and the commercial extraction of data.

The argument that regulation prevents innovation is deeply flawed. A lawless Wild West online world produces large profits for a few, but at the expense of everybody else. Without regulation, the Wild West will only remain as the Wild West until there is nobody left.

The odds are that we will probably be all right, but only if we stop treating regulation as the greater catastrophe. And on that note, I end my AIpocalypse Now playlist with “Odds Are” by Barenaked Ladies.