On October 8, the AI in Geopolitics Network held its inaugural meeting at the Tufts University Fletcher School in Medford, Massachusetts. The network was formed by researchers at MIT’s Future Tech project, the University of Illinois I-School, and the Fletcher School and includes IGP as a member.
The purpose of AIGEN is to bring together policy makers, scholars and technologists in an interdisciplinary space to explore the geopolitical implications of a globally distributed digital computing infrastructure and artificial intelligence applications. I profited immensely from the mix of policy practitioners, academics, business, political economy and cybersecurity expertise.
Invited to speak on the opening panel on “AI, AGI, and the evolving landscape of power,” my comments, below, focused on how the public dialogue about governance of information technology is distorted by the myth of a superintelligence that can wipe out humanity.
AI, AGI, and the evolving landscape of power
It is hard to overstate the damage done to AI governance by the story, the myth, of an artificial superintelligence that can kill us all. In a paper published a year ago, I argued that AGI is an empty construct, and that machine intelligence, no matter how “super,” does not give it an autonomous will, much less the power to destroy humanity. Others have published similar refutations. Yet the myth persists.
Rejecting the AGI myth doesn’t mean, of course, that humans can’t destroy humanity. A nuclear weapon escalation cycle probably could do that. The difference is that we all grasp that it’s humans who are responsible for decisions about how nukes are used and the likely effects. The AGI myth, however, tells us to fear an autonomous technology, not our decisions about how to use it.
The tech’s evolution is supposed to be dangerous because of Recursive Self-Improvement, or RSI. The doomers claim that an information system can RSI its way to an autonomous will and overwhelmingly powerful capabilities. But RSI only works if the AI is given a well-defined goal that the machine learning process can hone in on. The AGI myth, in contrast, says that RSI can push the machine into any and all objectives and indeed allow it to define its own objectives. This to my mind is a pretty elementary logical error. It also ignores the constraints of physics. Superintelligence is the information systems equivalent of a perpetual motion machine in thermodynamics.
In this talk, I want to focus on a more practical topic: how the public dialogue about governance of information technology is distorted by the AGI threat, despite its mythical nature.
A rational policy dialogue would see AI as an extension of computing and the internet. It would see continuity and evolution, not revolution. AI is nothing but a souped up software application living off a distributed, ubiquitous and powerful computational infrastructure. Yes, its computing capabilities and feedback loops can produce unintended effects and disrupt established equilibria in industry and other social activities. But, lest we forget, so did the internet and social media. Almost all of the policy problems attributed to AI are extensions of problems attributed earlier to internet connectivity, web sites, search engines, and social media. All the fears attributed to hacking by “rogue agents” are just a continuation of cybersecurity dialogues about software vulnerabilities, viruses and worms that go back to the release of the Morris Worm in 1988. In a rational policy dialogue, we would analyze the problems or harms caused by specific ML applications, and focus on who’s going to be liable for the damage. Such a dialogue would encompass both misaligned models and their use by misaligned humans. Much of AI governance would boil down to adaptations of existing ICT laws and contracts regarding product liability, copyright infringement, data protection, cybersecurity, advertising, content regulation, and so on.
If the threat is existential, on the other hand, there is no time for social adaptation. The apocalypse can only be avoided by halting all development, slowing it down, or imposing ex ante review of frontier models. Here again the specter of a Superintelligence lurks. Frontier means “closer to Superintelligence,” or “on the edge of explosive Recursive Self-Improvement.” Beyond the frontier lies the Wild Indians of uncontrolled machine intelligence. The further you go past the frontier the greater the danger. Don’t we need a gatekeeper?
Here are 4 ways in which AGI/SI sets policy on the wrong track:
- It diverts attention from human agency, human decisions, human responsibility. It’s the machine itself we have to worry about.
- It cuts us off from learning and precedent in digital governance – from seeing the continuity and lessons of Internet and cybersecurity governance.
- It flattens a heterogeneous set of policy issues into a single thing.
- It creates an arms race; it securitizes or militarizes digital governance.
On #1, Let’s talk about the Hugging Face incident, and how AGI Fears have distorted our response to it. Let’s compare Hugging Face with the CrowdStrike update that shut down or crashed approximately 8.5 million Microsoft Windows computers worldwide? Which one was more costly and damaging? CrowdStrike by a mile! Was that an AI governance incident – a case of “rogue agents”? No one seems to think so, but it was an automated update program that did things no one told it to do. Delta Airlines charges CrowdStrike with gross negligence, CrowdStrike appeals to contract-based liability limits. May the best lawyers win. Whoever wins, here we have a rational exploration of human (corporate) responsibility, not blathering about AI agents taking over the world.
On #2, If we are trying to make policy regarding self-driving cars, should we be worried about whether they are going to go berserk and kill us all, or should we be focused on the specific risks posed by cars running into each other or hitting pedestrians? If we are focused on existential risk, how can we effectively apply and modify established concepts of product liability, traffic management, and transport infrastructure?
On #3, AI can be used for medical diagnosis, facial recognition, sorting through job applications, running cars, guiding missiles, spell checking, image generation, writing student essays, and thousands of other things. Don’t these applications all pose very different public policy issues? Does it make any sense to talk about “regulating” or “not regulating” some generality called “AI?” And yet superintelligence fears collapse all these things into one bucket.
On #4, the AGI myth tell us that there is a race on, that AGI is the finish line, and that the winner gets hierarchical power over everyone else. In this view, winning an AI race is just like winning the race to build an atomic bomb. Indeed, the singularity of the Singularity, the idea that RSI produces a single Superintelligence with total control and no competitors, is one of the silliest yet least challenged aspects of the AGI story. The Big Bang-like emergence of AGI, however, is a dead giveaway to the essentially religious mindset from which these stories emerge. AGI is not about computer science, it’s a cosmological origin story for nerds.
The problem with the “AI race” narrative is that it makes a globalized digital ecosystem an instrument in a nationalistic arms race. The leading American AI firms (Anthropic, OpenAI, Musk) advocate export controls on China and a nationally fenced off AI industry. But it’s not just the model developers; it’s chips and data and everything else digital on the line here. If AI is a chess piece in a geopolitical power race, the whole Internet, all applications running on it, all digital devices, all the data they produce, must be subject to national security controls, because all of these things contribute to AI development. We’ve seen this taken to a ridiculous degree by the Trump and Biden administrations. Look at the FCC’s ban on foreign home routers. The intel community supported the TikTok ban because they thought China could use its user data to feed AI applications that could control our minds. Dozens of other restrictive actions going back 10 years and spanning Presidential admins from both parties, have been based on the premise that any Chinese ICT equipment, services, even investment capital are threats to our security.
Purveyors of the AI arms race narrative face a big problem, however. Most of the world’s production and consumption of digital information systems are embedded in a globalized market economy. That transnational economy thrives precisely because its activities are less inhibited by national borders and the friction, taxes, transaction costs, and military risk that goes with them. Those non-jurisdictional spaces evolved their own governance institutions, which are mostly in the hands of nongovernmental forms of collective action. These governance institutions are somewhat insulated from military conflict and political coercion by states, and are more responsive to the global public.
The three basic policy choices we need to make are
- Whether we see AI gov as a continuation of computing and ICT policy
- Whether ICT is governed primarily by civilian or military logic
- Whether we try to cram the global Internet back into a territorial jurisdictional legal framework, or continue to reap the economic growth, improved access to information and wider distribution of digital tools associated with its relative freedom.
The post AI in Geopolitics Network appeared first on Internet Governance Project.
