The idea that AI could lead to human extinction has been discussed on the fringes of the technology community for years … Like a magician’s sleight of hand, it draws attention away from the real issue: the societal harms that AI systems and tools are causing now, or risk causing in future. Governments and regulators in particular should not be distracted by this narrative and must act decisively to curb potential harms. And although their work should be informed by the tech industry, it should not be beholden to the tech agenda.
Nature editorial, “Stop talking about tomorrow’s AI doomsday when AI poses risks today“
1. Series introduction
Interventions aimed at reducing existential risk are usually evaluated by multiplying the probability of preventing risk by the value of risk prevention. If the resulting quantity is higher than the value of competing short-termist interventions, it is concluded that existential risk mitigation should be favored.
This approach neglects an important aspect of the value of existential risk mitigation. Interventions taken to reduce existential risk have not only potential benefits, but also potential harms. This series, “Harms” discusses some of the most important harms risked by leading existential risk mitigation efforts.
It is important to discuss the harms of existential risk mitigation for two reasons. First, incorporating harms into the picture allows us to correctly evaluate the case for existential risk mitigation. Second, investigating harms and their causes may help us to identify ways to lessen the harms risked by current existential risk mitigation efforts.
2. Post introduction
One of the most frequent objections to AI risk discourse is that it distracts from the very real harms caused by artificial systems today. This objection becomes polarizing and unhelpful when it is used as a way to avoid engaging with AI risk arguments, or incorporated into a default type of zero-sum thinking which assumes without argument that attention paid to existential risks must necessarily detract from other risks posed by AI.
This is a shame. There are important things to be said in favor of the distraction objection, and these can be rigorously and calmly stated and discussed in a way that will move debates forward. Today’s post aims to investigate what might lie behind the distraction objection.
3. AI harms: Beyond existential risk
Artificial intelligence is responsible for a great number of harms in society today. These harms are important, and deserve to be addressed. Here are a few of the many harms, beyond existential risk, imposed by contemporary systems.
First, many systems are biased against some of the most vulnerable communities, including people of color, women, and transgender individuals. This exacerbates existing racial inequities in the criminal justice system, and leads to a suite of technologies that work significantly better for light-skinned men than for women of color.
Second, existing systems often lack meaningful transparency, interpretability or explainability. For example, workers are evaluated, promoted, hired and fired by algorithms with little or no explanation, threatening important rights such as the right to informed self-advocacy.
Third, intellectual property is increasingly fed wholesale into generative AI systems with little in the way of compensation for artists, journalists, and other content creators.
Fourth, artificial intelligence is concentrating power among a small number of technology firms and nations, already among the world’s wealthiest and most powerful.
Fifth, an escalating global arms race to generate AI systems has led to heightened international tensions, protective trade policies, and escalating military investment in AI-powered weapons systems.
Sixth, privacy is increasingly impossible, with harms falling most strongly on vulnerable groups. As I write these words (but hopefully not when I post them), deepfake pornography of Taylor Swift is circulating widely over the internet, and women across the world are worried that they will never again regain possession of the digital representation of their own bodies.
Seventh, climate change is surpassing even many of our worst fears, and the world continues to miss targets for climate abatement and mitigation. Compute-hungry AI systems are significant contributors to climate change, the harms of which fall strongest on many of the world’s most vulnerable groups.
Finally, misinformation is beginning to run rampant. There are credible fears that this year’s elections in many countries will be significantly sabotaged by AI-generated misinformation, though it is important not to exaggerate these fears.
These are important harms. Regardless of our opinions about existential risk, we should think that it would be valuable to reduce these harms and costly to distract from them. However, existing AI risk discourse threatens to distract from near-term harms of artificial intelligence in several ways.
4. Corporate behavior
Corporations such as OpenAI, Anthropic, X, Microsoft and Google are directly responsible for many of the near-term harms of AI systems. Each of these companies consumes a staggering amount of electricity to train generative AI models. The deepfake pornography of Taylor Swift mentioned above was generated by AI systems and is circulating on social media, in particular on X.
All of the above-named firms have seen strong growth in firm value as well as the power to shape international discourse around the harms generated by AI. We are currently experiencing a remarkable situation in which the very same firms producing harmful technologies are able to use risk discourse to position themselves as leading and benevolent actors in global discourse about the risks posed by artificial intelligence.
By positioning themselves as combatants in a fight against existential risk, corporations are able to paint themselves as safety-focused, thereby reducing pressure to mitigate these harms.
Elon Musk, widely thought to be one of the worst actors in propagating immediate harms, paints his new AI company, xAI, as safety-focused by appointing the director of the EA-Aligned Center for AI Safety, Dan Hendrycks, as its safety advisor. Rather than address rampant misinformation, harassment, racism and other problems on X, Musk tells us that he is helping humanity to survive by colonizing Mars, and addressing existential risk to prevent potential threats to a future Martian colony.
OpenAI commits 20% of its compute to superintelligence alignment, while at the same time facing a range of lawsuits for harms including intellectual property infringement. And for all of OpenAI’s talk about aligning superintelligence, even many effective altruists doubt the company’s focus on safety: OpenAI has continually rushed to release powerful generative AI models, and their discourse around existential risk is so notoriously unconnected to action that effective altruists recently took to picketing OpenAI. The recent board shakeup at OpenAI was precipitated in part by safety-focused criticism of OpenAI by EA-aligned board members, and the resounding defeat of those board members suggested to many that OpenAI may not be as safety-focused as it claims.
At the very least, discourse and spending around existential risk allow powerful AI companies to distract from their own roles in causing near-term harms and to paint themselves as significantly more safety-focused than they often are. At the worst, existential risk discourse can be used to gain a reputation for virtue against a background of severe misbehavior.
We have spoken on this blog about how Sam Bankman-Fried used his commitment to longtermism to gain a reputation for trustworthiness and to secure public influence that led to increased investment and delayed important government investigations into fraud at FTX. While I would not go so far as to allege that Bankman-Fried’s behavior is typical of safety-focused corporate executives, this is far from the first time that EA-aligned technology leaders have found themselves in criminal trouble. Another EA megadonor, Ben Delo, pled guilty to violations of the Bank Secrecy Act for turning a blind eye to money laundering on his cryptocurrency platform.
More generally, a recent investigation of Giving Pledge signatories suggests that a whopping 41% of signatories have been accused of substantial misconduct, 10% convicted, and 4% jailed. Readers would be well within their rights to ask whether the altruistic commitments of many of these parties are entirely sincere, and whether their professed altruism may have helped some of these parties to secure a financially useful reputation for undeserved virtue.
5. Public attention
It is widely held that we have shifted from an information economy to an attention economy. With many problems competing for public attention, only the loudest voices are heard, and it is possible to get away with a great deal so long as the public is not paying attention.
Political leaders are making precious little progress on many of the near-term harms of artificial intelligence. They can, however, distract from these failures by announcing initiatives directed at the most attention-grabbing harms. There is no harm more attention-grabbing than human extinction, so it should come as little surprise that politicians seize on existential risk discourse in part as a way to distract from their failures to address more immediate harms.
For example, recent effective altruist lobbying in London is widely touted as a success of the policy arm of effective altruism. No less a figure than Prime Minister Rishi Sunak convened an international AI Safety Summit, leading to the creation of a UK AI Safety Institute explicitly focused on mitigating extreme threats including existential risks.
With an initial £100 million investment in extreme AI risk, the UK government proudly announces that it is “providing more funding for AI safety than any other country in the world.” But what the government does not stress is just how little is being done to address more mundane harms, such as bias, transparency, misinformation and climate impacts.
It is worth asking whether vocally supportive politicians such as Rishi Sunak and business leaders such as Sam Altman might be taking effective altruists for a ride, using the public interest in existential risk together with the effective altruist funding and public relations machinery to boost their own reputation for safety on the cheap. Sunak, at least, could certainly use the reputation boost.
A second political concern about existential risk discourse is that it may not only distract from, but even directly exacerbate key short-term harms. For example, I noted earlier that a global AI arms race is underway, and this arms race is accompanied by increasing tensions between key players scrambling for supremacy and control of key resources. It has been widely reported that longtermist lobbying in Washington led to strengthened export restrictions on chips sent to China, worsening an already tense relationship between the world’s largest superpowers. Worsening US-China relations are not something to take lightly, and I hope that they will be correctly figured on the balance sheet of AI lobbying.
6. Academic research
I have spoken before on this blog about the extent to which money talks within academia. Academics need money, and few funders can compete with Silicon Valley. The result, I argued, is an inflation of authority and seriousness afforded to positions favored by effective altruist donors; topical biases in the topics discussed by researchers; and opinion biases in the opinions expressed in published research.
Philanthropic money can be used to capture or create research centers, journals, conferences, scholarly societies, funding boards, and other powerful research institutions. Thankfully, studies of existential risk from artificial intelligence have had a difficult time making inroads into academic discussions, but they have lately had some notable successes.
For example, the EA-funded Center for AI Safety attracted a remarkably good class of philosophy fellows, many of whom had not previously worked on or expressed sympathy for discussions of existential risk. This collaboration led, among other things, to a special issue of a leading philosophical journal, Philosophical Studies whose contents are likely to be substantially more skewed towards discussions of extreme risks than is the norm in philosophical research.
There are some other signs that longtermists may be making inroads into academic discussions. For example, the National Science Foundation partnered with Open Philanthropy and Good Ventures to offer $20 million in funding for “safe learning-enabled systems”. I hold out hope that the National Science Foundation, which is widely considered to be a sensible and mainstream funding agency, does not intend to use this money to fund fringe research into existential risks, but some scholars are concerned that this money may drive the field closer towards opinions and topics favored by longtermists, and I think that this must be what Open Philanthropy intends to get for their money.
Of all the distractions considered in this post, I have to say that I am least concerned about turns within academic research. Academics have been, as a rule, quite skeptical of existential risk arguments, and are unlikely to change their minds unless longtermists can provide a good deal more evidence in support of those arguments. However, longtermists have been remarkably successful and working their way into other corners of academia, and we should not ignore the possibility that they will be able to substantially shift academic discourse around AI risk.
7. A track record of ignoring present harms
An important lens into the longtermist push to downplay present harms comes from the observation that this is not the first, but rather the second time that the longtermist community has advocated longtermist policies at the expense of near-term harms.
Many longtermists, including Eliezer Yudkowsky, previously advocated building superintelligent artificial systems and sought to build the systems themselves. What did they say about the problems facing the world right now? At least for Yudkowsky’s part, he suggested ignoring those problems because future artificial systems would solve them:
I have had it. I have had it with crack houses, dictatorships, torture chambers, disease, old age, spinal paralysis, and world hunger. I have had it with a planetary death rate of 150,000 sentient beings per day. I have had it with this planet. I have had it with mortality. None of this is necessary. The time has come to stop turning away from the mugging on the corner, the beggar on the street. … Our fellow humans are screaming in pain, our planet will probably be scorched to a cinder or converted into goo, we don’t know what the hell is going on, and the Singularity will solve these problems. I declare reaching the Singularity as fast as possible to be the Interim Meaning of Life, the temporary definition of Good, and the foundation until further notice of my ethical system.
Here Yudkowsky begins with a correct recognition that the world faces many problems today. Instead of advocating the natural solutions, such as global health and development work as well as climate mitigation, Yudkowsky suggests ignoring them and devoting our efforts to developing superintelligent AI systems which will then take care of most or all present problems. On this basis, Yudkowsky founded an institute dedicated to bringing about the singularity as a means of solving the world’s problems.
Today, longtermists have changed their views about artificial intelligence. They no longer see artificial intelligence as the solution to the world’s ills, but rather as the chief threat to human civilization. But longtermists retain their predecessors’ enthusiasm for diverting money previously reserved for evidence-based solutions to short-term problems to study future AI systems instead.
This discussion suggests that neglecting and distracting from present problems is not a new phenomenon within the longtermist community, and it may not be tied to any particular worldview.
8. What might be done
Distraction is an eminently soluble problem. Everyone knows how to draw attention to an issue, so there is plenty that longtermists can do to mitigate the effects of distraction if they are willing.
First and foremost, longtermists can fund and conduct research and advocacy around risks beyond existential risk. This is the most direct way of ensuring that a broad range of research is conducted and publicized.
Second, longtermists can consider and weigh ways in which their own efforts may contribute to near-term risks. For example, they should be mindful that even favored companies such as Anthropic and (previously, OpenAI) do a great deal to exacerbate near-term harms, and that supporting such companies may contribute to near-term harms. Longtermists should also carefully consider the effects of lobbying on the AI policy agenda in Washington, London and elsewhere.
Finally, longtermists should aim never to dismiss or speak casually about non-existential risks posed by artificial intelligence. Treating risks with the seriousness that they deserve can help to ensure that these risks are addressed seriously.
Of course, to say that longtermists know how to address distraction is not to guarantee that these efforts will be made. I rather suspect that most longtermists regard strategies for mitigating distraction as cost-ineffective. But if that is right, then longtermists should say so directly, and should figure the full cost of distraction on the balance sheet of existential risk mitigation efforts.
Leave a Reply