Of course, it is prudent to take measures to prepare against the possibility of a biological weapons attack and concerted action across a policy continuum that extends from prevention through preparedness to consequence management is necessary. However … any bioterrorism attack will most likely be one using a pathogen strain with less than optimal characteristics disseminated through crude delivery methods under imperfect conditions, and the potential casualties of such an attack are likely to be much lower than the mass casualty scenarios frequently portrayed. This is not to say that speculative thinking should be discounted … however, problems arise when these speculative scenarios for the future are distorted and portrayed as scientific reality.
Jefferson, Lentzos and Marris, “Synthetic biology and biosecurity: Challenging the `myths’“.
1. Recap
This is Part 10 of my series Exaggerating the risks. In this series, I look at some places where leading estimates of existential risk look to have been exaggerated.
Part 1 introduced the series. Parts 2-5 (sub-series: “Climate risk”) looked at climate risk. Parts 6-8 (sub-series: “AI risk”) looked at the Carlsmith report on power-seeking AI.
Part 9 began a new sub-series on biorisk. In Part 9, we saw that many leading effective altruists give estimates between 1.0-3.3% for the risk of existential catastrophe from biological causes by 2100. I think these estimates are a bit too high.
Because I have had a hard time getting effective altruists to tell me directly what the threat is supposed to be, my approach is to first survey the reasons why many biosecurity experts, public health experts, and policymakers are skeptical of high levels of near-term existential biorisk. Then I will review the arguments for high levels of existential biorisk that have been provided by effective altruists and argue that they are insufficient.
Part 9 began by reviewing three preliminary reasons why many are skeptical of existential biorisk claims: that engineering biological events that lead to existential catastrophe is a very difficult problem; that biological events have rarely led to mammalian extinction; and that producing all of the needed mutations at once is so difficult that a well-funded (alleged) Soviet bioweapons program didn’t come close.
Today’s post continues this theme by looking at four more preliminary reasons for skepticism: the possibility of effective public health responses; a scarcity of motivated omnicidal bioterrorists; expert skepticism about existential biorisk; and challenges related to the inscrutability of future biorisk.
2. Public health response
One of the strongest lessons from the COVID-19 pandemic is that simple non- pharmaceutical interventions such as masking, social distancing, and travel restrictions, can be highly effective if governments have the will to enforce them. For example, a study in Nature of early COVID measures in mainland China found that non-pharmaceutical interventions reduced cases by a factor of 67 by the end of February 2020. Here are their estimates for the course of the outbreak in Wuhan (b) and Hubei (f) with and without intervention.
That is not to say that non-pharmaceutical interventions are a panacea. They work less well when they are not consistently enforced. And in the case of China, they ultimately failed to counteract the effects of insufficient vaccination among the elderly. But used properly, non-pharmaceutical interventions are a remarkably effective tool in slowing the spread of disease.
Another novel feature of the COVID-19 pandemic is that, for the first time, an active pandemic was brought to an end through real-time development and deployment of vaccines. Although vaccines took over a year to bring to market, serviceable vaccines were quite quick to develop, and a society facing existential catastrophe might well bring a serviceable vaccine to market far more quickly, on a scale of months or even weeks, and advances in medical technology might bring further improvements beyond this. This would be particularly true if societies were willing to loosen restrictions on human trials and regulatory approval, which a society facing existential catastrophe might well do.
It is hard to exaggerate how novel and exciting this development is. Historically, societies have been unable to develop novel pharmaceutical responses to pandemic pathogens in time to affect the course of an initial pandemic. This would be deeply frightening in the face of potentially sophisticated engineered pathogens. But it is not just our ability to engineer pathogens that is advancing. We are also increasingly gaining the ability to develop effective vaccines to counter novel pathogens.
Would-be omnicidal bioterrorists need to develop a pathogen that is not only in principle able to destroy all living humans, but which will succeed in practice despite our best efforts to stop it. Humans are not sitting ducks, and we are unlikely to take a world-ending pandemic complacently.
3. Motivations
Here is a deceptively simple question. Suppose that it were to become technologically feasible for sophisticated groups of bioterrorists to cause extinction-level biothreats. Who would do such a thing?
It may seem obvious that there are plenty of sophisticated bioterrorists clamoring for the chance to eradicate humanity. But in fact, history holds few if any examples of sophisticated bioterrorists bent on destroying humanity.
We saw in Part 2 of the series on epistemics that when pressed to produce an example of a group of sophisticated, omnicidal bioterrorists, effective altruists almost inevitably appeal to Aum Shinrikyo. For example, here is Toby Ord in The Precipice:
People with the motivation to wreak global destruction are mercifully rare. But they exist. Perhaps the best example is the Aum Shinrikyo cult in Japan, active between 1984 and 1995, which sought to bring about the destruction of humanity. They attracted several thousand members, including people with advanced skills in chemistry and biology. And they demonstrated that it was not mere misanthropic ideation. They launched multiple lethal attacks using VX gas and sarin gas, killing 22 people and injuring thousands. They attempted to weaponise anthrax, but did not succeed. What happens when the circle of people able to create a global pandemic becomes wide enough to include members of such a group? Or members of a terrorist organisation or rogue state that could try to build an omnicidal weapon for the purposes of extortion or deterrence?
Only, we saw that this is simply not true. Aum Shinrikyo was not omnicidal. Aum Shinrikyo aimed to produce an event that they themselves would survive. From there, they aimed to rebuild Japan and later the entire world. They prepared in detail for this eventuality, including tasking their qualified medical staff with drawing plans for survival, as well as outlining a strategy for the creation of `lotus villages’ that would survive an attack.
Could history hold another example of sophisticated bioterrorists with omnicidal intentions? Experts are skeptical. Kathleen Vogel and Sonia Ben Ouagrham-Gormley explain in a study of emerging biosecurity threats that:
Since the 1980s, there have been a variety of actors “crying wolf” about how states and terrorists will adopt and use genetic engineering techniques for harm. Yet we have little empirical data over the past 30 years that show a specific state or terrorist group using any of these new biotechnological innovations to create biological weapons.
That skepticism is echoed elsewhere. A former bioweapons inspector and diplomat interviewed for BioSocieties explains:
You’ve got to start thinking about for whom this would be a valid weapon of choice. I think for the small-scale attack it would still be of interest to terrorists because it would be a small-scale attack with big impact. But if you start talking about the stuff that can cause huge damage – live, highly transmissible, highly pathogenic organisms – there’s very few actors I can think of who would want to use that. I think you have to come up with hypothetical actors rather than real-world actors.
Importantly, this expert does not deny the likelihood of actors being motivated to carry out small-scale attacks. But he is highly skeptical that many actors would want to destroy all of humanity.
Insofar as effective altruists struggle to produce a single, credible historical example of a sophisticated omnicidal bioterrorist group, and insofar as experts describe such actors as “hypothetical actors rather than real-world actors”, there are serious questions to be raised about the assumption that sophisticated actors in this century will be motivated to use biological weapons to bring about an existential catastrophe.
4. Expert skepticism
A continual theme in discussions of existential risk is that leading experts are highly skeptical of the existential risk estimates put forward by effective altruists. I hope to discuss the role of attitudes towards expertise in my series on epistemics. Experts need to be taken seriously: when the experts think your views are highly implausible, that’s usually because the views are highly implausible. This is doubly true when the banner of infohazards is used to obviate the need for detailed public argument against expert consensus.
A group of health researchers from King’s College, London describe the statement that “terrorists want to pursue biological weapons for high consequence, mass casualty attacks” as a “myth”, writing:
While most leading biological disarmament and non-proliferation experts believe that the risk of a small-scale bioterrorism attack is very real and very present, they consider the risk of sophisticated large-scale bioterrorism attacks to be very small.
The claim here is not merely that existential catastrophe is unlikely to result from bioterrorism. The study’s authors rather claim that most experts consider any sophisticated large-scale bioterrorist attack to be unlikely.
One of the authors of that study, Filippa Lentzos, followed up by convening an expert panel on risks posed by bioweapons. The experts unanimously echoed the view that high-casualty attacks are extremely unlikely, and that even mainstream political actors tend to greatly exaggerate the risks of such attacks. Speaking for the consensus, she put it to them that:
You all seem to agree that what we should be most worried about is a small-scale bioterrorism attack – that the likelihood of that is much higher – where we’re talking about deaths in the tens and not in the thousands. Your main concern, then, is not the same as the Obama administration’s: major biological weapons attacks on the world’s major cities that cause “as much death and economic and psychological damage as a nuclear attack”.
A former bioweapons inspector and microbiologist responded straightaway: “No, definitely not.”
In another paper, David Sarpong and colleagues lament “Sensational and alarmist headlines about DiY science” which “argue that the practice could serve as a context for inducing rogue science which could potentially lead to a ‘zombie apocalypse’.”
Experts widely believe that existential biorisk in this century is quite low. The rest of us could do worse than to follow their example.
5. Inscrutability of future biorisks
In this series, I have discussed a pattern that I call a regression to the inscrutable.
Effective altruists began by discussing risks such as asteroids, nuclear war and super-volcanoes that are to some extent tractable using traditional empirical methods. Careful research revealed that most of these risks were not very high.
As a result, research shifted to a range of increasingly esoteric and inscrutable threats. The frustration for critics of effective altruism is that it is agreed on all sides that these challenges are highly inaccessible to traditional, or even slightly nontraditional forms of empirical research. As we regress further into the realm of inscrutable threats, there is increasingly little that can be said using traditional methods. And critics of effective altruism, who are often skeptical of more speculative methods, think that should be the end of the matter.
So far, we have seen a regression to the inscrutable in at least two places:
- Climate risk: Effective altruists place substantial emphasis on the very least scrutable climate risks, namely moist and runaway greenhouse effects. (Sub-series: Climate risk).
- AI risk: Effective altruists place substantial emphasis on concerns about existential catastrophe from future artificial systems. Developments of this scale in AI are, by nature, very difficult to get a grip on using traditional methods. (Sub-series: AI risk).
This regression to the inscrutable contrasts with effective altruists’ early emphasis on scrutability, but also makes it difficult to press the case for inscrutable risks in a way that will be convincing to a wide audience.
Existential biorisk is relatively inscrutable in at least two ways. First, it relies on speculative developments in future technology for engineering and distributing pathogens. Everyone agrees that existential biorisk is not high now, because we do not currently have the technology to design and distribute a pathogen that could lead to existential catastrophe. However, we are asked to believe that biorisk will be much higher in the future, due to developments in genetic engineering and other technologies. It is hard to get a handle on what these developments might be, since they are by nature speculations about future technologies.
A second reason why existential biorisk is relatively inscrutable is that effective altruists are often hesitant to share arguments for existential biorisk due to concern about infohazards. This means that even if, in principle, it were possible to scrutinize claims about existential biorisk, it will be very hard in practice for those asked to believe these claims to subject them to rigorous scrutiny.
Concerns about scrutability underlie many experts’ resistance to existential biorisk claims, because there is an abundant record of far-future technological predictions landing far from the mark. Let’s return to the study of emerging biosecurity threads by Kathleen Vogel and Sonia Ben Ouagrham-Gormley cited earlier in this post. What is driving Vogel and Ouagrham-Gormley’s skepticism? In large part, they are worried about a track record of mistaken predictions about far-future technology. Vogel and Ouagrham-Gormley illustrate their skepticism by considering previous speculation about the future use of nuclear energy:
In a 1955 issue of Ladies’ Home Journal, it was touted that in the near future, nuclear energy would create a world “in which there is no disease … where hunger is unknown … where food never rots and crops never spoil … and routine household tasks are just a matter of pushing a few buttons … a world where no one stokes a furnace or curses the smog.” “Imagine,” the article continued, “the world of the future … the world that nuclear energy can create for us.” As historian of technology Stephen Del Sesto writes, these thoughts were not those of an overzealous journalist but of Harold E Stassen, President Dwight D Eisenhower’s special assistant on disarmament. Other prominent experts believed that “atomic batteries would power automobiles, washing machines, and even tiny wrist-watch radios.” At that time, optimism about nuclear energy was shared by many Americans after decades of technological enthusiasm inculcated by a variety of American popular culture and press accounts and scientific, academic, and government pronouncements — these technological dreams about nuclear energy captured the U.S. popular imagination for nearly 30 years. Del Sesto writes, “these forces reinforced one another making the dreams appear more plausible and closer at hand than they really were.” What was left out of these utopian imaginaries was any consideration of how a variety of social, economic, organizational, and political realities might shape the development, adoption, and use of nuclear energy that would hinder these dreams from becoming reality. It is quite easy to dream up a large number of fanciful imaginaries for any given technological innovation — the critical question, then, is how any of these imaginaries connect with on-the-ground reality as a technology develops?
This post began with a passage from work by health researchers at King’s College, London, who as we saw urged caution in the interpretation of speculative reasoning about future technologies:
[We do not] say that speculative thinking should be discounted … however, problems arise when these speculative scenarios for the future are distorted and portrayed as scientific reality.
It is very easy for speculation about relatively inscrutable future technological developments to become highly optimistic and utopian, and equally easy for this speculation to become highly pessimistic and dystopian. Like Vogel and Ouagrham-Gormley, and like Jefferson and colleagues, I hope that reflection on past instances of overly optimistic and pessimistic speculation about inscrutable future technological developments may encourage moderation in our predictions about future developments in biology and related areas.
6. Taking stock
Today’s post discussed four additional reasons for skepticism about high estimates of existential biorisk in this century.
First, biological threats can be met with effective public health responses. These may be as simple as the now-familiar package of non-pharmaceutical interventions such as masking, social distancing and travel restrictions. But they can also be as complex as the increasingly rapid and sophisticated techniques for bringing vaccines to market in time to counteract a novel pandemic.
Second, it is hard to find credible examples of sophisticated actors with both the capability and the motivation to carry out omnicidal attacks. Many groups wish to cause harm to some fraction of humanity, but far fewer groups wish to kill all of us.
Third, experts are generally skeptical of existential biorisk claims. Experts do not doubt the possibility or seriousness of small-scale biological attacks, and indeed such attacks have occurred in the past. But experts are largely unconvinced that there is a serious risk of large-scale biological attacks, particularly on a scale that could lead to existential catastrophe.
Fourth, the biorisks being appealed to are relatively inscrutable in two ways: they involve speculation about far-off future technologies, and much of this speculation is hidden from public view. This makes it difficult to analyze the case for existential biorisk, and also fits an emerging pattern of appeal to inscrutable risks. It is as though the weightiest risks always seem to lie just near enough in our temporal and technological future to make them intelligible, but not so near that they could be easily refuted by standard forms of analysis, scientific or otherwise.
As always, these are not decisive refutations of the possibility of existential biorisk. They are rather preliminary reasons to be skeptical of high estimates of existential biorisk. Together, these and other reasons for preliminary skepticism will combine to raise the evidential bar for establishing high levels of existential biorisk. Future posts will argue that this bar has not been cleared.
Leave a Reply