Overconfident predictions about AI are as old as the field itself.
Melanie Mitchell, “Why AI is harder than we think“
1. Introduction
This is Part 22 of my series Exaggerating the risks. In this series, I look at some places where leading estimates of existential risk look to have been exaggerated.
Part 1 introduced the series. Parts 2-5 (sub-series: “Climate risk”) looked at climate risk. Parts 6-8 (sub-series: “AI risk”) looked at the Carlsmith report on power-seeking AI. Parts 9-17 (sub-series: “Biorisk“) look at biorisk.
Part 18 continued my sub-series on AI risk by introducing the AI 2027 report. Parts 19, 20 and 21 argued against key contentions of the AI 2027 report. Today’s post concludes my discussion of the AI 2027 report.
2. Recap
Part 18 of this series introduced the AI 2027 report. The most widely read component of that report is a 71-page scenario written by the authors and then rewritten by Scott Alexander “in an engaging style.” Based on a series of table-top exercises beginning in the titular year 2027, the scenario describes a variety of hectic ways in which the future of artificial intelligence might proceed. Most end badly for humanity.
The report’s technological predictions are based on two reports: an AI timelines forecast predicting the time needed to develop superintelligent coders, and an AI takeoff forecast predicting the time needed to move from superintelligent coders to artificial superintelligence.
Part 19 and Part 20 of this series looked at the AI timelines forecast. This forecast consists of two component models. We saw in Part 19 that the first model largely bakes in the hyperbolic growth trajectory that it aims to establish. And we saw in Part 20 that the second model relies on a series of sparsely evidenced forecasts of the dates when a series of milestones will be reached.
Part 21 of this series looked at the AI takeoff forecast. We saw that this forecast relies on weak data, under-justified parameter forecasts, and very wide uncertainty ranges over possible future developments.
In the rest of this post, I want to conclude by drawing three lessons from this discussion.
3. The epistemic status of AI risk claims
I have spent a great deal of time publicly refusing to respond to many leading arguments for existential risk from artificial intelligence. I have done this because I think that most such arguments fall significantly short of the evidential standards needed to generate substantial credence in their conclusions, or even to enable an evidence-based response.
This is not always the case. To my mind, at least two sets of AI risk claims meet the bar for a serious response: arguments for the singularity hypothesis by Chalmers, Bostrom and others, and power-seeking arguments by Alex Turner and colleagues. In both cases, I have responded with scholarly papers taking the contrary view.
I have no plans to address the AI 2027 report in a scholarly paper. The only reason why I addressed the report is that it garnered significant popular attention. This made it imperative to do what I could to reduce the attention paid to the report, and to demonstrate again why I take such arguments to fall beneath the epistemic standards needed to generate serious credence in their conclusions.
In this vein, perhaps the most important legacy of the AI 2027 report is a reminder of just how many prominent claims about the levels of existential risk from artificial intelligence fall beneath the epistemic standards that they should aim to meet. If those concerned about existential risk from artificial agents wish to ground substantial rational credence in their risk estimates, they need to present good models based on solid data. This is too seldom done.
4. Epistemic dangers of public advocacy
Early critics of effective altruism held that the movement was adept at scholarship, but woefully inept at public-facing advocacy. Effective altruists responded at the time that they would rather take the time to make a research-backed case for their interventions instead of devoting their efforts to popular outreach. If the best arguments took too much time and data to read, then so much the worse for those who did not read them.
Since then, effective altruists have significantly increased their public advocacy. Indeed, 80,000 Hours recently launched a video series entitled AI in context whose first video, based on the AI 2027 report, garnered over 7 million views.
I cannot say that I entirely fault effective altruists for this move. However, the foray into public advocacy has significant epistemic costs. The arguments that move public opinion are not always the best arguments, or even particularly good arguments.
The vast majority of readers encountering the AI 2027 report saw Twitter and YouTube summaries. If they were lucky, they glanced at the 71-page fictional description dramatizing how the future of artificial intelligence might go. Some of the most devoted readers may have even finished reading this fictional scenario. And a precious few will have spent some time reading the reports underlying this fictional scenario, though many will not have finished them and few of these will have substantially examined their claims.
The danger of this type of public advocacy is that very little of its persuasive force depends in any substantive way on the accuracy of its claims. The research underlying the AI 2027 scenario could have been substantially better or worse without notable change in the reactions of most readers.
If effective altruists are very confident that they are right, then this should not be especially troubling. When the truth must be told, it should be communicated by any available means, rational or otherwise.
But those who harbor significant doubts about the correctness of AI risk claims should be concerned about the effectiveness of persuasive methods which do not essentially rely on the strength of the research-backed case for AI risk. If these arguments are wrong, they will have arationally persuaded a great number of readers to take the wrong view of the future of artificial intelligence, and more generally to invest inappropriately in the mitigation of existential risk from artificial intelligence instead of in competing causes.
5. Two types of errors
Much of our discussion of the AI 2027 forecasts oscillated between two complaints. The first was a complaint against model-driven forecasts. Here, the complaint was that the models often built in the very hyperbolic functional form that they were aiming to derive.
The second complaint was directed at model-light forecasts. Here, the complaint is that most of the weight of the forecast fell upon forecasts of model parameters. These models are then no more reliable than the ability of modelers to reliably forecast model parameters. In many cases, we saw that the forecasts are driven by broad-stroke speculative reasoning beyond the supporting data, and of a kind that we have little reason to think the forecasters should be able to carry off with any reliability.
These two complaints can and should be pressed against many models forecasting rapid future growth in artificial intelligence. On the one hand, we should demand models which avoid baking-in the conclusions being derived. This means that there should be a substantive role for substantive questions about model parameters to determine model behavior.
On the other hand, we should demand parameter estimates grounded in data and made in domains and on timescales for which the forecasters have a good claim to be operating substantially above chance-level forecasting.
Threading the needle between these two complaints is a difficult task. If I am right, then the AI 2027 team did not succeed in threading this needle. Neither, to my mind, do many other leading estimates. Readers should scrutinize related forecasts to see that they both avoid baking growth assumptions into models, and also avoid relatively ungrounded parameter estimates. If it turns out that many forecasts fail to thread the needle between these two complaints, then that is some reason to be skeptical about risk claims based on fast growth trajectories.
6. Taking stock
This concludes my discussion of the AI 2027 report.
While the report is based on several other research reports, to my mind these reports are often not as weighty, novel, or influential in determining the report’s conclusions as the timelines and takeoff forecast were.
For this reason, I do not currently plan to address the other research reports put out by the AI 2027 team. Nor am I in the habit of responding to narratives. Everyone loves a good story, but a good story must be backed by solid research if we are to take it as true.
I hope to continue my sub-series on AI risk in the future if I am able to identify reports or papers which could warrant a detailed response.
Leave a Reply