One obvious point of contrast between the paradigm examples of ways to attain high near-future vs. far-future expected benefits is that the former tend to involve high probabilities of relatively modest benefits, whereas the latter tend to involve tiny probabilities of enormous benefits … Even among those who are sympathetic in general to expected utility theory, many balk at its apparent implications for cases of this latter type … One might, then, seek a ‘non-fanatical’ decision theory—one that does not require the agent to sacrifice arbitrarily much, with probability arbitrarily close to 1 in ‘fanatical’ pursuit of an extremely unlikely but enormously larger payoff.
Hilary Greaves and Will MacAskill, “The case for strong longtermism“
1. Introduction
One feature of many longtermist interventions is that they offer a very low chance of a very large payoff. Longtermist interventions stand in direct competition with short-termist interventions such as global health and development which offer a very high chance of a sizable, but not astronomical payoff.
Longtermist interventions look best on fanatical decision theories, such as expected utility maximization. These theories are fanatical in the sense that for any certain payoff N, no matter how large, and any small probability p, no matter how small, there is some larger payoff M such that having only p chance of getting M is better than getting N for sure.
Longtermist interventions look worse on anti-fanatical decision theories which lack this property. Anti-fanatical decision theories tend to be significantly less enthusiastic about chasing small chances of large gains, and significantly more enthusiastic about chasing sizable and likely wins.
For this reason, a crucial question for longtermists is whether fanatical decision theories are plausible. This post begins a new series, Fanaticism, examining the plausibility of fanaticism.
2. Aims of this series
Fanaticism is one of many topics relevant to longtermism with the following characteristics. First, there is an excellent and sizable academic literature on the topic. Second, much of this literature is not well-known to rank-and-file longtermists. And third, this literature is not well-known because it is difficult and locked behind paywalls.
My aim in this series, and in future series of its kind, is to help longtermists understand the best arguments for and against fanaticism so that they can make an informed choice. In particular, I hope to convince readers of three claims.
First, there is something deeply troubling about fanaticism. There are many plausible reasons to be concerned about fanaticism, and few of us would implement fanatical decision theories in our own personal lives.
Second, there are excellent arguments in favor of fanaticism. For this reason, it is by no means obvious that the troubles with fanaticism should lead us to reject fanaticism.
Finally, there are problems with all leading anti-fanatical decision theories. For this reason, those who would reject fanaticism would do well to supply a better alternative. Many, including myself, would be much more willing to reject fanaticism if given a better theory, but until that theory is provided, it is hard to invest very high credence in anti-fanaticism.
More generally, I hope that readers will be deeply puzzled by the existing literature on fanaticism. If I can convince my readers that questions about fanaticism are difficult and unresolved, I will have done my job.
3. Defining fanaticism
To define fanaticism, we need some notation. Let M and N be outcomes: results of chancy options. Payoffs can be expressed in dollars, DALYS, lives saved, or any other unit. Throughout this series, I will abuse notation slightly and let M and N also denote options which are certain to yield M or N, respectively.
For any probability p, let Np be an option which yields N with probability p and otherwise nothing. For example, N0.1 has a 10% chance of yielding N and a 90% chance of yielding nothing.
(Axiological) fanaticism says that for any outcome M, no matter how large, and for any probability p > 0, no matter how small, there is another outcome N such that Np is better than M.
It is important to stress that fanaticism is a claim about any certain payoff M, no matter how large, and any nonzero probability p, no matter how small. For example, fanaticism holds that there is some prize N such that a one-in-a-quintillion chance of receiving N (and otherwise nothing!) should be preferred to the preservation of the world today.
4. Fanaticism and longtermism
Before evaluating the case for fanaticism, it is important to think carefully about the relationship between fanaticism and longtermism.
Certainly, anti-fanaticism is unlikely to do longtermists any favors. But there are at least two reasons to think carefully about the relationship between fanaticism and longtermism.
First, anti-fanatical decision theories often punish very low probabilities of large gains, but once we ask how low these probabilities must be, there is some room to hope that leading longtermist interventions may not be unduly punished. For example, while the Marquis de Condorcet recommended ignoring probabilities near one in ten thousand, a more recent anti-fanatical theory due to Bradley Monton recommends ignoring probabilities below one in five hundred quadrillion. While many longtermist interventions do not have a one-in-ten-thousand chance of succeeding, they may have at least a one-in-five-hundred-quadrillion chance of succeeding.
Second, as Hilary Greaves and colleagues stress, longtermist interventions seem least likely to succeed when we evaluate the difference that a single individual is likely to make. For example, if we ask how likely a single thousand-dollar donation is to prevent human extinction, this probability may well be extremely low.
However, suppose we ask a different question. Suppose we begin by asking how likely humanity is to suffer existential catastrophe. That likelihood may well be above thresholds such as one-in-five-hundred-quadrillion. On some natural anti-fanatical decision theories, the probability of existential catastrophe will then not be significantly discounted, nor will the effects of our actions on the probability of existential catastrophe be significantly discounted, however small they may be.
None of this is meant to suggest that anti-fanaticism should be untroubling for longtermists. A number of anti-fanatical decision theories will reduce the importance of moderately low probabilities in decisionmaking. And when empirical uncertainty about the impacts of longtermist interventions is combined with the uncertainty generated by many overlapping challenges to longtermism, the resulting probabilities of scenarios most favorable to longtermists may well be vanishingly small.
But for all this, it is important to bear in mind that there is no airtight relationship between anti-fanaticism and the denial of longtermism. Readers are invited to think carefully about the implications of various anti-fanatical arguments for their favorite longtermist interventions given other background views.
5. Are you really a fanatic?
Suppose you are indeed committed to fanaticism. Let M be the outcome in which your life continues as normal. Let p be a vanishingly small probability – say, one in a quintillion.
Fanaticism then implies that there is some prize N such that it would be better to receive a one-in-a-quintillion chance of N (and otherwise death), than the certain chance of your own life M.
Fanaticism implies the very same thing if the probability of the excellent prize is shortened to , or ten to the power of the largest number you care to name. And fanaticism implies the same if the excellent prize M is sweetened to a billion years of bliss.
Please take a moment to ask yourself: is there any prize so great that you would accept a chance at receiving this prize over a certain billion years of bliss? I hope that most readers will at least think twice about this. It is, of course, possible that we are irrationally disposed to favor certainties over risky gambles, or that we would think differently if we were able to truly comprehend the large numbers involved.
But for my part, I do not think I would take this gamble, and I do not think my refusal is due to any irrationality on my part. I would not take this gamble because I do not think it is better to have a tiny chance of an unbelievably good outcome than to enjoy a certain billion years of bliss.
If you think the same, then you must reject fanaticism.
6. Pascal’s mugging
There are a number of other reasons to be skeptical of fanaticism. Many are on the technical side, and will be left for the next post in the series. But I’d like to begin by considering one famous objection due to Nick Bostrom.
Look at your wallet. Hopefully, it contains some money, but if nothing else, I trust that you value the wallet itself. Let M be the outcome in which you keep your wallet.
Nevertheless, I propose that you should give me your wallet. Why? Well, if you do, I will use my magic powers to reward you.
Don’t trust me? I don’t blame you. But surely there is some nonzero probability p that I am telling the truth. By fanaticism, there is some very good outcome N such that Np is better than M.
Fine, then. I promise to give you reward N if you give me your wallet. Now, it seems, it would be better to hand it over.
And in fact, it is not clear that I have to promise you, as a longtermist, anything. As Dylan Balfour notes, I might as well just promise to do something very, very good for your future descendants.
I can repeat the same trick every day of your life, increasing N as needed, to ensure that you never again cash a paycheck. That seems wrong. But what has gone wrong? A natural culprit is fanaticism.
7. Responding to Pascal’s mugging
I don’t think that Pascal’s mugging is the end of the story about fanaticism. In fact, I don’t think that Pascal’s mugging is even the best argument against fanaticism. But for all that, it is not so easy to say what is wrong with the mugging.
Should we assign credence zero to the mugger’s claims being true? This contravenes the famous Bayesian requirement of regularity, which asks us to avoid at all costs assigning zero credence to empirical claims because it is difficult or impossible to ever update a zero-probability claim, no matter how much contrary evidence we later receive.
Should we bite the bullet and claim there is nothing counter-intuitive about Pascal’s mugging? It is one thing to think that Pascal’s mugging is a bullet the fanatic must bite, but quite another to claim to enjoy biting bullets. There is something counter-intuitive about Pascal’s mugging, and this threatens to count, at least to some degree, against the plausibility of fanaticism.
There is at least one response to Pascal’s mugging worth taking seriously. This response, due to Peter Baumann, appeared in the same issue of Analysis as Bostrom’s original article on Pascal’s mugging. Here is the idea.
Suppose you have $100 in your wallet. By fanaticism, there is some prize N such that a one-in-a-million chance of N is better than a certain $100.
I promise you N in exchange for your wallet. You might refuse, not because you reject fanaticism, but because you assign probability p below one-in-a-million to my promise.
No problem, I say. So long as p is nonzero, there is some prize N’ for which a probability p of N’ is better than a certain $100. So I promise to give you N’.
Again, you refuse. As implausible as it was to think that I would deliver N, it is still more implausible to you that I will deliver N’. After all, N’ is both harder to procure, and more valuable, giving you more reason to withhold it. So you don’t have credence p’ in my giving you N’.
No problem, I say. So long as you assign some nonzero probability p” to my giving you N’, then I can promise you some larger N” which makes it better to hand over your wallet.
Again, you protest, you assign lower probability p”’ to my going through with this larger offer.
Can you resist the mugging in this way? Yes, you can, provided we occupy a very particular kind of diminishing marginal returns regime: the probability that the mugger will successfully follow through decays faster than the reward he promises.
Certainly this kind of diminishing returns regime is not impossible. It is precisely the kind of diminishing returns regime I rely on in my argument from rapid diminution in my paper and blog series “The scope of longtermism.”
To be honest, I am sympathetic to this sort of reply to Pascal’s mugging and I hope it works. One fact which gives me pause is that we are not betting on a single concrete diminishing returns regime, but on every possible diminishing returns regime of this type. We think that no matter the kind of mugger we are faced with, the circumstances of his offer and the plausibility of these circumstances, the probability of the mugger’s follow-through must always decay more quickly than its value. That is a strong claim about the large and vast space of possible muggings, and I hope that those who make such a claim make it tentatively.
8. Conclusion
Today’s post introduced a new series on fanaticism. Fanatics hold that for any certain payoff M, no matter how large, and for any nonzero probability p, no matter how small, there is some prize N for which a probability p of receiving N is better than a certainty of receiving M.
I hope in this series to convince you that (1) fanaticism is deeply troubling, but (2) there are good arguments in favor of fanaticism and (3) all leading anti-fanatical decision theories have problems. Today’s post began the first project by looking at some concerns for fanaticism.
The thought experiment in Section 5 is meant to illustrate how even many longtermist readers may have reservations about fanaticism. Few of us would be willing to follow the dictates of fanaticism to their limits in our own lives, and this might not indicate a problem with our moral discipline, but rather with fanaticism.
Sections 6-7 looked at one of the most famous challenges to fanaticism: Pascal’s mugging. I hope readers will agree both that there is some plausibility to this challenge and that many of the most natural responses, while not without merit, are by no means guaranteed to succeed.
If these were the only concerns for fanaticism, then fanatics might breathe somewhat easily. But in the next post in this series, we will see that there are a good many concerns beyond these.
Leave a Reply