On your deathbed, God brings good news. Although, as you already knew, there’s no afterlife in store, he’ll give you a ticket that can be handed to the reaper, good for an additional year of happy life on Earth. As you celebrate, the devil appears and asks, ‘Won’t you accept a small risk to get something vastly better? Trade that ticket for this one: it’s good for 10 years of happy life, with probability 0.999.’ You accept, and the devil hands you a new ticket. But then the devil asks again, ‘Won’t you accept a small risk to get something vastly better? Trade that ticket for this one: it is good for 100 years of happy life—10 times as long—with probability 0.9992 – just 0.1% lower. An hour later … you find yourself with a ticket for 1050,000 years of happy life that only works with probability 0.99950,000, less than one chance in 1021. Predictably, you die that very night.
Nick Beckstead and Teru Thomas, “A paradox for tiny probabilities and enormous values“
1. Introduction
This is Part 3 in my series Fanaticism.
Longtermist interventions look best on fanatical decision theories on which any certain gain can be outweighed by arbitrarily small probabilities of sufficiently large gains. This series evaluates the case for fanaticism.
Part 1 introduced fanaticism and its relationship to longtermism, then gave some reasons to doubt fanaticism. Part 2 surveyed further reasons for doubt.
There are certainly good reasons to doubt fanaticism. But there are also good arguments in favor of fanaticism. Today’s post surveys some of those arguments.
2. The spectrum argument
Suppose you deny fanaticism. You think there is some large reward M and some small probability p such that for no reward N, no matter how great, is the option Np of receiving N with probability p better than the certain reward M. Then boy, have I got a forced march for you (Wilkinson 2022).
Consider two lottery options. Option 1 provides a certain reward of M utils. Option 2 provides M2 utils with probability 0.9999. Which one is better? Plausibly, Option 2 is better.
Now consider Option 3, which provides M3 utils with probability 0.99993. Plausibly, Option 3 is better than Option 2.
Proceeding in this way, I can generate a whole spectrum of gambles where Option i pays Mi utils with probability 0.9999i-1. Plausibly, for each i, Option i+1 is better than Option i.
| Option 1 | Option 2 | Option 3 … | Option i … | |
| Probability | 1 | 0.9999 | 0.99992 | 0.9999i-1 |
| Payoff | M | M2 | M3 | Mi |
Now we arrive at a contradiction. Because you deny fanaticism, there is some large reward M and some small probability p such that for no reward N is Np better than M.
Find some i such that 0.9999i-1 <= p. Option i is better than Option i-1, which is better than Option i-2, and proceeding in this way we find that Option i is better than Option 1. So Mi0.9999i-1 is better than M. But then certainly Mip is better than M, since p is at least as large as 0.9999i-1. Take N = Mi to derive a contradiction.
This spectrum argument made very few assumptions. First, it assumed that betterness is transitive. That is plausible enough.
Second, it assumed that a larger chance at a fixed prize is at least as good as a smaller chance at a fixed prize. Again, that is hard to deny.
What if we want to get off the spectrum, so that some Option i is strictly better than the next Option i+1? Well, then perhaps we could increase the step size. Suppose that each prize were 17 orders of magnitude better than the last, instead of one order of magnitude. And suppose that each successive probability were reduced by a factor of 1 – 10-20 rather than 1 – 10-4. Would we still want to get off the boat? This seems to show undue sensitivity to very small differences in probability.
There is some slipperiness here in the order that gambles are specified relative to the choice of parameters M, N and p, though spelling out this slipperiness would take more time than we have here. And perhaps unpacking this slipperiness, together perhaps with other strategies, will be enough to push back against the spectrum argument. But many, including myself, have found much that is intuitively compelling in the spectrum argument.
3. Separability failures
Another consequence of anti-fanaticism is that the betterness of earthly goings-on may depend on uncertainty about events on distant planets that we cannot affect (Beckstead and Thomas 2024).
To see the problem, consider options whose payoffs are future happy lives that will be lived. We can show that anti-fanaticism entails the following: there are some numbers N >> n of happy lives and probabilities q << p such that np+q is not worse than (n+N)p.
Let us start with a warm-up question. Which of the following options is better? (Here the probabilities of outcomes are displayed in the top row, and outcomes are displayed below them).
| p | q | 1-p-q | |
| Option 2 | n | ||
| Option 3 | N |
That is an easy one. Option 3 is better than Option 2. After all, Option 3 provides a much larger chance (p rather than q) of a much larger number of lives (N rather than n).
Now suppose that with probability p, n lives will be lived on Alpha Centauri. You cannot affect them, but we can register your background uncertainty in the following option:
| p | q | 1 – p – q | |
| Option 1 | n |
Incorporating your background uncertainty, Options 2 and 3 revise to:
| p | q | 1-p-q | |
| Option 2‘ | n | n | |
| Option 3‘ | n+N |
Surely Option 3′ remains better than Option 2′. After all, the only thing that has changed is your background uncertainty about matters you cannot affect.
However, this cannot be. By anti-fanaticism, we were able to choose n, N, p and q such that np+q is not worse than (n+N)p. But Option 2′ is the option np+q and Option 3′ is the option (n+N)p.
So even though Option 3 is better than Option 2, once our background uncertainty Option 1 about Alpha Centauri is incorporated, Option 3′ is no longer better than Option 2′. This type of separability violation has seemed strange to many, and again the natural culprit is anti-fanaticism.
4. Taking stock
We saw in Part 1 and Part 2 of this series that there are good reasons to doubt fanaticism. In particular, I would not play the game of Russian roulette introduced in Part 1, and I suspect that fanaticism is the culprit here.
However, we saw today that there are at least two good reasons to accept fanaticism. First, through the spectrum argument (Section 2), those who deny fanaticism can be made to confront a forced march of increasingly attractive gambles. They must get off the boat somewhere, on pain of accepting fanaticism. But it is hard to find a principled reason to jump ship.
Second, anti-fanaticism leads to separability violations (Section 3). One option can be obviously better than another, until our background uncertainty about matters outside our sphere of influence is incorporated. Then, mysteriously, the first option is no longer better than the second. It is hard to see why this should matter.
There is another thing to be said in favor of fanaticism. Many fanatics will grant that fanatical decision theories such as expected utility theory have some counterintuitive consequences. Nevertheless, they counter, anti-fanatical decision theories have at least as many counterintuitive consequences of their own. As a result, until a viable anti-fanatical decision theory is specified and defended, we might see the counterintuitive consequences of fanatical decision theories as mere illustrations of the dictum that the shoe always pinches somewhere.
The next post in this series makes good on this argument by exploring problems for anti-fanatical decision theories.
Leave a Reply