There is no falsification before the emergence of a better theory.
Imre Lakatos, “Falsification and the methodology of scientific research programmes.”
1. Introduction
This is Part 4 in my series Fanaticism.
Longtermist interventions look best on fanatical decision theories on which any certain gain can be outweighed by arbitrarily small probabilities of sufficiently large gains. This series evaluates the case for fanaticism.
Part 1 introduced fanaticism and its relationship to longtermism, then gave some reasons to doubt fanaticism. Part 2 surveyed further reasons for doubt.
However, we saw in Part 3 that there are also good reasons to doubt anti-fanaticism.
Continuing in this vein, today’s post begins with a simple challenge: is there an anti-fanatical decision theory that performs better than leading fanatical competitors? We will see that two leading anti-fanatical decision theories both face serious challenges. As a result, even those unhappy with fanaticism may also be unhappy with anti-fanaticism.
2. Bounded utilities
Many longtermists are sympathetic to expected utility theory. They think that agents should assign probabilistic credences p to states and utilities u to outcomes, then act to maximize expected utility given p and u.
Within expected utility theory, there is a remarkably simple characterization of anti-fanaticism. Expected utility maximizers are anti-fanatical if, and only if their utility function is bounded.
To see that bounded utilities are sufficient for anti-fanaticism, let n be a utility value at least halfway to the upper bound of u(x). Then for any p < 1/2, a p-wise chance at any prize, no matter how large, has expected utility beneath n. So our bounded expected utility maximizer will not accept any uncertain prize with probability p < 1/2 over a certain gain of utility n.
To see that bounded utilities are necessary for anti-fanaticism, let n be any utility value and let p be any positive probability. If u were unbounded, then we could find an outcome with value greater than np. A p-wise chance at this outcome would then be preferred to a certain gain of n. Since p,n were arbitrary, fanaticism follows.
What should we make of bounded utility theory? Certainly, it is a well-pedigreed academic theory. And there are many circumstances in which bounded utilities are plausible. They are especially plausible for consumption goods, because there are only so many of them that we can consume or even sell.
For example, consider the utility of pints of beer. Plausibly, one is much better than none, since you can drink it. Likewise, two million pints of beer are somewhat better than a million, because you can sell them. But two quadrillion pints of beer are not appreciably better than one quadrillion pints of beer. There is not much you can do to drink or sell the excess.
However, bounded utilities are less plausible when we consider outcomes expressed in years of happy life. Suppose the world contained a quadrillion happy life-years. It is not so clear that this would make additional years of happy life insignificant, or even less valuable than previous years.
If utilities were really bounded when expressed in years of happy life, then there would have to be some number N such that, in a universe with N years of happy life, adding a quadrillion more years of happy life would do less good than curing a hangnail would do today. That is a very hard pill to swallow. It is plausible for pints of beer, but not for years of happy life. It is just not obvious that the value of happy life-years should tend to an asymptotic bound in the same way that the utilities of consumption goods should.
3. Nicolausian discounting
The best-known anti-fanatical decision theory is Nicolausian discounting (Monton 2019). Nicolausian discounting holds that very low probabilities should be ignored in decisionmaking. That is, Nicolausian disocunting posits a threshold ε such that outcomes with probability below ε are ignored during decisionmaking.
This proposal cannot be quite right as it stands. Suppose I offer to extend your life by a real number r of years, with r selected uniformly from the interval [0,100]. Each outcome r has probability 0, so will be discounted. Nicolausian discounting therefore forces you to value my gamble at the value of the status quo. That isn’t plausible. There are good bugfixes to this problem (Beckstead and Thomas 2024) which need not concern us here, other than to remember that formulating a good anti-fanatical decision theory is sometimes trickier than it appears.
The challenge for Nicolausian discounting is that it does not always look rational to ignore low-probability events. For example, consider an option which pays a nickel with probability 1-p and destroys human civilization with probability p. As Yoaav Isaacs (2016) notes, for p < ε, Nicolausian discounters ignore the low-probability harm and value this option at the value of a nickel. But I’ll wager that many of my readers would like to refuse this gamble, and that is not because they dislike free money.
Another problem is that Nicolausian discounting forces rational numbers to be assigned as the expected utilities of gambles, so long as each outcome has rational utility. That is because only finitely many outcomes will remain undiscounted, and the rational numbers are closed under multiplication and finite summation. However, it is not clear that we should always want to assign rational expected utilities to gambles of this sort.
Consider the Pasadena game (Nover and Hajek 2004). The Pasadena game is a modification of the St Petersburg gamble that we met in Part 2. In this game, you flip a fair coin until it lands heads. You then receive payoff with utility -1n-12n/n, where n is the number of flips. For example, you receive utility 2 for one flip, -2 for two flips, 8/3 for three flips, and so on. While there is no agreed-upon proposal for how the Pasadena game should be valued, one leading proposal has it that the game should be valued at its weak expectation ln(2) (Easwaran 2008).
But Nicolausian discounters cannot say this. Under Nicolausian discounting, the Pasadena game has rational expected utility. And ln(2) is not a rational number. The upshot of this discussion is that Nicolausian discounting may be a more restrictive theory than it appears.
4. Taking stock
Every decision theory has problems. A virtue of fanaticism is that fanatics are very often willing to state and defend fanatical decision theories. While these theories have problems, fanatics stress that opposing theories are not often developed, and when they are, they have problems of their own.
This post surveyed two leading anti-fanatical decision theories. The first (Section 2) works within expected utility theory, bounding utilities to achieve anti-fanaticism. We saw that bounded utilities may be plausible for typical consumer goods. However, it is much harder to justify bounded utility functions in moral outcomes such as years of human life.
The second anti-fanatical decision theory, Nicolausian discounting, ignores outcomes with sufficiently low probability in calculating expected utilities (Section 3). This has some tough consequences. For example, options that promise trivial gains against catastrophic losses can be valued at the value of the gains, if the catastrophic losses are improbable enough. Moreover, Nicolausian discounting imposes hidden restrictions on the value of gambles, such as the requirement that gambles whose outcomes have rational utility be assigned rational expected values. These requirements can be more restrictive than they appear.
Are the drawbacks of these anti-fanatical decision theories enough to outweigh their benefits? Should new anti-fanatical decision theories be developed and considered? Perhaps. But it would be nice to have a better-performing anti-fanatical bird in hand before rejecting fanaticism. At least for my own part, I cannot name any anti-fanatical decision theory that I am ready to endorse. That gives me pause in rejecting fanaticism.
Leave a Reply