Suppose that a highly educated person in the year 1500 tried to make the longterm future go well. They would be aware of some relevant things, such as the persistence of laws, religions, and political institutions. But many issues wouldn’t occur to them. The ideas that the Earth’s habitable life span could be a billion years and that the universe could be so utterly enormous, yet almost entirely uninhabited, would not have been on the table. Crucial conceptual tools for dealing with uncertainty, such as probability theory and expected value, had not yet been developed. They would not have been exposed to the arguments for a moral worldview in which the interests of all people are equal. They wouldn’t have known what they didn’t know.
Will MacAskill, What we owe the future
1. Introduction
This is Part 4 of my series “The scope of longtermism,” discussing a paper of the same name.
Part 1 introduced the scope question for longtermism: how wide is the class of decision situations for which longtermism holds? I clarified my target, swamping axiological strong longtermism (swamping ASL) and my view: the scope of swamping ASL, while probably nonempty, is smaller than many longtermists suppose.
Specifically, I introduce three scope-limiting factors: probabilistic and decision-theoretic factors that are present in many contemporary decision problems and which, when present, substantially reduce the prospects for swamping ASL to hold of that problem.
Part 2 introduced the first scope-limiting factor: rapid diminution. Part 3 introduced the second scope-limiting factor: washing out. Today’s post introduces the final scope-limiting factor: unawareness.
2. Unawareness
Suppose you were a strangely benevolent Egyptian pharaoh. More specifically, you wanted to do what you could to make the twenty-first century C.E. go as well as possible. What might you do?
Let us focus, for illustration, on some of the leading challenges facing humanity in this century. The climate is warming, relations are cooling between the United States and China, and we are running dangerously short of key minerals such as lithium.
One challenge that might arise is that it is hard to see how you could make much of an impact on any of these problems. You could, for example, warn future generations about the dangers of global warming, but if warnings by leading scientists were ignored, what chance do you have with hieroglyphs? It is not clear what you could do to improve relations between the United States and China, or to increase the availability of lithium. These problems were discussed in Part 2 under the heading of rapid diminution.
Another challenge that might arise is that your actions could have negative consequences for the future. For example, you might well have thought that future generations would appreciate the preservation of such elements of Egyptian culture as corporal punishment and chattel slavery. You might have done what you could to preserve these institutions. If you managed this, you would have done future generations great harm. These problems were discussed in Part 3 under the heading of washing out.
A final challenge is that you are not even aware of many ways that the future could be, or many acts that you could take to improve it. You have no doubt never heard of lithium, the United States, or China, have never considered that greenhouse gas emissions from fossil fuels could lead to global warming, and are likely unaware that some of the fuels you burn are quite carbon-intensive. There might, perhaps, be some strange and yet-unconsidered act you might take to meaningfully improve on global challenges facing the twenty-first century, yet once this act is specified it is likely so strange and convoluted that you have not considered it. This is the problem of unawareness, and the subject of today’s post.
3. Why unawareness matters
We saw in Part 1 that this paper and series focus on ex ante versions of longtermism. The ex ante perspective is meant to capture the normative relevance of a decisionmaker’s limited epistemic status. For example, from an ex post perspective we might say that you should bet the house on a two-seven offsuit while playing poker if in fact, and unbeknownst to you, this hand will go on to win. However, from an ex ante perspective we would say that given the information available to you, it is unlikely that you will win your bet, and so you should fold.
However, your epistemic status is not just limited by the information you have. It is also limited by your awareness. Becoming aware of new theories, concepts, or world-states is an epistemic achievement — indeed, it is often a significant achievement, as in the postulation of new entities in fundamental physics. Insofar as the ex ante perspective aims to reflect your limited epistemic status, it should therefore reflect your current state of awareness and not the awareness you have yet to achieve.
If we did not incorporate unawareness into the ex ante perspective, the results would be about as strange and counterintuitive as the claim that you should bet on a two-seven offsuit. Suppose, for example, that it is possible to use current technology to construct a cutting-edge spaceship, although this fact will not be discovered for many centuries. Suppose further that I am a capable mechanic and that, if I thought hard enough about the actions needed to construct this spaceship, I would see that they would produce a cutting-edge spaceship. If my current unawareness of these facts were not relevant to ex ante decisionmaking, we would get the result that I ought currently to build the yet-to-be-discovered spaceship. That is quite a strange thing to say.
These are some of the reasons why many theorists have thought that ex ante decisionmaking should be sensitive to unawareness.
4. How unawareness of actions matters
To see how unawareness matters, we need to distinguish two types of unawareness. First, we are unaware of actions that we could take. Second, we are unaware of states that the world could be in and outcomes that could result from our actions.
Begin with unawareness of actions. It is generally thought that oughts apply to options. Decisionmakers have a range of options available to them. Options are things they can do now. What decisionmakers ought to do is to take the best available option.
Not everything is an option for me. Some things fail to be options because they are beyond my physical abilities. I cannot, for example, lift a thousand-pound weight, so that is not an option for me. Other things fail to be options because I am not aware that I could do them. It is not an option for me now to construct the spaceship described in Section 4, because I am not aware of the possibility of doing so. Unawareness, then, functions to remove options from my choice-set.
One of the strongest parts of the case for longtermism is the reminder that it just takes one good longtermist intervention to bear out longtermism. Even if problems such as rapid diminution and washing out render the vast majority of longtermist interventions inviable, there are millions of possible longtermist interventions that could be attempted now. If but a single one of these interventions is viable and can be seen to be so based on current evidence, then longtermism holds of our present decision situation.
However, decisionmakers with restricted awareness typically do not consider millions of options. They are probably considering at most a few dozen longtermist options. While it might be surprising for problems such as rapid diminution and washing out to make all possible longtermist options inviable, it would not be so surprising for them to render an arbitrarily chosen two-dozen options inviable. Hence, the fact that it only takes one option to vindicate longtermism does not take the longtermist nearly so far as it did before awareness restrictions were considered.
Longtermists can and do object that it may be correct for decisionmakers who are not currently aware of viable longtermist interventions to take meta-options such as searching out and evaluating further longtermist options. This is certainly the right thing for longtermists to say. But we need to be honest about how far this reply will take us. There is no possibility of awareness-restricted agents considering and evaluating any meaningful fraction of the space of possible longtermist interventions. As such, even these meta-interventions can only do so much to boost the number of available longtermist options. Even if a few viable longtermist interventions are possible, it is not altogether surprising to think that agents would be highly unlikely to find and identify them.
5. How unawareness of states and outcomes matters
We saw in the last section that unawareness of actions works to rule out possible options for a decisionmaker. Unawareness of states and outcomes functions differently. How, exactly, unawareness of states and outcomes should be represented is an active area of research, but here is one common proposal.
On this proposal, agents use a catch–all state to stand in for any states that they might be unaware of. They assign probabilities to all other states, then give the remaining probability mass to the catch-all.
Similarly, agents use a catch-all outcome to stand in for any outcomes they might be unaware of. They then assign a probability and value to that outcome.
There are many things that might be said about decisionmaking under state and outcome unawareness. But I think one minimal lesson is a counsel to hold our models lightly. It is not so strange to maximize expected value if the probabilities and utilities reflect careful consideration of relevant possibilities. But how do you assign a value to the set of outcomes that you are not even aware of? Not, I take it, by considering what those outcomes are like and how good or bad it would be to encounter them. And how do you assign a probability to the set of states that you are not even aware of? Not, again, by considering what those states are like and how likely they are to arise.
Even many economists, decision-theorists, and other researchers who recommend expected utility maximization under better conditions do not think that it is always prudent to maximize expected utility over models constructed under such poor conditions.
What should we do instead? That is, unfortunately, a very long and complicated question. But the short answer is that on many competing approaches, the case for longtermism is substantially less obvious that it would be under expected value theory.
6. Conclusion
So far, we have discussed three scope-limiting factors which serve to limit the scope of (swamping, axiological strong) longtermism.
Part 2 discussed the problem of rapid diminution. Affecting the future is hard, so the probability of large far-future impacts often decays faster than their values, leading the largest impacts to be of marginal importance to ex-ante decisionmaking.
Part 3 discussed the problem of washing out. Forecasting the future effects of our actions is hard, and we have very little evidence to go on, so we should be substantially uncertain about the effects that our actions might have. This uncertainty leads to significant cancellation between possible positive and negative effects from the ex ante perspective.
Today’s post discussed a final scope-limiting factor: unawareness. We are unaware of many things we could do to improve the future, many outcomes that our actions could have, and many world states relevant to determining what those outcomes will be. We saw that unawareness of acts worsens the case for longtermism by restricting the number of options that might vindicate longtermism in any given decision problem, whereas unawareness of states and outcomes worsens the case for longtermism by putting pressure against the models such as expected value models on which longtermism typically performs best.
How do these three scope-limiting factors get a take on the case for longtermism in practice? The next post in this series takes up that question, focusing on the value of existential risk mitigation efforts.
Leave a Reply