It is the sense of power attached to a sense of knowledge that makes men desirous of believing, and afraid of doubting. This sense of power is the highest and best of pleasures when the belief on which it is founded is a true belief, and has been fairly earned by investigation. … But if the belief has been accepted on insufficient evidence, the pleasure is a stolen one. Not only does it deceive ourselves by giving us a sense of power which we do not really possess, but it is sinful, because it is stolen in defiance of our duty to mankind. That duty is to guard ourselves from such beliefs as from pestilence, which may shortly master our own body and then spread to the rest of the town.
William Clifford, “The ethics of belief”
1. About this series
Effective altruists use the term `epistemics’ to describe practices that shape knowledge, belief and opinion within a community. The term is understood broadly enough to encompass diverse categories such as forms of publication; discourse norms; institutional structures; attitudes towards authority; and means of reporting uncertainty.
Effective altruists care a lot about epistemics. They want to have good epistemics and learn to avoid bad epistemics. I think that in some ways, effective altruists already have good epistemics. In other ways, community epistemics could do with improvement.
This series will focus on areas in which community epistemics could be productively improved. Today, I want to introduce the series by giving three examples of the topics that I will be concerned about. Let me emphasize that these are only sketches of topics to be covered in the future: they are intended as previews, not as complete outlines of claims about community epistemics, and certainly not as full defenses of those claims.
2. Money talks
Effective altruists have at their fingertips billions (previously tens of billions) of dollars of pledged funding. They use this funding to influence the direction of research and debate. Effective altruists have established research institutes at major universities including Oxford and Cambridge, and funded lavish prizes including $100k in prizes for papers presented at a single conference symposium, and $1.5m for short arguments about the risks posed by artificial intelligence.
Money talks, and this kind of money introduces at least three types of distortions in the direction of research and discussion. First, there is an inflation of authoritativeness and seriousness in which money is used to buy markers of prestige for authors and topics that would otherwise be taken less seriously within debates. Research institutes at leading universities and purchased symposia at major machine learning conferences lend an aura of respectability to a movement often driven more strongly by blogs and forum posts than by scholarly research.
Second, philanthropic spending leads to topical biases in research: topics such as existential risk and AI safety are rocketed to the top of research agendas, not because scholars judge them to be the most serious or pursuit-worthy topics, but simply because research on these topics is more likely to be funded. Scholars willing to do work on these topics find themselves with prime, research-only positions at institutes flush with cash, and these conditions are attractive to many researchers who would otherwise prefer to study something else.
Finally, philanthropic funding leads to opinion biases in which the opinions expressed in published research are skewed towards those popular with funders. Opinion biases can be produced intentionally, as scholars seek to ingratiate themselves with funders. But opinion biases can also be produced unintentionally, as funders seek out and reward work that resonates with them.
I have said before that one thing I admire about effective altruists is their willingness to support scholarly research. Perhaps epistemic distortions are the price to pay for scholars’ full bellies. But even so, we should be honest about what foundations purchase in return for research funding, and how these purchases can shape research in directions unrelated to, or even contrary to truth or scholarly interest.
3. Publication practices
Publication practices matter. The way in which material is published affects its likely audience; how the material will be vetted and evaluated; and what it is likely to discuss. For example, this blog differs from my scholarly work in targeting a wider and more generalist audience; being largely unvetted; and allowing me to discuss a wide range of material that would not be of interest to leading academic journals.
Effective altruists often disseminate their work using a variety of nonscholarly media such as blog posts, forum discussions, podcasts, and reports commissioned and published by philanthropic foundations. While these media have advantages, such as accessibility, they also present unique epistemic challenges.
One question concerns the nature of peer review. Scholarly work is anonymously vetted by independent experts at the cutting edge of their fields, who are given the power to accept, reject, or demand changes to the research they evaluate. By contrast, work published by effective altruists is often unvetted, and even when reviews are commissioned, reviewers often lack terminal degrees, have limited publication records, are ideologically aligned with effective altruists, and are given no power to reject or alter the contents of research.
Another question concerns the evidentiary practices used by different publication media. For example, nonscholarly publications often make substantially fewer citations to other work and often cite their claims to other nonscholarly publications rather than to scholarly articles and books. It may be worth asking how these practices can serve to strengthen or weaken the collective evidential basis for claims made by effective altruists, as well as whether they might serve to promote a coalescing of opinion or a false sense of consensus driven by an overlapping network of citations to similarly-motivated authors.
A final question concerns the authorship of publications. Many materials circulated by effective altruists, including a good number of canonical texts, are written by authors lacking terminal degrees in their fields. These authors are frequently young, inexperienced, and lack substantial records of scholarly publication. Outside observers might be forgiven for asking whether these authors are qualified to be making confident claims about subject matters which often overlap with traditional research programs and whether it might be appropriate to show more deference to traditional scholarly work than to publications authored by untrained authors in less reliable formats.
4. Deference and authority
Effective altruism is, in many ways, a remarkably democratic and egalitarian movement. Anyone can post on the EA Forum and have their ideas engaged with by the community. But although anyone can speak, not everyone’s voice is treated equally. There are a number of celebrity figures within the movement whose opinions are much more likely to be read and believed than the opinions of average members.
Many of the questions to be asked about deference and authority have nothing to do with the specific figures to whom authority is granted. For example, too much trust in authority may allow false or undersupported claims to go unchallenged; may promote premature convergence of opinion; and may worsen citation practices as the opinions of authorities begin to be treated as legitimate citations in their own right.
However, at times we should also ask whether the specific individuals granted authority are deserving of our trust. For example, Eliezer Yudkowsky has a track record of bombastic and inaccurate predictions and has been known to contradict scholarly consensus on the basis of what turned out to be elementary confusions. Now, Yudkowsky is taken quite seriously when he urges that humanity has lost the battle for survival against artificial intelligence and should shift the focus of our efforts towards dying with dignity. Does such an individual, or such a claim, deserve the level of deference and respect shown within the community?
Sometimes, questions about the role of EA celebrities go beyond trust and become questions of amplification. Should the voices of leading figures be amplified, even when those doing the amplifying suspect that those voices are wrong? What kind of epistemic price is paid when the voices amplified hold views unsupported by the best available evidence?
5. Looking forward
In addition to the issues discussed above, this series will discuss many other questions about epistemics. For example:
- In what ways may biases of group deliberation such as familiar biases of self-selecting, homogenous, internet-driven groups lead to failures of group deliberation?
- May the community’s focus on probabilization and statements of credence at times serve to over-represent authors’ certainty, or to foster unwarranted convergence of opinion within the community?
- What role do canonical texts play within the community? Do these texts deserve to play the role that they have been given?
- Might views imported from Silicon Valley, such as a preference for longshots, quantification, and technology-driven solutions, be given undue weight in new areas to which they are less well-suited?
Please do let me know if there are other questions about epistemics that you would like to hear discussed.
Leave a Reply