A Rebuttal to Pedro Domingos’ Rebuttal to my Remarks Opposed to Signing the Open Letter on Academic “Free Speech”
I appreciate that Pedro took the time to write an extensive response to my initial post taking a position against signing the open letter that is currently circulating. Now it is my turn. Before getting to his specific rebuttal points, I offer some context.
I am assuming good faith among the signers; I do not believe their intent is to harm. I do not say this to patronize, but because I understand that some see simply signing the letter as justification to attack the signers, and I disagree. Indeed, I am not attacking the signers’ views at all because — due to the ambiguity of the letter — I do not know what their views are. My focus is on the text of the letter, not the views of the signers.
I am generally a supporter of free speech, even when it makes people uncomfortable. I often find myself disagreeing with “cancellations”, and defending my position against friends arguing for them. I consider the letter and Pedro’s rebuttal to be reasonable debate; there is nothing in them that I find out-of-bounds for discussion. In fact, I can easily imagine versions of the open letter that I would have signed. However, I oppose signing this particular letter because of its ambiguity and absolutism, as I’ve already explained in my previous post.
I have the easier side of this argument: my goal is to show that the letter can be misused — that there exist plausible interpretations of the letter that people should not support. The writers on the other hand need to demonstrate that all plausible interpretations of the letter are acceptable to a potential signer. And unlike the letter-writers, since I am not asking people to sign anything, but only to refrain from signing, the only harm in my posts being ambiguous is that they may fail to convince. But Pedro’s rebttal, and other commentary I’ve seen, shows a misunderstanding of my objections, so I’ll try to clarify them here. I aim to point out natural interpretations of the letter that I reject, and which provide reasons not to sign on. As before, I’ll tackle one bullet at a time, but I’ll try to provide more specific examples of the letter’s problems.
The Bullets
- Scientific work should be judged on the basis of scientific merit, independent of the researcher’s identity or personal views.
In my first speculative interpretation of bullet 1 as addressing scholarship submitted for publication, I argued that peer review is expressly designed to provide independent judgment of such work. Pedro rejected this argument by discussing “attacks for what they publish after it’s published, people being fired, ostracized, antagonized, and more.” So, it seems that his intended meaning of the ambiguous point here is the broader interpretation I explored immediately after, of judgment of the scholars themselves.
So let’s consider this broader point. Pedro and I may find ourselves in partial agreement here. I will readily agree with Pedro that there are cases where academics have faced excessive retaliation when their scientific work is inconvenient. Perhaps like me Pedro will sign the Standing with Dr. Timnit Gebru petition which argues that denying publication of her research and firing her was not an appropriate response to the disagreement about her scientific findings.
But while we might agree that Timnit’s firing (and some others that have been in the news) was excessive, Pedro seems to argue that every response is excessive. This is a fallacy of overgeneralization: the existence of overreactions does not mean that every reaction is an overreaction. There are cases where even “meritorious science” should have consequences for the scientist.
Consider for example the infamous Tuskegee Syphilis Study. This study asked the scientifically “meritorious” question of how syphilis affects African Americans. The experimenters deceived the subjects and left them untreated or mistreated, and many died of the disease. We now recognize the severe moral violations in this study; they informed the Belmont Report which guides many institutional review boards (IRBs) overseeing the use of humans as experimental subjects. Research that violates the Belmont principles is forbidden by our institutions no matter how meritorious the scientific questions are. And a scientist who violates these principles should be judged for doing so.
So what exactly does the open letter mean when it speaks of “judging on scientific merit”? Do they mean that the Tuskegee experiment was actually justified because it gathered scientifically useful data? Or do they mean that IRBs should be abolished, with the decision about appropriate ethical boundaries left to the scientists? Or do they consider it obvious that ethical review is part of “scientific merit”? And if the last, then what exactly are the bounds of “scientific merit” and what is being excluded? These interpretations are widely varying, but all consistent with bullet 1. Which is a signatory supporting?
The AMA’s Code of Medical Ethics also addresses this issue. Like me, and unlike the letter, it recognizes that there is no black and white answer, but instead that complex considerations must be weighed against each other in deciding how scientific work should be judged.
Pedro goes on to suggest that there is a contradiction between my objections to bullet 1 and my argument that we must avoid racism and sexism in judging scientific work. But this is another fallacious overgeneralization: while I agree that it is not valid to judge scientific work based on the researcher’s race and gender, there is no contradiction in simultaneously arguing that it is appropriate to include other considerations, such as ethical concerns, in such judgments.
Pedro then addresses the Nature Communications work on mentorship by women, and objects that it is unfair that this work was retracted (by the authors) only because it got so much attention which led to the identification of scientific flaws. If Pedro has some technique for ensuring that all scientific work receives equal scrutiny, I’d love to know it because I think many of my papers haven’t gotten the attention they deserved. I’m not sure what remedy Pedro is advocating here (and it certainly isn’t clear from bullet 1). Should we limit the number of people permitted to view a given article? Or should people not be permitted to question the science in papers they read if they disagree with the conclusions? Should authors not be permitted to retract? Pedro also claims this is an example where “the science is attacked because of the scientist” but goes on to remark that another paper by the same authors was not attacked, refuting his own claim. Overall, I couldn’t find a clear argument to rebut here.
Pedro then disagrees with my argument that it will sometimes be appropriate to exclude scientists from a conference even if we accept their work to be published there. I described the specific example of banning someone who has publicly stated that certain races or genders are inferior. Pedro asserts this is “letting objections to the person’s views affect the reception of the scientific work.” To make this concrete, let’s consider Ludwig Bieberbach, a talented mathematician and a Nazi who made the racial inferiority argument that “the spatial imagination is a characteristic of the Germanic races, while pure logical reasoning has a richer development among Romanic and Hebraic races” and “was enthusiastically involved in the efforts to dismiss his Jewish colleagues.” (Yes, I know this will generate piles of “David says Pedro Domingos is a Nazi” quotes from people who can’t read, but I think it is a useful test case.) He was dismissed from his position in 1945 because of his Nazi work, but was invited to lecture in 1949 by others who “considered Bieberbach’s political views irrelevant to his contributions to the field of mathematics”. So you can see that the disagreement between me and Pedro isn’t new. Bieberbach’s work is of course cited, and there’s a conjecture named after him. But, if he were alive today, would we want him at our conferences? I would say no. The letter, at least in Pedro’s interpretation, seems to say yes — so I cannot sign it.
Pedro finally argues that “the whole point of Principle 1 is to separate judgments of the science from judgments of the scientist.” Can we really do this? Long ago I served on various STOC and FOCS program committees. In those days the page limits meant that a lot of math got left out of the proof “sketches” that were submitted. At one of our committee meetings, one of our greatest researchers told me they were opposed to double-blind review because “you need to know who wrote the paper in order to decide whether you trust the proof sketches”. I disagreed with this particular argument, but it continues to be the case that a great deal of assessment of scientific work relies at a minimum on our trust in the author. When I read a paper describing execution of code that yielded certain data, I don’t run the code and I don’t compare its output to the data claimed. I trust the author. If this author had been convicted of research misconduct in the past, I would demand a higher standard of proof that they did the work they claimed. This would certainly be a case of judging science dependent on the scientist, and I think it would be appropriate.
2. Discussion and debate in the scientific community must be free of prior restraint as to topic or viewpoint.
I turn now to bullet 2 on prior restraint. Pedro claims “Karger says that this principle is vacuously satisfied” and that this shows a “shocking naivete.” He must have failed to notice my “taken literally” qualifier. My “shocking” statement was proposed only to point out that since nobody is literally prevented from publishing on the internet, the letter writers must mean something more than this. I then proceeded to argue against these broader interpretations.
In particular, I observed that conferences have always exercised prior restraint on which topics they will accept. Pedro appears to accept this point. But at the same time he objects to the restraint imposed by the NeurIPS ethics review. I’m not sure why Pedro says “Karger claims that (for example) NeurIPS’s requirement of broader impact statements and ethics reviews is not really a restraint” when in fact I described it as “a pretty clear prior restraint.” But overall Pedro seems to be saying that topic constraints are valid but that ethics review is an invalid restraint. This is inconsistent with the absolutist message of the letter. Bullet 2 does not say that discussions must be free of invalid restraints (which everyone would agree with, but would push off the entire debate into which restraints are invalid); instead it levels a blanket objection to any restraint. This would include the restraints on topic that we all expect. So how can anyone signing this letter understand exactly what kinds of restraints are covered by the principle? The letter writers might have a specific set of restraints in mind, but they remain hidden there.
3. No individual should suffer harassment or attack based on their personal or political views, religion, nationality, race, gender, or sexual orientation.
Finally we come to bullet 3 on attacks. Again we agree about a good part of this, but we disagree about the inclusion of personal or political views as grounds. Here, Pedro raises a valid objection to something I worded poorly. In my post, I explained that “If someone declares that a certain group is inferior, I want that “personal or political view” to be “attacked”. Or let’s consider another example: someone at a conference expressing their “personal view” that a particular scientist is sexy. I stand by the assertion that such views should lead to attack. But I then argued that these attacks “should not be ad hominem — -they should attack the individual on the grounds of the views they hold.” This was poorly worded; the attack should not be on the grounds of the views they hold, but on the grounds of their expression of those views.
What is the difference? Well, once you go from merely holding views to expressing them, that expression impacts those around you. Looking to my specific examples, I have heard from many researchers (especially students), of the psychological toll it takes to be told that you are inferior or unwanted or sexy. It burdens every interaction with concern about how one is being perceived and a pressure to be perfect. It discourages scientists from doing their best work. It drives people from the field. It is bad for science. Most important, it is hurting someone. Indeed, I would say these kinds of assertions are perfect examples of the attacks that bullet 3 claims should be off limits. I therefore believe it is entirely justified to respond to such attacks. Of course we can split hairs over whether and when such responses are themselves attacks. Is telling someone their comment was sexist an attack? What if it actually was sexist? Is telling them they are sexist an attack? Is refusing to have dinner with them an attack? Even if such actions are declared to be attacks, I think there are situations in which they are appropriate responses, and again the letter offers no guidance on the scope of “attack” that it rejects.
Balancing Harms
Pedro finishes by addressing my argument that we need a better balance between false positives and false negatives on accusations of racism and sexism. Here Pedro agrees with my high-level point but disagrees on the direction in which we need to shift. Pedro claims that we are tilted strongly in the direction of (imagining) “racism and sexism everywhere”. I have the opposite perspective: while I have seen a few eminent researchers like Pedro, and a few students, called out as racist or sexist for their expressed views, it seems that just about every female or minority student or researcher I have spoken to can tell me about their personal experiences of sexism and racism that have gone unaddressed in our field. Indeed, I suspect it is the rarity of call-outs that has made them newsworthy enough to be collected as anecdotal evidence of a “cancel culture”, while the experiences of those facing racism and sexism are so common that they go unremarked. To me it appears that our unwillingness to allow even the minimum of false positives has led to an astronomical cost in false negatives.
Pedro’s main anecdote here is from his own experience of being called a racist, sexist, misogynist bigot. This is unpleasant; I know because I too have been called these things. Not as often as Pedro, but enough to know that I’d like to avoid the experience. I’ve even modified my behavior a little bit to reduce the chances of it happening again; I think of it as “reflection and self-improvement.” But in the end, Pedro and I are pretty well situated to handle these kinds of attacks: we have prestige, strong social ties in academia, money, and tenure. False positives cost us relatively little. Contrast this with a smart, talented female undergraduate who dropped my class this semester because she faced sexism from the male students she needed as collaborators. For her the cost was very high. If we at the apex can reduce harm to our vulnerable members at the cost of a degree of discomfort for ourselves, deserved or not, that’s a trade worth making.
Conclusion
In sum, I cannot support a letter that takes such a black-and-white view on principles while simultaneously presenting significant ambiguity on exactly what those principles are. Pedro asks “where I draw the line.” As a scientist I get it; we like to draw lines and define absolute boundaries. But while our science may be precise, we do that science in a real world that is much more complicated. The relative costs of tolerating or rejecting certain positions, speech or behaviors depend on the speaker, the audience, the topic, and many other factors. The value of freedom in speech and scientific inquiry is high, sufficient to outweigh many other costs, but it is not infinite. And when we accept the costs that come with free inquiry, we have a responsibility to spread those costs fairly over our community, instead of letting them fall entirely on our most vulnerable members.
Lest some argue that my arguments reject signing any statement of principles, here’s an example of something I could sign: the World Wide Web Consortium’s code of conduct. It covers some of the same ground as the current letter — for example, like bullet 3, it rejects attacks based on “gender, gender identity and gender expression, sexual orientation, disability (both visible and invisible), mental health, neurotype, physical appearance, body, age, race, socio-economic status, ethnicity, caste, nationality, language, or religion.” Notably, parallel to my arguments above, it does not include “personal or political views” in this list. But it does expect members to “be inclusive and promote diversity” and asserts that “diversity of views and of people powers innovation, even if it is not always comfortable.” This matches a core theme of the open letter. But crucially, instead of laying out diversity of views as an inviolable principle, the code recognizes that it is a value that can be in tension with and overridden by others, and that such tensions ought to be resolved with empathy, in favor of those who would be most harmed. Unlike the current letter, the code’s precision and detail give me a better understanding of what I would be supporting. I’d be happy to sign a letter committing to the principles of this code.
