Claude vs. The Pentagon: How Much Should Being Ethical Cost?

Until recently, Anthropic AI’s Large Language Model “Claude” was one of the most widely used in the US military. This status was tested in February when Anthropic refused to allow the military to use Claude for autonomous weapons or domestic mass surveillance. Anthropic has always defined itself as being ethics and AI safety conscious. This time, standing their ground cost them. The Pentagon responded by labeling Anthropic a “supply chain risk,” endangering Claude’s existing contracts with the Department of Defense and other defense contractors. Anthropic asserted this was mere retaliation — no US-based company has ever been labelled a supply chain risk — and sought a legal order to halt the designation. A previous judgment affirmed this stance. However, in bad news for Anthropic, a panel of the DC Circuit Court recently declined to stop the supply chain risk designation while the dispute continues.

The ethical choice is not always the easy choice. People have lost money, jobs, and even their lives to maintain their principles. Anthropic, while their conviction is laudable, is perhaps not the ultimate example of ethical sacrifice. Part of the reason Claude had so much presence in the military is their partnership with Palantir, a major player behind AI-facilitated surveillance. Moreover, as a multi-billion dollar company, Anthropic is able to far better weather the consequences of saying no to the US military than most.

Nonetheless, it invites the more general question: What price should we be willing to pay to do the right thing?

The answer is often surprisingly high. Many philosophers consider ethics to have an “overriding“ character. Not only is ethics a reason for action (or to withhold from action), but it is the most important reason, overriding all non-ethical ones. There might be competing moral reasons: While I know stealing is wrong, if I don‘t steal, then I can‘t feed my family. Contrast this with an unjustified alternative: Stealing is wrong unless one finds a legal loophole.

Central to most ethical approaches are that other lives matter. This is true for Immanuel Kant, who stressed that we must treat each other as fellow rational beings with their own ends, and not merely as means for our ends. It is equally true for many consequence-based frameworks (those that evaluate moral conduct based on what has the best consequences), which value all wellbeing equally.

The combination of overridingness and concern for others, means that being ethical can be quite demanding. However, there is a reason we are impressed by those who stand by their principles in the face of threats and intimidation. Take for instance the German families that hid Jewish children. It is not easy. We value our lives, we value our stuff. Even in less extreme cases, being ethical can be challenging. Confronting a close friend about their drug use may damage the friendship, but it may nonetheless be the right thing to do as a true friend.

In practice, if we care about ethical behavior, it is in our interest to make behaving ethically as convenient as possible. Nations have practical reasons to prevent the harmful effects of stealing, violence, and corruption. We, in fact, expect a well-run society to stop such things. One thing the law can do is align incentives, such that right conduct is favored and wrong conduct punished. Of course, we may prefer people not to steal because it’s unethical, and not because it’s against the law, but you can’t always get what you want.

This relates to an effective system of justice and accountability, but also to the opportunity cost of ethical behavior. Consider two vastly different stories we can tell about cheating. In both cases, cheating to get ahead is morally wrong. In the first story, one is also very likely to get caught and punished, negating any benefit of cheating. In the second, punishment is unlikely, and those who don’t cheat should be prepared to fall behind those who do. In fact, they should probably expect one of those cheaters to be their boss.

Sometimes, sacrifice is required; it may take courage to do the right thing. But what does it mean if sacrifice is required over, and over, and over again? What does it mean if being ethical looks like a fool’s game, a way to set oneself up for nothing but struggle and failure? It doesn’t mean people should stop being ethical. It is, however, an indication the incentive structure of the society is badly misaligned.

Whatever one thinks about the specific ethics of Anthropic, the DC Circuit Panel had an opportunity to signal that if one stands by their principles, the law will protect them from retaliation. They chose not to. What message does this send to those of us who are not billion dollar companies?