← Return to search results
Back to Prindle Institute

machine learning

Posted on

Virtually Inhumane: Is It Wrong to Speak Cruelly to Chatbots?

photograph of middle school boy using computer

Smartphone app trends tend to be ephemeral, but one new app is making quite a few headlines. Replika, the app that promises you an AI “assistant,” gives users the option of creating all different sorts of artificially-intelligent companions. For example, a user might want an AI “friend,” or, for a mere $40 per year, they can upgrade to a “romantic partner,” a “mentor,” or a “see how it goes” relationship where anything could happen. The “friend” option is the only kind of AI the user can create and interact with for free, and this kind of relationship has strict barriers. For example, any discussions that skew toward the sexual will be immediately shut down, with users being informed that the conversation is “not available for your current relationship status.” In other words: you have to pay for that.

A recent news story concerning Replika AI chatbots discusses a disturbing trend: male app users are paying for a “romantic relationship” on Replika, and then displaying verbally and emotionally abusive behavior toward their AI partner. This behavior is further encouraged by a community of men presumably engaging in the same hobby, who gather on Reddit to post screenshots of their abusive messages and to mock the responses of the chatbot.

While the app creators find the responses of these users alarming, one thing they are not concerned about is the effect of the AI itself: “Chatbots don’t really have motives and intentions and are not autonomous or sentient. While they might give people the impression that they are human, it’s important to keep in mind that they are not.” The article’s author emphasizes, “as real as a chatbot may feel, nothing you do can actually ‘harm’ them.” Given these educated assumptions about the non-sentience of the Replika AI, are these men actually doing anything morally wrong by writing cruel and demeaning messages? If the messages are not being received by a sentient being, is this behavior akin to shouting insults into the void? And, if so, is it really that immoral?

From a Kantian perspective, the answer may seem to be: not necessarily. As the 17th century Prussian philosopher Immanuel Kant argued, we have moral duties toward rational creatures — that is, human beings, including yourself — and that their rational nature is an essential aspect of why we have duties toward them. Replika AI chatbots are, as far as we can tell, completely non-sentient. Although they may appear rational, they lack the reasoning power of human agents in that they cannot be moved to act based on reasons for or against some action. They can act only within the limits of their programming. So, it seems that, for Kant, we do not have the same duties toward artificially-intelligent agents as we do toward human agents. On the other hand, as AI become more and more advanced, the bounds of their reasoning abilities begin to escape us. This type of advanced machine learning has presented human technologists with what is now known as the “black box problem”: algorithms that have learned so much on “their own” (that is, without the direct aid of human programmers) that their code is too long and complex for humans to be able to read it. So, for some advanced AI, we cannot really say how they reason and make choices! A Kantian may, then, be inclined to argue that we should avoid saying cruel things to AI bots out of a sense of moral caution. Even if we find it unlikely that these bots are genuine agents whom we have duties toward, it is better to be safe than sorry.

But perhaps the most obvious argument against such behavior is one discussed in the article itself: “users who flex their darkest impulses on chatbots could have those worst behaviors reinforced, building unhealthy habits for relationships with actual humans.” This is a point that echoes the discussion of ethics of the ancient Greek philosopher Aristotle. In book 10 of his Nicomachean Ethics, he writes, “[T]o know what virtue is is not enough; we must endeavour to possess and to practice it, or in some other manner actually ourselves to become good.” Aristotle sees goodness and badness — for him, “virtue” and “vice” — as traits that are ingrained in us through practice. When we often act well, out of a knowledge that we are acting well, we will eventually form various virtues. On the other hand, when we frequently act badly, not attempting to be virtuous, we will quickly become “vicious.”

Consequentialists, on the other hand, will find themselves weighing some tricky questions about how to balance the predicted consequences of amusing oneself with robot abuse. While behavior that encourages or reinforces abusive tendencies is certainly a negative consequence of the app, as the article goes on to note, “being able to talk to or take one’s anger out on an unfeeling digital entity could be cathartic.” This catharsis could lead to a non-sentient chatbot taking the brunt of someone’s frustration, rather than their human partner, friend, or family member. Without the ability to vent their frustrations to AI chatbots, would-be users may choose to cultivate virtue in their human relationships — or they may exact cruelty on unsuspecting humans instead. Perhaps, then, allowing the chatbots to serve as potential punching bags is safer than betting on the self-control of the app users. Then again, one worries that users who would otherwise not be inclined toward cruelty may find themselves willing to experiment with controlling or demeaning behavior toward an agent that they believe they cannot harm.

How humans ought to engage with artificial intelligence is a new topic that we are just beginning to think seriously about. Do advanced AI have rights? Are they moral agents/moral patients? How will spending time engaging with AI affect the way we relate to other humans? Will these changes be good, or bad? Either way, as one Reddit user noted, ominously: “Some day the real AIs may dig up some of the… old histories and have opinions on how well we did.” An argument from self-preservation to avoid such virtual cruelty, at the very least.

Posted on

Correcting Bias in A.I.: Lessons from Philosophy of Science

image of screen covered in binary code

One of the major issues surrounding artificial intelligence is how to deal with bias. In October, for example, a protest was held by Uber drivers, decrying the algorithm the company uses to verify its drivers as racist. Many Black drivers were unable to verify themselves because the software fails to recognize them. Because of this, many drivers cannot get verified and are unable to work. In 2018, a study showed that a Microsoft algorithm failed to identify 1 in 5 darker-skinned females, and 1 in 17 darker-skinned males.

Instances like these prompt much strategizing about how we might stamp out bias once and for all. But can you completely eliminate bias? Is the solution to the problem a technical one? Why does bias occur in machine learning, and are there any lessons that we can pull from outside the science of AI to help us consider how to address such problems?

First, it is important to address a certain conception of science. Historically, scientists – mostly influenced by Francis Bacon – espoused the notion that science was purely about investigation into the nature of the world for its own sake in an effort to discover what the world is like from an Archimedean perspective, independent of human concerns. This is also sometimes called the “view from nowhere.” However, many philosophers who would defend the objectivity of science now accept that science is pursued according to our interests. As philosopher of science Philip Kitcher has observed, scientists don’t investigate any and all forms of true claims (many would be pointless), but rather they seek significant truth, where what counts as significant is often a function of the interests of epistemic communities of scientists.

Next, because scientific modeling is influenced by what we take to be significant, it is often influenced by assumptions we take to be significant, whether there is good evidence for them or not. As Cathy O’Neil notes in her book Weapons of Math Destruction, “a model…is nothing more than an abstract representation of some process…Whether it’s running in a computer program or in our head, the model takes what we know and uses it to predict responses to various situations.” Modeling requires that we understand the evidential relationships between inputs and predicted outputs. According to philosopher Helen Longino, evidential reasoning is driven by background assumptions because “states of affairs…do not carry labels indicating that for which they are or for which they can be taken as evidence.”

As Longino points out in her book, often these background assumptions cannot always be completely empirically confirmed, and so our values often drive what background assumptions we adopt. For example, clinical depression involves a myriad of symptoms but no single unifying biological cause has been identified. So, what justifies our grouping all of these symptoms into a single illness? According to Kristen Intemann, what allows us to infer the concept “clinical depression” from a group of symptoms are assumptions we have that these symptoms impair functions we consider essential to human flourishing, and it is only through such assumptions that we are justified in grouping symptoms with a condition like depression.

The point philosophers like Intemann and Longino are making is that such background assumptions are necessary for making predictions based off of evidence, and also that these background assumptions can be value-laden. Algorithms and models developed in AI also involve such background assumptions. One of the bigger ethical issues involving bias in AI can be found in criminal justice applications.

Recidivism models are used to help judges assess the danger posed by each convict. But people do not carry labels saying they are recidivists, so what would you take as evidence that would lead you to conclude someone might become a repeat offender? One assumption might be that if a person has had prior involvement with the police, they are more likely to be a recidivist. But if you are Black or brown in America where stop-and-frisk exists, you are already disproportionately more likely to have had prior involvement with the police, even if you have done nothing wrong. So, because of this background assumption, a recidivist model would be more likely to predict that a Black person is going to be a recidivist than a white person who is less likely to have had prior run-ins with the police.

But whether the background assumption that prior contact with the police is a good predictor of recidivism is questionable, and in the meantime this assumption creates biases in the application of the model. To further add to the problem, as O’Neil notes in her analysis of the issue, recidivism models used in sentencing involve “the unquestioned assumption…that locking away ‘high-risk’ prisoners for more time makes society safer,” adding “many poisonous assumptions are camouflaged by math and go largely untested and unquestioned.”

Many who have examined the issue of bias in AI often suggest that the solutions to such biases are technical in nature. For example, if an algorithm creates a bias based on biased data, the solution is to use more data to eliminate such bias. In other cases, attempts to technically define “fairness” are used where a researcher may require models that have equal predictive value across groups or require an equal number of false and negative positives across groups. Many corporations have also built AI frameworks and toolkits that are designed to recognize and eliminate bias. O’Neil notes how many responses to biases created by crime prediction models simply focus on gathering more data.

On the other hand, some argue that focusing on technical solutions to these problems misses the issue of how assumptions are formulated and used in modeling. It’s also not clear how well technical solutions may work in the face of new forms of bias that are discovered over time. Timnit Gebru argues that the scientific culture itself needs to change to reflect the fact that science is not pursued as a “view from nowhere.” Recognizing how seemingly innocuous assumptions can generate ethical problems will necessitate greater inclusion of people from marginalized groups.  This echoes the work of philosophers of science like Longino who assert that not only is scientific objectivity a matter of degree, but science can only be more objective by having a well-organized scientific community centered around the notion of “transformative criticism,” which requires a great diversity of input. Only through such diversity of criticism are we likely to reveal assumptions that are so widely shared and accepted that they become invisible to us. Certainly, focusing too heavily on technical solutions runs the risk of only exacerbating the current problem.

Posted on

Who Is Accountable for Inductive Risk in AI?

computer image of programming decision trees

Many people are familiar with algorithms and machine learning when it comes to applications like social media or advertising, but it can be hard to appreciate all of the diverse applications that machine learning has been applied to. For example, in addition to regulating all sorts of financial transactions, an algorithm might be used to evaluate teaching performances, or in the medical field to help identify illness or those at risk of disease. With this large array of applications comes a large array of ethical factors which become relevant as more and more real world consequences are considered. For example, machine learning has been used to train AI to detect cancer. But what happens when the algorithm is wrong? What are the ethical issues when it isn’t completely clear how the AI is making decisions and there is a very real possibility that it could be wrong?

Consider the example of applications of machine learning in order to predict whether someone charged with a crime is likely to be a recidivist. Because of massive backlogs in various court systems many have turned to such tools in order to get defendants through the court system more efficiently. Criminal risk assessment tools consider a number of details of a defendant’s profile and then produce a recidivism score. Lower scores will usually mean a more lenient sentence for committing a crime, while higher scores will usually produce harsher sentences. The reasoning is that if you can accurately predict criminal behavior, resources can be allocated more efficiently for rehabilitation or for prison sentences. Also, the thinking goes, decisions are better made based on data-driven recommendations than the personal feelings and biases that a judge may have.

But these tools have significant downsides as well. As Cathy O’Neil discusses in her book Weapons of Math Destruction, statistics show that in certain counties in the U.S. a Black person is three times more likely to get a death sentence than a white person, and so the application of computerized risk models intended to reduce prejudice, are no less prone to bias. As she notes, “The question, however, is whether we’ve eliminated human bias or simply camouflaged it with technology.” She points out that questionnaires used in some models include questions like when “the first time you ever were involved with the police” which is likely to yield very different answers depending on whether the respondent is white or Black. As she explains “if early ‘involvement’ with the police signals recidivism, poor people and racial minorities look far riskier.” So, the fact that such models are susceptible to bias also means they are not immune to error.

As mentioned, researchers have also applied machine learning in the medical field as well. Again, the benefits are not difficult to imagine. Cancer-detecting AI has been able to identify cancer that humans could not. Faster detection of a disease like lung cancer allows for quicker treatment and thus the ability to save more lives. Right now, about 70% of lung cancers are detected in late stages when it is harder to treat.

AI not only has the potential to save lives, but to also increase efficiency of medical resources as well. Unfortunately, just like the criminal justice applications, applications in the medical field are also subject to error. For example, hundreds of AI tools were developed to help deal with the COVID-19 pandemic, but a study by the Turing Institute found that AI tools had little impact. In a review of 232 algorithms for diagnosing patients, a recent medical journal paper found that none of them were fit for clinical use. Despite the hype, researchers are “concerned that [AI] could be harmful if built in the wrong way because they could miss diagnoses and underestimate the risk for vulnerable patients.”

There are lots of reasons why an algorithm designed to detect things or sort things might make errors. Machine learning requires massive amounts of data and so the ability of an algorithm to perform correctly will depend on how good the data is that it is trained with. As O’Neil has pointed out, a problematic questionnaire can lead to biased predictions. Similarly, incomplete training data can cause a model to perform poorly in real-world settings. As Koray Karaca’s recent article on inductive risk in machine learning scenarios explains, creating a model requires methodological and precise choices to be made. But these decisions are often driven by certain background assumptions – plagued by simplification and idealization – and which create problematic uncertainties. Different assumptions can create different models and thus different possibilities of error. However, there is always a gap between a finite amount of empirical evidence and an inductive generalization, meaning that there is always an inherent risk in using such models.

If an algorithm determines that I have cancer and I don’t, it could dramatically affect my life in all sorts of morally salient ways. On the other hand, if I have cancer and the algorithm says I don’t, it can likewise have a harmful moral impact on my life. So is there a moral responsibility involved and if so, who is responsible? In a 1953 article called “The Scientist Qua Scientist Makes Value Judgments” Richard Rudner argues that “since no scientific hypothesis is completely verified, in accepting a hypothesis the scientist must make the decision that evidence is sufficiently strong or that the probability is sufficiently high to warrant the acceptance of the hypothesis…How sure we need to be before we accept a hypothesis will depend on how serious a mistake would be.”

These considerations regarding the possibility of error and the threshold for sufficient evidence represent calculations of inductive risk. For example, we may judge that the consequences of asserting that a patient does not have cancer when they actually do to be far worse than the consequences of asserting that a patient does have cancer when they actually do not. Because of this, and given our susceptibility to error, we may accept a lower standard of evidence for determining that a patient has cancer but a higher standard for determining the patient does not have cancer to mitigate and minimize the worst consequences if an error occurs. But how do algorithms do this? Machine learning involves optimization of a model by testing it against sample data. Each time an error is made, a learning algorithm updates and adjusts parameters to reduce the total error which can be calculated in different ways.

Karaca notes that optimization can be carried out either in cost-sensitive or -insensitive ways. Cost-insensitive training assigns the same value to all errors, while cost-sensitive training involves assigning different weights to different errors. But the assignment of these weights is left to the modeler, meaning that the person who creates the model is responsible for making the necessary moral judgments and preference orderings of potential consequences. In addition, Karaca notes that inductive risk concerns arise for both the person making methodological choices about model construction and later for those who must decide whether to accept or reject a given model and apply it.

What this tells us is that machine learning inherently involves making moral choices and that these can bear out in evaluations of acceptable risk of error. The question of defining how “successful” the model is is tied up with our own concern about risk. But this only poses an additional question: How is there accountability in such a system? Many companies hide the results of their models or even their existence. But, as we have seen, moral accountability in the use of AI is of paramount importance. At each stage of assessment, we encounter an asymmetry in information that pits the victims of such AI to “prove” the algorithm wrong against available evidence that demonstrates how “successful” the model is.

Posted on

Ethical Considerations of Deepfakes

computer image of two identical face scans

In a recent interview for MIT Technology Review, art activist Barnaby Francis, creator of deepfake Instagram account @bill_posters_uk, mused that deepfake is “the perfect art form for these kinds of absurdist, almost surrealist times that we’re experiencing.” Francis’ use of deepfakes to mimic celebrities and political leaders on Instagram is aimed at raising awareness about the danger of deepfakes and the fact that “there’s a lot of people getting onto the bandwagon who are not really ethically or morally bothered about who their clients are, where this may appear, and in what form.” While deepfake technology has received alarmist media attention in the past few years, Francis is correct in his assertion that there are many researchers, businesses, and academics who are pining for the development of more realistic deepfakes.

Is deepfake technology ethical? If not, what makes it wrong? And who holds the responsibility to prevent the potential harms generated by deepfakes: developers or regulators?

Deepfakes are not new. The first mention of deepfake was by a reddit user in 2017, who began using the technology to create pornographic videos. However, the technology soon expanded to video games as a way to create images of people within a virtual universe. However, the deepfake trend suddenly turned toward more global agendas, with fake images and videos of public figures and political leaders being distributed en masse. One altered video of Joe Biden was so convincing that even President Trump fell for it. Last year, there was a deepfake video of Mark Zuckerberg talking about how happy he was to have thousands of people’s data. At the time, Facebook maintained that deepfake videos would stay up, as they did not violate their terms of agreement. Deepfakes have only increased since then. In fact, there exists an entire YouTube playlist with deepfake videos dedicated to President Trump.

In 2020, those who have contributed to deepfake technology are not only individuals in the far corners of the internet. Researchers at the University of Washington have also developed deepfakes using algorithms in order to combat their spread. Deepfake technology has been used to bring art to life, recreate the voices of historical figures, and to use celebrities’ likeness to communicate powerful public health messages. While the dangers of deepfakes have been described by some as dystopian, the methods behind their creation have been relatively transparent and accessible.

One problem with deepfakes are that they mimic a person’s likeness without their permission. The original Deepfakes, which used photos or videos of a person mixed with pornography uses a person’s likeness for sexual gratification. Such use of a person’s likeness might never personally affect them, but could still be considered wrong, since they are being used as a source of pleasure and entertainment, without consent. These examples might seem far-fetched, but in 2019 a now-defunct app called DeepNude, sought to do exactly that. Even worse than using someone’s likeness without their knowledge, is if the use of their likeness is intended to reach them and others, in order to humiliate or damage their reputation. One could see the possibility of a type of deepfake revenge-porn, where scorned partners attempt to humiliate their exes by creating deepfake pornography. This issue is incredibly pressing and might be more prevalent than the other potential harms of deepfakes. One study, for example, found that 96% of existing deepfakes take the form of pornography.

Despite this current reality, much of the moral concern over deepfakes is grounded in their potential to easily spread misinformation. Criticism around deepfakes in recent years has been mainly surrounding their potential for manipulating the public to achieve political ends. It is becoming increasingly easy to spread a fake video depicting a politician who is clearly incompetent or spreading a questionable message, which might detract from their base. On a more local level, deepfakes could be used to discredit individuals. One could imagine a world in which deepfakes are used to frame someone in order to damage their reputation, or even to suggest they have committed a crime. Video and photo evidence is commonly used in our civil and criminal justice system, and the ability to manipulate videos or images of a person, undetected, arguably poses a grave danger to a justice system which relies on our sense of sight and observation to establish objective fact. Perhaps even worse than framing the innocent could be failing to convict the guilty. In fact, a recent study in the journal Crime Science found that deepfakes pose a serious crime threat when it comes to audio and video impersonation and blackmail. What if a deepfake is used to replace a bad actor with a person who does not exist? Or gives plausible deniability to someone who claims that a video or image of them has been altered?

Deepfakes are also inherently dishonest. Two of the most popular social media networks, Instagram and TikTok, inherently rely upon visual media which could be subject to alteration by self-imposed deepfakes. Even if a person’s likeness is being manipulated with their consent and also could have positive consequences, it still might be considered wrong due to the dishonest nature of its content. Instagram in particular has been increasingly flooded with photoshopped images, as there is an entire app market that exists solely for editing photos of oneself, usually to appear more attractive. The morality of editing one’s photos has been hotly contested amongst users and between feminists. Deepfakes only stand to increase the amount of media that is self-edited and the moral debates that come along with putting altered media of oneself on the internet.

Proponents of deepfakes argue that their positive potential far outweighs the negative. Deepfake technology has been used to spark engagement with the arts and culture, and even to bring historical figures back to life, both for educational and entertainment purposes. Deepfakes also hold the potential to integrate AI into our lives in a more humanizing and personal manner. Others, who are aware of the possible negative consequences of deepfakes, argue that the development and research of this technology should not be impeded, as the advancement of the technology can also contribute to research methods of spotting it. And there is some evidence backing up this argument, as the development of deepfake progresses, so do the methods for detecting it. It is not the moral responsibility of those researching deepfake technology to stop, but rather the role of policymakers to ensure the types of harmful consequences mentioned above do not wreak havoc on the public. At the same time, proponents such as David Greene, of the Electronic Frontier Foundation, argue that too stringent limits on deepfake research and technology will “implicate the First Amendment.”

Perhaps then it is not the government nor deepfake creators who are responsible for their harmful consequences, but rather the platforms which make these consequences possible. Proponents might argue that the power of deepfakes is not necessarily from their ability to deceive one individual, but rather the media platforms on which they are allowed to spread. In an interview with Digital Trends, the creator of Ctrl Shift Face (a popular deepfake YouTube channel), contended that “If there ever will be a harmful deepfake, Facebook is the place where it will spread.” While this shift in responsibility might be appealing, detractors might ask how practical it truly is. Even websites that have tried to regulate deepfakes are having trouble doing so. Popular pornography website, PornHub, has banned deepfake videos, but still cannot fully regulate them. In 2019, a deepfake video of Ariana Grande was watched 9 million times before it was taken down.

In December, the first federal regulation pertaining to deepfakes passed through the House, the Senate, and was signed into law by President Trump. While increased government intervention to prevent the negative consequences of deepfakes will be celebrated by some, researchers and creators will undoubtedly push back on these efforts. Deepfakes are certainly not going anywhere for now, but it remains to be seen if the potentially responsible actors will work to ensure their consequences remain net-positive.

Posted on

Establishing Liability in Artificial Intelligence

Entrepreneur Li Kin-kan is suing over “investment losses triggered by autonomous machines.” Raffaele Costa convinced Li to let K1, a machine learning algorithm, manage $2.5 billion—$250 million of his own cash and the rest leverage from Citigroup Inc. The AI lost a significant amount of money in a decision that the company claims it wouldn’t have made if it was as sophisticated as they had been led to believe. Because of the autonomous decision-making structure of K1, trying to locate appropriate liability is a provocative question: is the money-losing decision the fault of K1, its designers, Li, or, as Li alleges, the salesman who made claims about K1’s potential?  

Developed by Austria-based AI company 42.cx, the supercomputer named K1 would “comb through online sources like real-time news and social media to gauge investor sentiment and make predictions on U.S. stock futures. It would then send instructions to a broker to execute trades, adjusting its strategy over time based on what it had learned.”

Our current laws are designed to assign responsibility on the basis of intention or ability to predict an injury. Algorithms do neither, but are being put to more and more tasks that can produce legal injuries in novel ways. In 2014, the Los Angeles Times published an article that carried the byline: “this post was created by an algorithm written by the author.” The author of the algorithm, Ken Schwencke, allowed the code to produce a story covering and earthquake, not an uncommon event around LA, so tasking an algorithm to produce the news was a time-saving strategy. However, journalism by code can lead to complicated libel suits, as legal theorists discussed when Stephen Colbert used an algorithm to match Fox News personalities with movie reviews from Rotten Tomatoes. Though the claims produced were satire, there could have been a case for libel or defamation, though without a human agent as the direct producer of the claim: “The law would then face a choice between holding someone accountable for a result she did not specifically intend, or permitting without recourse what most any observer would take for defamatory or libelous speech.”

Smart cars are being developed that can cause physical harm and injury based on the decisions of their machine learning algorithms. Further, artificial speech apps are behaving in unanticipated ways: “A Chinese app developer pulled its instant messaging “chatbots”—designed to mimic human conversation—after the bots unexpectedly started criticizing communism. Facebook chatbots began developing a whole new language to communicate with each other—one their creators could not understand.”

Consider: machine-learning algorithms accomplish tasks in ways that cannot be anticipated in advance (indeed, that’s why they are implemented – to do creative, not purely scripted work); and thus they increasingly blur the line between person and instrument, for the designer did not explicitly program how the task will be performed.

When someone directly causes injury, for instance by causing bodily harm with their body, it is easy to isolate them as the cause. If someone stomps on your foot, this could cause a harm. According to the law, then, they can be held liable if they have the appropriate mens rea, or guilty mind. For instance, if they intended to cause that injury, knowingly caused the injury, recklessly caused the injury, or negligently caused the injury.

This structure for liability seems to work just as well if the person in question used a tool or instrument. If someone uses a sledgehammer to break your foot, they still are isolated as the cause (as the person moving the sledgehammer around), and can be held liable depending on what their mental state was regarding the sledgehammer-hitting-your-foot (perhaps it was a non-culpable accident). Even if they use a  complicated Rube Goldberg Machine to break your foot, the same structure seems to work just fine. If someone uses a foot-breaking Rube Goldberg Machine to break your foot, they’ve caused you an injury, and depending on their particular mens rea will be liable for some particular legal violation.

Machine learning algorithms put pressure on this framework, however, because when they are used it is not to produce a specific result in the way the Rube Goldberg foot-breaking machine does. The Rube Goldberg foot-breaking machine, though complex, is transparent and has an outcome that is “designed in”: it will smash feet. With machine learning algorithms, there is a break between the designer or user and the product. The outcome is not specifically intended in the way smashing feet is intended by a user of the Rube Goldberg machine. Indeed, it is not even known by the user of the algorithm.

The behavior or choice in cases of machine learning algorithms originate in the artificial intelligence in a way that foot smashing doesn’t originate in the Rube Goldberg machine. Consider: we wouldn’t hold the Rube Goldberg machine liable for a broken foot, but would rather look to the operator or designer.  However, in cases of machine learning, the user or designer didn’t come up with the output of the algorithm.

When Deepmind won at Go, it was making choices that surprised all of the computer scientists involved. AI make complex decisions and take actions completely unforeseen by their creators, so when their decisions result in injury, where do we look to apportion blame? It is still the case that you cannot sue algorithms or AI (and, further, the remuneration or punishment would be difficult to imagine).  

One model for AI liability interprets machine learning functions in terms of existing product liability frameworks that put burdens of appropriate operation on the producers. The assumptions here are that any harm resulting by products is due to faulty products and the company is liable regardless of mens rea (See, for instance, Escola v Coca-Cola Bottling Co.). In this framework, the companies that produce the algorithms would be liable for harms that result from smart cars or financial decisions.

Were this framework adopted, Li could be suing the AI company that produced or sold K1, 42.cx, but as it stands, the promises involved in the sale conform to our current legal standards. The interpretations at stake are whether K1 could have been predicted to make the decision that resulted in losses given the description in the terms of sale.

Posted on

Racist, Sexist Robots: Prejudice in AI

Black and white photograph of two robots with computer displays

The stereotype of robots and artificial intelligence in science fiction is largely of a hyper-rational being, unafflicted by the emotions and social infirmities like biases and prejudices that impair us weak humans. However, there is reason to revise this picture. The more progress we make with AI the more a particular problem comes to the fore: the algorithms keep reflecting parts of our worst selves back to us.

In 2017, research showed compelling evidence that AI picks up deeply ingrained racial- and gender-based prejudices. Current machine learning techniques rely on algorithms interacting with people in order to better predict correct responses over time. Because of the dependence on interacting with humans for standards of correctness, the algorithms cannot detect when bias informs a correct response or when the human is engaging in a non-prejudicial way. Thus, the best working AI algorithms pick up the racist and sexist underpinnings of our society. Some examples: the words “female” and “woman” were more closely associated with arts and humanities occupations and with the home, while “male” and “man” were closer to maths and engineering professions. Europeans were associated with pleasantness and excellence.

In order to prevent discrimination in housing, credit, and employment, Facebook has recently been forced to agree to an overhaul of its ad-targeting algorithms. The functions that determined how to target audiences for ads relating to these areas turned out to be racially discriminatory, not by design – the designers of the algorithms certainly didn’t encode racial prejudices – but because of the way they are implemented. The associations learned by the ad-targeting algorithms led to disparities in the advertising of major life resources. It is not enough to program a “neutral” machine learning algorithm (i.e., one that doesn’t begin with biases). As Facebook learned, the AI must have anti-discrimination parameters built in as well. Characterizing just what this amounts to will be an ongoing conversation. For now, the ad-targeting algorithms cannot take age, zip code, or gender into consideration, as well as legally protected categories.

The issue facing AI is similar to the “wrong kind of reasons” problem in philosophy of action. The AI can’t tell a systemic bias of humans from a reasoned consensus: both make us converge on an answer and support the algorithm to select what we may converge on. It is difficult to say what, in principle, the difference is between the systemic bias and a reasoned consensus is. It is difficult, in other words, to give the machine learning instrument parameters to tell when there is the “right kind of reason” supporting a response and the “wrong kind of reason” supporting the response.

In philosophy of action, the difficulty of drawing this distinction is illustrated by a case where, for instance, you are offered $50,000 to (sincerely) believe that grass is red. You have a reason to believe, but intuitively this is the wrong kind of reason. Similarly, we could imagine a case where you will be punished unless you (sincerely) desire to eat glass. The offer of money doesn’t show that “grass is red” is true, similarly the threat doesn’t show that eating glass is choice-worthy. But each somehow promote the belief or desire. For the AI, a racist or sexist bias leads to a reliable response in the way that the offer and threat promote a behavior – it is disconnected from a “good” response, but it’s the answer to go with.

For International Women’s Day, Jeanette Winterson suggested that artificial intelligence may have a significantly detrimental effect on women. Women make up 18% of computer science graduates and thus are left out of the design and directing of this new horizon of human development. This exclusion can exacerbate the prejudices that can be inherent in the design of these crucial algorithms that will become more critical to more arenas of life.