← Return to search results
Back to Prindle Institute

artificial intelligence

Posted on

AI and Pure Science

Pixelated image of a man's head and shoulders made up of pink and purple squares

In September 2019, four researchers wrote to the academic publisher Wiley to request that it retract a scientific paper relating to facial recognition technology. The request was made not because the research was wrong or reflected bad methodology, but rather because of how the technology was likely to be used. The paper discussed the process by which algorithms were trained to detect faces of Uyghur people, a Muslim minority group in China. While researchers believed publishing the paper presented an ethical problem, Wiley defended the article noting that it was about a specific technology, not about the application of that technology. This event raises a number of important questions, but, in particular, it demands that we consider whether there is an ethical boundary between pure science and applied science when it comes to AI development – that is, whether we can so cleanly separate knowledge from use as Wiley suggested.

The 2019 article for the journal WIREs Data Mining and Knowledge Discovery discusses discoveries made by the research term in the work on ethic group facial recognition which included datasets of Chinese Uyghur, Tibetan, and Korean students at Dalian University. In response a number of researchers, believing that it is disturbing that academics tried to build such algorithms, called for the article to be retracted. China has been condemned for its heavy surveillance and mass detention of Uyghurs, and this study and a number of other studies, some scientists claim, are helping to facilitate the development of technology which can make this surveillance and oppression more effective. As Richard Van Noorden reports, there has been a growing push by some scientists to get the scientific community to take a firmer stance against unethical facial-recognition research, not only denouncing controversial uses of technology, but the research foundations of it as well. They call on researchers to avoid working with firms or universities linked to unethical projects.

For its part, Wiley has defended the article, noting “We are aware of the persecution of the Uyghur communities … However, this article is about a specific technology and not an application of that technology.” In other words, Wiley seems to be adopting an ethical position based on the long-held distinction between pure and applied science. This distinction is old, tracing back to the time of Francis Bacon and the 16th century as part of a compromise between the state and scientists. As Robert Proctor reports, “the founders of the first scientific societies promised to ignore moral concerns” in return for funding and for freedom of inquiry in return for science keeping out of political and religious matters. In keeping with Bacon’s urging that we pursue science “for its own sake,” many began to distinguish science as “pure” affair, interested in knowledge and truth by themselves, and applied science which seeks to use engineering to apply science in order to secure various social goods.

In the 20th century the division between pure and applied science was used as a rallying cry for scientific freedom and to avoid “politicizing science.” This took place against a historical backdrop of chemists facilitating great suffering in World War I followed by physicists facilitating much more suffering in World War II. Maintaining the political neutrality of science was thought to make it more objective by ensuring value-freedom. The notion that science requires freedom was touted by well-known physicists like Percy Bridgman who argued,

The challenge to the understanding of nature is a challenge to the utmost capacity in us. In accepting the challenge, man can dare to accept no handicaps. That is the reason that scientific freedom is essential and that artificial limitations of tools or subject matter are unthinkable.

For Bridgman, science just wasn’t science unless it was pure. He explains, “Popular usage lumps under the single world ‘science’ all the technological activities of engineering and industrial development, together with those of so-called ‘pure science.’ It would clarify matters to reserve the word science for ‘pure’ science.” For Bridgman it is society that must decide how to use a discovery rather than the discoverer, and thus it is society’s responsibility to determining how to use pure science rather than the scientists’. As such, Wiley’s argument seems to echo those of Bridgman. There is nothing wrong with developing the technology of facial recognition in and of itself; if China wishes to use that technology to oppress people with it, that’s China’s problem.

On the other hand, many have argued that the supposed distinction between pure and applied science is not ethically sustainable. Indeed, many such arguments were driven by the reaction to the proliferation of science during the war. Janet Kourany, for example, has argued that science and scientists have moral responsibilities because of the harms that science has caused, because science is supported through taxes and consumer spending, and because society is shaped by science. Heather Douglas has argued that scientists shoulder the same moral responsibilities as the rest of us not to engage in reckless or negligent research, and that due to the highly technical nature of the field, it is not reasonable for the rest of society to carry those responsibilities for scientists. While the kind of pure knowledge that Bridgman or Bacon favor has value, these values need to be weighed against other goods like basic human rights, quality of life, and environmental health.

In other words, the distinction between pure and applied science is ethically problematic. As John Dewey argues the distinction is a sham because science is always connected to human concerns. He notes,

It is an incident of human history, and a rather appalling incident, that applied science has been so largely made equivalent for use for private and economic class purposes and privileges. When inquiry is narrowed by such motivation or interest, the consequence is in so far disastrous both to science and to human life.

Perhaps this is why many scientists do not accept Wiley’s argument for refusing retraction; discovery doesn’t happen in a vacuum. It isn’t as if we don’t know why the Chinese government has an interest in this technology. So, at what point does such research become morally reckless given the very likely consequences?

This is also why debate around this case has centered on the issue of informed consent. Critics charge that the Uyghur students who participated in the study were not likely fully informed of its purposes and this could not provide truly informed consent. The fact that informed consent is relevant at all, which Wiley admits, seems to undermine their entire argument as informed consent in this case appears explicitly tied to how the technology will be used. If informed consent is ethically required, this is not a case where we can simply consider pure research with no regard to its application. And these considerations prompted scientists like Yves Moreau to argue that all unethical biometric research should be retracted.

But regardless of how we think about these specifics, this case serves to highlight a much larger issue: given the large number of ethical issues associated with AI and its potential uses we need to dedicate much more of our time and attention to the question of whether some certain forms of research should be considered forbidden knowledge. Do AI scientists and developers have moral responsibilities for their work? Is it more important to develop this research for its own sake or are there other ethical goods that should take precedence?

Posted on

Virtually Inhumane: Is It Wrong to Speak Cruelly to Chatbots?

photograph of middle school boy using computer

Smartphone app trends tend to be ephemeral, but one new app is making quite a few headlines. Replika, the app that promises you an AI “assistant,” gives users the option of creating all different sorts of artificially-intelligent companions. For example, a user might want an AI “friend,” or, for a mere $40 per year, they can upgrade to a “romantic partner,” a “mentor,” or a “see how it goes” relationship where anything could happen. The “friend” option is the only kind of AI the user can create and interact with for free, and this kind of relationship has strict barriers. For example, any discussions that skew toward the sexual will be immediately shut down, with users being informed that the conversation is “not available for your current relationship status.” In other words: you have to pay for that.

A recent news story concerning Replika AI chatbots discusses a disturbing trend: male app users are paying for a “romantic relationship” on Replika, and then displaying verbally and emotionally abusive behavior toward their AI partner. This behavior is further encouraged by a community of men presumably engaging in the same hobby, who gather on Reddit to post screenshots of their abusive messages and to mock the responses of the chatbot.

While the app creators find the responses of these users alarming, one thing they are not concerned about is the effect of the AI itself: “Chatbots don’t really have motives and intentions and are not autonomous or sentient. While they might give people the impression that they are human, it’s important to keep in mind that they are not.” The article’s author emphasizes, “as real as a chatbot may feel, nothing you do can actually ‘harm’ them.” Given these educated assumptions about the non-sentience of the Replika AI, are these men actually doing anything morally wrong by writing cruel and demeaning messages? If the messages are not being received by a sentient being, is this behavior akin to shouting insults into the void? And, if so, is it really that immoral?

From a Kantian perspective, the answer may seem to be: not necessarily. As the 17th century Prussian philosopher Immanuel Kant argued, we have moral duties toward rational creatures — that is, human beings, including yourself — and that their rational nature is an essential aspect of why we have duties toward them. Replika AI chatbots are, as far as we can tell, completely non-sentient. Although they may appear rational, they lack the reasoning power of human agents in that they cannot be moved to act based on reasons for or against some action. They can act only within the limits of their programming. So, it seems that, for Kant, we do not have the same duties toward artificially-intelligent agents as we do toward human agents. On the other hand, as AI become more and more advanced, the bounds of their reasoning abilities begin to escape us. This type of advanced machine learning has presented human technologists with what is now known as the “black box problem”: algorithms that have learned so much on “their own” (that is, without the direct aid of human programmers) that their code is too long and complex for humans to be able to read it. So, for some advanced AI, we cannot really say how they reason and make choices! A Kantian may, then, be inclined to argue that we should avoid saying cruel things to AI bots out of a sense of moral caution. Even if we find it unlikely that these bots are genuine agents whom we have duties toward, it is better to be safe than sorry.

But perhaps the most obvious argument against such behavior is one discussed in the article itself: “users who flex their darkest impulses on chatbots could have those worst behaviors reinforced, building unhealthy habits for relationships with actual humans.” This is a point that echoes the discussion of ethics of the ancient Greek philosopher Aristotle. In book 10 of his Nicomachean Ethics, he writes, “[T]o know what virtue is is not enough; we must endeavour to possess and to practice it, or in some other manner actually ourselves to become good.” Aristotle sees goodness and badness — for him, “virtue” and “vice” — as traits that are ingrained in us through practice. When we often act well, out of a knowledge that we are acting well, we will eventually form various virtues. On the other hand, when we frequently act badly, not attempting to be virtuous, we will quickly become “vicious.”

Consequentialists, on the other hand, will find themselves weighing some tricky questions about how to balance the predicted consequences of amusing oneself with robot abuse. While behavior that encourages or reinforces abusive tendencies is certainly a negative consequence of the app, as the article goes on to note, “being able to talk to or take one’s anger out on an unfeeling digital entity could be cathartic.” This catharsis could lead to a non-sentient chatbot taking the brunt of someone’s frustration, rather than their human partner, friend, or family member. Without the ability to vent their frustrations to AI chatbots, would-be users may choose to cultivate virtue in their human relationships — or they may exact cruelty on unsuspecting humans instead. Perhaps, then, allowing the chatbots to serve as potential punching bags is safer than betting on the self-control of the app users. Then again, one worries that users who would otherwise not be inclined toward cruelty may find themselves willing to experiment with controlling or demeaning behavior toward an agent that they believe they cannot harm.

How humans ought to engage with artificial intelligence is a new topic that we are just beginning to think seriously about. Do advanced AI have rights? Are they moral agents/moral patients? How will spending time engaging with AI affect the way we relate to other humans? Will these changes be good, or bad? Either way, as one Reddit user noted, ominously: “Some day the real AIs may dig up some of the… old histories and have opinions on how well we did.” An argument from self-preservation to avoid such virtual cruelty, at the very least.

Posted on

Correcting Bias in A.I.: Lessons from Philosophy of Science

image of screen covered in binary code

One of the major issues surrounding artificial intelligence is how to deal with bias. In October, for example, a protest was held by Uber drivers, decrying the algorithm the company uses to verify its drivers as racist. Many Black drivers were unable to verify themselves because the software fails to recognize them. Because of this, many drivers cannot get verified and are unable to work. In 2018, a study showed that a Microsoft algorithm failed to identify 1 in 5 darker-skinned females, and 1 in 17 darker-skinned males.

Instances like these prompt much strategizing about how we might stamp out bias once and for all. But can you completely eliminate bias? Is the solution to the problem a technical one? Why does bias occur in machine learning, and are there any lessons that we can pull from outside the science of AI to help us consider how to address such problems?

First, it is important to address a certain conception of science. Historically, scientists – mostly influenced by Francis Bacon – espoused the notion that science was purely about investigation into the nature of the world for its own sake in an effort to discover what the world is like from an Archimedean perspective, independent of human concerns. This is also sometimes called the “view from nowhere.” However, many philosophers who would defend the objectivity of science now accept that science is pursued according to our interests. As philosopher of science Philip Kitcher has observed, scientists don’t investigate any and all forms of true claims (many would be pointless), but rather they seek significant truth, where what counts as significant is often a function of the interests of epistemic communities of scientists.

Next, because scientific modeling is influenced by what we take to be significant, it is often influenced by assumptions we take to be significant, whether there is good evidence for them or not. As Cathy O’Neil notes in her book Weapons of Math Destruction, “a model…is nothing more than an abstract representation of some process…Whether it’s running in a computer program or in our head, the model takes what we know and uses it to predict responses to various situations.” Modeling requires that we understand the evidential relationships between inputs and predicted outputs. According to philosopher Helen Longino, evidential reasoning is driven by background assumptions because “states of affairs…do not carry labels indicating that for which they are or for which they can be taken as evidence.”

As Longino points out in her book, often these background assumptions cannot always be completely empirically confirmed, and so our values often drive what background assumptions we adopt. For example, clinical depression involves a myriad of symptoms but no single unifying biological cause has been identified. So, what justifies our grouping all of these symptoms into a single illness? According to Kristen Intemann, what allows us to infer the concept “clinical depression” from a group of symptoms are assumptions we have that these symptoms impair functions we consider essential to human flourishing, and it is only through such assumptions that we are justified in grouping symptoms with a condition like depression.

The point philosophers like Intemann and Longino are making is that such background assumptions are necessary for making predictions based off of evidence, and also that these background assumptions can be value-laden. Algorithms and models developed in AI also involve such background assumptions. One of the bigger ethical issues involving bias in AI can be found in criminal justice applications.

Recidivism models are used to help judges assess the danger posed by each convict. But people do not carry labels saying they are recidivists, so what would you take as evidence that would lead you to conclude someone might become a repeat offender? One assumption might be that if a person has had prior involvement with the police, they are more likely to be a recidivist. But if you are Black or brown in America where stop-and-frisk exists, you are already disproportionately more likely to have had prior involvement with the police, even if you have done nothing wrong. So, because of this background assumption, a recidivist model would be more likely to predict that a Black person is going to be a recidivist than a white person who is less likely to have had prior run-ins with the police.

But whether the background assumption that prior contact with the police is a good predictor of recidivism is questionable, and in the meantime this assumption creates biases in the application of the model. To further add to the problem, as O’Neil notes in her analysis of the issue, recidivism models used in sentencing involve “the unquestioned assumption…that locking away ‘high-risk’ prisoners for more time makes society safer,” adding “many poisonous assumptions are camouflaged by math and go largely untested and unquestioned.”

Many who have examined the issue of bias in AI often suggest that the solutions to such biases are technical in nature. For example, if an algorithm creates a bias based on biased data, the solution is to use more data to eliminate such bias. In other cases, attempts to technically define “fairness” are used where a researcher may require models that have equal predictive value across groups or require an equal number of false and negative positives across groups. Many corporations have also built AI frameworks and toolkits that are designed to recognize and eliminate bias. O’Neil notes how many responses to biases created by crime prediction models simply focus on gathering more data.

On the other hand, some argue that focusing on technical solutions to these problems misses the issue of how assumptions are formulated and used in modeling. It’s also not clear how well technical solutions may work in the face of new forms of bias that are discovered over time. Timnit Gebru argues that the scientific culture itself needs to change to reflect the fact that science is not pursued as a “view from nowhere.” Recognizing how seemingly innocuous assumptions can generate ethical problems will necessitate greater inclusion of people from marginalized groups.  This echoes the work of philosophers of science like Longino who assert that not only is scientific objectivity a matter of degree, but science can only be more objective by having a well-organized scientific community centered around the notion of “transformative criticism,” which requires a great diversity of input. Only through such diversity of criticism are we likely to reveal assumptions that are so widely shared and accepted that they become invisible to us. Certainly, focusing too heavily on technical solutions runs the risk of only exacerbating the current problem.

Posted on

Who Is Accountable for Inductive Risk in AI?

computer image of programming decision trees

Many people are familiar with algorithms and machine learning when it comes to applications like social media or advertising, but it can be hard to appreciate all of the diverse applications that machine learning has been applied to. For example, in addition to regulating all sorts of financial transactions, an algorithm might be used to evaluate teaching performances, or in the medical field to help identify illness or those at risk of disease. With this large array of applications comes a large array of ethical factors which become relevant as more and more real world consequences are considered. For example, machine learning has been used to train AI to detect cancer. But what happens when the algorithm is wrong? What are the ethical issues when it isn’t completely clear how the AI is making decisions and there is a very real possibility that it could be wrong?

Consider the example of applications of machine learning in order to predict whether someone charged with a crime is likely to be a recidivist. Because of massive backlogs in various court systems many have turned to such tools in order to get defendants through the court system more efficiently. Criminal risk assessment tools consider a number of details of a defendant’s profile and then produce a recidivism score. Lower scores will usually mean a more lenient sentence for committing a crime, while higher scores will usually produce harsher sentences. The reasoning is that if you can accurately predict criminal behavior, resources can be allocated more efficiently for rehabilitation or for prison sentences. Also, the thinking goes, decisions are better made based on data-driven recommendations than the personal feelings and biases that a judge may have.

But these tools have significant downsides as well. As Cathy O’Neil discusses in her book Weapons of Math Destruction, statistics show that in certain counties in the U.S. a Black person is three times more likely to get a death sentence than a white person, and so the application of computerized risk models intended to reduce prejudice, are no less prone to bias. As she notes, “The question, however, is whether we’ve eliminated human bias or simply camouflaged it with technology.” She points out that questionnaires used in some models include questions like when “the first time you ever were involved with the police” which is likely to yield very different answers depending on whether the respondent is white or Black. As she explains “if early ‘involvement’ with the police signals recidivism, poor people and racial minorities look far riskier.” So, the fact that such models are susceptible to bias also means they are not immune to error.

As mentioned, researchers have also applied machine learning in the medical field as well. Again, the benefits are not difficult to imagine. Cancer-detecting AI has been able to identify cancer that humans could not. Faster detection of a disease like lung cancer allows for quicker treatment and thus the ability to save more lives. Right now, about 70% of lung cancers are detected in late stages when it is harder to treat.

AI not only has the potential to save lives, but to also increase efficiency of medical resources as well. Unfortunately, just like the criminal justice applications, applications in the medical field are also subject to error. For example, hundreds of AI tools were developed to help deal with the COVID-19 pandemic, but a study by the Turing Institute found that AI tools had little impact. In a review of 232 algorithms for diagnosing patients, a recent medical journal paper found that none of them were fit for clinical use. Despite the hype, researchers are “concerned that [AI] could be harmful if built in the wrong way because they could miss diagnoses and underestimate the risk for vulnerable patients.”

There are lots of reasons why an algorithm designed to detect things or sort things might make errors. Machine learning requires massive amounts of data and so the ability of an algorithm to perform correctly will depend on how good the data is that it is trained with. As O’Neil has pointed out, a problematic questionnaire can lead to biased predictions. Similarly, incomplete training data can cause a model to perform poorly in real-world settings. As Koray Karaca’s recent article on inductive risk in machine learning scenarios explains, creating a model requires methodological and precise choices to be made. But these decisions are often driven by certain background assumptions – plagued by simplification and idealization – and which create problematic uncertainties. Different assumptions can create different models and thus different possibilities of error. However, there is always a gap between a finite amount of empirical evidence and an inductive generalization, meaning that there is always an inherent risk in using such models.

If an algorithm determines that I have cancer and I don’t, it could dramatically affect my life in all sorts of morally salient ways. On the other hand, if I have cancer and the algorithm says I don’t, it can likewise have a harmful moral impact on my life. So is there a moral responsibility involved and if so, who is responsible? In a 1953 article called “The Scientist Qua Scientist Makes Value Judgments” Richard Rudner argues that “since no scientific hypothesis is completely verified, in accepting a hypothesis the scientist must make the decision that evidence is sufficiently strong or that the probability is sufficiently high to warrant the acceptance of the hypothesis…How sure we need to be before we accept a hypothesis will depend on how serious a mistake would be.”

These considerations regarding the possibility of error and the threshold for sufficient evidence represent calculations of inductive risk. For example, we may judge that the consequences of asserting that a patient does not have cancer when they actually do to be far worse than the consequences of asserting that a patient does have cancer when they actually do not. Because of this, and given our susceptibility to error, we may accept a lower standard of evidence for determining that a patient has cancer but a higher standard for determining the patient does not have cancer to mitigate and minimize the worst consequences if an error occurs. But how do algorithms do this? Machine learning involves optimization of a model by testing it against sample data. Each time an error is made, a learning algorithm updates and adjusts parameters to reduce the total error which can be calculated in different ways.

Karaca notes that optimization can be carried out either in cost-sensitive or -insensitive ways. Cost-insensitive training assigns the same value to all errors, while cost-sensitive training involves assigning different weights to different errors. But the assignment of these weights is left to the modeler, meaning that the person who creates the model is responsible for making the necessary moral judgments and preference orderings of potential consequences. In addition, Karaca notes that inductive risk concerns arise for both the person making methodological choices about model construction and later for those who must decide whether to accept or reject a given model and apply it.

What this tells us is that machine learning inherently involves making moral choices and that these can bear out in evaluations of acceptable risk of error. The question of defining how “successful” the model is is tied up with our own concern about risk. But this only poses an additional question: How is there accountability in such a system? Many companies hide the results of their models or even their existence. But, as we have seen, moral accountability in the use of AI is of paramount importance. At each stage of assessment, we encounter an asymmetry in information that pits the victims of such AI to “prove” the algorithm wrong against available evidence that demonstrates how “successful” the model is.

Posted on

The Insufficiency of Black Box AI

image of black box spotlighted and on pedestal

Google and Imperial College London have collaborated in a trial of an AI system for diagnosing breast cancer. Their most recent results have shown that the AI system can outperform the uncorroborated diagnosis of a single trained doctor and perform on par with pairs of trained diagnosticians. The AI system was a deep learning model, meaning that it works by discovering patterns on its own by being trained on a huge database. In this case the database was thousands of mammogram images. Similar systems are used in the context of law enforcement and the justice system. In these cases the learning database is past police records. Despite the promise of this kind of system, there is a problem: there is not a readily available explanation of what pattern the systems are relying on to reach their conclusions. That is, the AI doesn’t provide reasons for its conclusions and so the experts relying on these systems can’t either.

AI systems that do not provide reasons in support of their conclusions are known as “black box” AI. In contrast to these are so-called “explainable AI.” This kind of AI system is under development and likely to be rapidly adopted within the healthcare field. Why is this so? Imagine visiting the doctor and receiving a cancer diagnosis. When you ask the doctor, “Why do you think I have cancer?” they reply only with a blank stare or reply, “I just know.” Would you find this satisfying or reassuring? Probably not, because you have been provided neither reason nor explanation. A diagnosis is not just a conclusion about a patient’s health but also the facts that lead up to that conclusion. There are certain reasons that the doctor might give you that you would reject as reasons that can support a cancer diagnosis.

For example an AI designed at Stanford University system being trained to help diagnosis tuberculosis used non-medical evidence to generate its conclusions. Rather than just taking into account the images of patients’ lungs, the system used information about the type of X-ray scanning device when generating diagnoses. But why is this a problem? If the information about what type of X-ray machine was used has a strong correlation with whether a patient  has tuberculosis shouldn’t that information be put to use? That is, don’t doctors and patients want to maximize the number of correct diagnoses they make? Imagine your doctor telling you, “I am diagnosing you with tuberculosis because I scanned you with Machine X, and people who are scanned by Machine X are more likely to have tuberculosis.” You would not likely find this a satisfying reason for a diagnosis. So if an AI is making diagnoses based on such facts this is a cause for concern.

A similar problem is discussed in philosophy of law when considering whether it is acceptable to convict people on the basis of statistical evidence. The thought experiment used to probe this problem involves a prison yard riot. There are 100 prisoners in the yard, and 99 of them riot by attacking the guard. One of the prisoners did not attack the guard, and was not involved in planning the riot. However there is no way of knowing specifically of each prisoner whether they did, or did not, participate in the riot. All that is known that 99 of the 100 prisoners participated. The question is whether it is acceptable to convict each prisoner based only on the fact that it is 99% likely that they participated in the riot.

Many who have addressed this problem answer in the negative — it is not appropriate to convict an inmate merely on the basis of statistical evidence. (However, David Papineau has recently argued that it is appropriate to convict on the basis of such strong statistical evidence.) One way to understand why it may be inappropriate to convict on the basis of statistical evidence alone, no matter how strong, is to consider the difference between circumstantial and direct evidence. Direct evidence is any evidence which immediately shows that someone committed a crime. For example, if you see Robert punch Willem in the face you have direct evidence that Robert committed battery (i.e., causing harm through touch that was not consented to). If you had instead walked into the room to see Willem holding his face in pain and Robert angrily rubbing his knuckles, you would only have circumstantial evidence that Robert committed battery. You must infer that battery occurred from what you actually witnessed.

Here’s the same point put another way. Given that you saw Robert punch Willem in the face, there is a 100% chance that Robert battered Willem — hence it is direct evidence. On the other hand, given that you saw Willem holding his face in pain and Robert angrily rubbing his knuckles, there is a 0- 99% chance that Robert battered Willem. The same applies to any prisoner in the yard during the riot: given that they were in the yard during the riot, there is at best a 99% chance that the prisoner attacked the guard. The fact that a prisoner was in the yard at the time of the riot is a single piece of circumstantial evidence in favor of the conclusion that that prisoner attacked the guard. A single piece of circumstantial evidence is not usually taken to be sufficient to convict someone — further corroborating evidence is required.

The same point could be made about diagnoses. Even if 99% of people examined by Machine X have tuberculosis, simply being examined by Machine X is not a sufficient reason to conclude that someone has tuberculosis. Not reasonable doctor would make a diagnosis on such a flimsy basis, and no reasonable court would convict someone on the flimsy basis in the prison yard riot case above. Black box AI algorithms might not be basing diagnoses or decisions about law enforcement on such a flimsy basis. But because this sort of AI system doesn’t provide its reasons, there is no way to tell what makes its accurate conclusions correct, or its inaccurate conclusions incorrect. Any domain like law or medicine where the reasons that underlie a conclusion are crucially important is a domain in which explainable AI is a necessity, and in which black box AI must not be used.

Posted on

In Search of an AI Research Code of Conduct

image of divided brain; fluid on one side, curcuitry on the other

The evolution of an entire industry devoted to artificial intelligence has presented a need to develop ethical codes of conduct. Ethical concerns about privacy, transparency, and the political and social effects of AI abound. But a recent study from the University of Oxford suggests that borrowing from other fields like medical ethics to refine an AI code of conduct is problematic. The development of an AI ethics means that we must be prepared to address and predict ethical problems and concerns that are entirely new, and this makes it a significant ethical project. How we should proceed in this field is itself a dilemma. Should we proceed in a top-down principled approach or a bottom up experimental approach?

AI ethics can concern itself with everything from the development of intelligent robots to machine learning, predictive analytics, and the algorithms behind social media websites. This is why it is such an expansive area with some focusing on the ethics of how we should treat artificial intelligence, others focusing on how we can protect privacy, or some on how the AI behind social media platforms and AI capable of generating and distributing ‘fake news’ can influence the political process. In response many have focused on generating a particular set of principles to guide AI researchers; in many cases borrowing from codes governing other fields, like medical ethics.

The four core principles of medical ethics are respect for patient autonomy, beneficence, non-maleficence, and justice. Essentially these principles hold that one should act in the best interests of a patient while avoiding harms and ensuring fair distribution of medical services. But the recent Oxford study by Brent Mittelstadt argues that the analogical reasoning relating the medical field to the AI field is flawed. There are significant differences between medicine and AI research which makes these principles not helpful or irrelevant.

The field of medicine is more centrally focused on promoting health and has a long history of focusing on the fiduciary duties of those in the profession towards patients. Alternatively, AI research is less homogeneous, with different researchers in both the public and private sector working on different goals and who have duties to different bodies. AI developers, for instance, do not commit to public service in the same way that a doctor does, as they may only responsible to shareholders. As the study notes, “The fundamental aims of developers, users, and affected parties do not necessarily align.”

In her book Towards a Code of Ethics for Artificial Intelligence Paula Boddington highlights some of the challenges of establishing a code of ethics for the field. For instance, those working with AI are not required to receive accreditation from any professional body. In fact,

“some self-taught, technically competent person, or a few members of a small scale start up, could be sitting in their mother’s basement right now dreaming up all sorts of powerful AI…Combatting any ethical problems with such ‘wild’ AI is one of the major challenges.”

Additionally, there are mixed attitudes towards AI and its future potential. Boddington notes a divide in opinion: the West is more alarmist as compared to nations like Japan and Korea which are more likely to be open and accepting.

Given these challenges, some have questioned whether an abstract ethical code is the best response. High-level principles which are abstract enough to cover the entire field will be too vague to be action-guiding, and because of the various different fields and interests, oversight will be difficult. According to Edd Gent,

“AI systems are…created by large interdisciplinary teams in multiple stages of development and deployment, which makes tracking the ethical implications of an individual’s decisions almost impossible, hampering our ability to create standards to guide those choices.”

The situation is not that different from work done in the sciences. Philosopher of science Heather Douglas has argued, for instance, that while ethical codes and ethical review boards can be helpful, constant oversight is impractical, and that only scientists can fully appreciate the potential implications of their work. The same could be true of AI researchers. A code of principles of ethics will not replace ethical decision-making; in fact, such codes can be morally problematic. As Boddington argues, “The very idea of parceling ethics into a formal ‘code’ can be dangerous.” This is because many ethical problems are going to be new and unique so ethical choice cannot be a matter of mere compliance. Following ethical codes can lead to complacency as one seeks to check certain boxes and avoid certain penalties without taking the time to critically examine what may be new and unprecedented ethical issues.

What this suggests is that any code of ethics can only be suggestive; they offer abstract principles that can guide AI researchers, but ultimately the researchers themselves will have to make individual ethical judgments. Thus, part of the moral project of developing an AI ethics is going to be the development of good moral judgment by those in the field. Philosopher John Dewey noted this relationship between principles and individual judgment, arguing:

“Principles exist as hypotheses with which to experiment…There is a long record of past experimentation in conduct, and there are cumulative, verifications which give many principles a well earned prestige…But social situations alter; and it is also foolish not to observe how old principles actually work under new conditions, and not to modify them so that they will be more effectual instruments in judging new cases.”

This may mirror the thinking of Brent Mittelstadt who argues for a bottom-up approach to AI ethics that focuses on sub-fields developing ethical principles as a response to resolving challenging novel cases. Boddington, for instance, notes the importance of equipping researchers and professionals with the ethical skills to make nuanced decisions in context; they must be able to make contextualized interpretations of rules, and to judge when rules are no longer appropriate. Still, such an approach has its challenges as researchers must be aware of the ethical implications of their work, and there still needs to be some oversight.

Part of the solution to this is public input. We as a public need to make sure that corporations, researchers, and governments are aware of the public’s ethical concerns. Boddington recommends that in such input there be a diversity of opinion, thinking style, and experience. This includes not only those who may be affected by AI, but also professional experts outside of the AI field like lawyers, economists, social scientists, and even those who have no interest in the world of AI in order maintain an outside perspective.

Codes of ethics in AI research will continue to develop. The dilemma we face as a society is what such a code should mean, particularly whether it will be institutionalized and enforced or not. If we adopt a bottom up approach, then such codes will likely be only there for guidance or will require the adoption of multiple codes for different areas. If a more principled top-down approach is adopted, then there will be additional challenges of dealing with the novel and with oversight. Either way, the public will have a role to play to ensure that its concerns are being heard.

Posted on

Do Self-Driving Cars Reinforce Socioeconomic Inequality?

A photo of the steering wheel of a Mercedes car.

Recently, Mercedes-Benz stepped into the spotlight after making a bold statement concerning the design of their self-driving cars. The development of autonomous cars has presented a plethora of moral conundrums, one of which is the most ethical way to program cars to respond to emergencies. The dilemma, as presented in a previous article, is one of trying to determine the value of and prioritize human life. Mercedes has declared that they will “program its self-driving cars to save the people inside the car. Every time.” This declaration sheds light on a new issue: is it ethical for car companies to create technology that widens the gap between socioeconomic classes and threatens current societal values?

Continue reading “Do Self-Driving Cars Reinforce Socioeconomic Inequality?”

Posted on

Waymo and the Morality of Self-Driving Cars

An image of a Waymo self-driving car.

What was once fiction is becoming a reality. In past decades, sci-fi novels and television have featured self-driving cars; this once-futuristic concept is finally coming to fruition. Will the result mirror the positive outcomes shown in fiction? Self-driving cars are intended to increase safety and efficiency in our society, but what are the moral implications and consequences that could come from such technology?

Continue reading “Waymo and the Morality of Self-Driving Cars”

Posted on

Sex in the Age of Sex Robots

Editor’s note: sources linked in this article contain images and videos that some readers may find disturbing.

From self-driving cars to smartphones, artificial intelligence has certainly made its way into our everyday lives. So have questions of robotic ethics. Shows like Westworld and Black Mirror have depicted some of the more controversial and abstract dangers of artificial intelligence. Human sex dolls have always been taboo, but a new development in the technology of these sex dolls, specifically their upgrade to robot status, is especially controversial. The whole notion of buying a robot to have sex with is taboo to say the least, but can these sexual acts become unethical, even if they are perpetrated upon a nonliving thing? Is using a sex robot to simulate rape or pedophilia morally permissible? And to what extent should sex robots be regulated?

Continue reading “Sex in the Age of Sex Robots”

Posted on

The Artificial Intelligence of Google’s AlphaGo

Last week, Google’s AlphaGo program beat Ke Jie, the Go world champion. The victory is a significant one, due to the special difficulties of developing an algorithm that can tackle the ancient Chinese game. It differs significantly from the feat of DeepBlue, the computer that beat then-chess world champion Garry Kasparov in 1997, largely by brute force calculations of the possible moves on the 8×8 board. The possible moves in Go far eclipse those of chess, and for decades most researchers didn’t consider it possible for a computer to defeat a champion-level Go player, because designing a computer with such complexity would amount to such great leaps towards creative intuition on the computer’s part.

Continue reading “The Artificial Intelligence of Google’s AlphaGo”

Posted on

Will Robots Ever Deserve Moral and Legal Rights?

Twenty-one years ago (February 10, 1996), Deep Blue, an IBM Supercomputer, defeated Russian Grand Master Gary Kasparov in a game of chess. Kasparov ultimately won the overall match, but a rematch in May of 1997 went to Deep Blue. About six years ago (February 14-15, 2011), another IBM creation named Watson defeated Champions Ken Jennings and Brad Rutter in televised Jeopardy! matches.

The capabilities of computers continue to expand dramatically and surpass human intelligence in certain specific tasks, and it is possible that computing power may develop in the next several decades to match human capacities in areas of emotional intelligence, autonomous decision making and artistic imagination. When machines achieve cognitive capacities that make them resemble humans as thinking, feeling beings, ought we to accord them legal rights? What about moral rights?

Continue reading “Will Robots Ever Deserve Moral and Legal Rights?”

Posted on

The Tay Experiment: Does AI Require a Moral Compass?

In an age of frequent technological developments and innovation, experimentation with artificial intelligence (AI) has become a much-explored realm for corporations like Microsoft. In March 2016, the company launched an AI chatbot on Twitter named Tay with the handle of TayTweets (@TayandYou). Her Twitter description read: “The official account of Tay, Microsoft’s A.I. fam from the Internet that’s got zero chill! The more you talk the smarter Tay gets.” Tay was designed as an experiment in “conversational understanding” –– the more people communicated with Tay, the smarter she would get, learning to engage Twitter users through “casual and playful conversation.”

Continue reading “The Tay Experiment: Does AI Require a Moral Compass?”

Posted on

Digital Decisions in the World of Automated Cars

We’re constantly looking towards the future of technology and gaining excitement for every new innovation that makes our lives easier in some way. Our phones, laptops, tablets, and now even our cars are becoming increasingly smarter. Most new cars on the market today are equipped with GPS navigation, cruise control, and even with some intelligent parallel parking programs. Now, self-driving cars have made their way to the forefront of the automotive revolution.

Continue reading “Digital Decisions in the World of Automated Cars”