← Return to search results
Back to Prindle Institute

artificial intelligence

Posted on

PSai: Singer’s Ethical Insights on Demand

Peter Singer is arguably the most influential living philosopher. For over 50 years, his articles, newspaper columns, blog posts, and books have challenged the views of those who read them. In 1975, he wrote what could be considered his most influential work, Animal Liberation: A New Ethics for Our Treatment of Animals. This text had such an impact that animal rights activists used it as a calling card during their raids on laboratories engaged in animal testing, leaving copies as a challenge to the staff from whom they rescued the animals. He has worked at the universities of Oxford, New York, Monash, and Princeton. His Google Scholar profile indicates that he has been cited over 80,820 times. He has won multiple awards, including the Berggruen Prize, which came with $1 million and, characteristically, Singer donated this money to charities he considered capable of doing the most good.

Yes, he has courted controversy. His opinions on infanticide and euthanasia have won him several vocal detractors. Yet, despite this, one cannot deny that he is a — if not the — modern-day giant in the philosophical field.

Given his academic stature and the sheer number of outputs he has had over his fifty years in the business, it seems reasonable that some might want to consult him when faced with ethical dilemmas. After all, if you’re in a moral quandary, who better to turn to than Singer for help? (I know other options are available, but humor me). The only problem is that as Singer has been writing and thinking about morality and ethics for over half a century, there’s much to consult. Unless you’re intimately familiar with his work (and that’s a lot of work to be familiar with), how can you be confident that what you think Singer might advise is accurate? Sure, you know he subscribes to a hedonistic version of utilitarianism, but knowing that fact alone might not be enough to make the right decision in the heat of the moment. When ethical uncertainty strikes, you need to have Singer with you, advising you on your best course of action. This possibility, however, is impossible (he’s a busy man, after all).

Luckily, we have the next best thing (or the early stages of the next best thing): an AI Peter Singer (PSai).

Created by Sankalpa Ghose, PSai, much like ChatGPT, responds to prompts and questions. While it lacks the robust knowledge portfolio of other generative AIs like ChatGPT, when asked a question about philosophy, which the real Peter Singer has written about, PSai can provide a not unreasonable answer; or at least, it can offer an answer that one might imagine Singer giving (although it as a strange habit of always trying to prompt you for a response). Indeed, Singer himself was involved in the AI’s development and was happy enough with the bots’ performance that he gave Ghose the go-ahead to make it public. Since that launch, PSai’s performance has steadily improved as its designers tweak it. As Ghose writes in the Journal of Medical Ethics: Forum:

It is in this way that Peter and I have progressed on PSai, releasing it publicly and reviewing how it performs in dialogue as measured by how those conversations with others go from our perspective of where it could be improved. Essentially, we have put forward a dialogue platform for others to engage, with that engagement driving its refinement.

The prospect of having a pocket-sized Peter Singer with you wherever you go, always ready to solve ethical riddles or dispense some needed wisdom, is an interesting one, and one that I must confess has some appeal. I like the idea of everyone having access to (a replica of) one of the foremost philosophical minds of our time whenever I need it. As a professional academic philosopher, I see some utility in being able to ask PSai a question about my research and how Singer’s writings might enhance what I’m working on. Rather than having to trawl through numerous books and papers, I could simply ask PSai what text might be the most useful when writing about de-extinction or how Singer might criticize a paper a draft paper on gene drives, which I could then respond and improve upon before submission. This could save a lot of time, which I could spend on other tasks.

For a less academic application, the app could simply function as a virtual Jiminy Cricket — a resource that’s on hand to share some wisdom in scenarios where the right course of action is obscured or where the user knows what’s right but needs some outside confirmation to solidify such knowledge and motivate action. This requires the user to consult the app before deciding to act, and the likelihood of this happening is, let’s face it, likely unlikely. Nevertheless, there is still a chance.

Of course, the idea that one might be able to consult Singer wherever they go is not universally appealing. As already mentioned, Singer has courted controversy when it comes to his historic claims around the permissibility of infanticide, as found in Practical Ethics (under very specific conditions, I must add). Suppose you are inclined to think that the arguments and claims made by the real Singer are incorrect or even repugnant. In that case, you are unlikely to find the possibility of consulting PSai when in ethical difficulty to be enticing. In such circumstances, people are likelier to wish they had a Socrates, Wollstonecraft, Anscombe, or Nietzsche AI. As far as I’m aware, however, these do not exist. But that doesn’t mean they cannot.

I do wonder if we might soon find ourselves awash with AIs trained on the philosophical writings of other philosophers. Might it be the case that, soon, instead of picking up several books, studying, and, over the years, becoming familiar with an ethical mode of life, you could simply jump on your phone, download the Aristotle AI, and have the clarity of morality in the palm of your hand? Such a possibility doesn’t strike me as too absurd. After all, if the PSai proves popular, it only seems natural to diversify.

This, however, brings us to two problems.

First, without wanting to make a point too bluntly, Singer is still alive. He was involved in the development of PSai and in efforts to refine it. This means that the AI has a real-life benchmark against which its performance can be measured. We don’t have to speculate if it says something that the real Singer wouldn’t. Singer himself can say, “nope, that’s wrong,” or, “the AI was correct in its reading of my work, but I’ve now changed my mind” (this is something which Singer and Häyry have considered). Ghose can then adjust the algorithm to perform better in the future. This isn’t possible with most philosophers whose names we are familiar with, as most of them are dead. So, we’d be stumbling around in the dark without a guideline in such cases, relying entirely on that person’s writings and the opinions of those familiar with their work. While an AI produced in such an environment might prove accurate, we lack the definitive authority to know.

Second, if we outsource our ethical deliberations to an AI, be that one based on Singer or any other philosopher, might we be doing a disservice to ourselves by simply following orders? Is it still doing the right thing if we don’t come to an understanding of right and wrong naturally but are simply instructed? For some philosophers, the answer may very well be in the negative. After all, for your classic virtue theorist, morality is like a muscle which one must train every day to be better than before. Each must deliberate using their faculties to identify a master in ethical living, emulate them, and always strive for more. I can’t help but laugh a little at the idea of an Aristotle AI, which, when asked by an unsuspecting user how they should behave in situation X, gets a response along the lines of “figure it out yourself, you lazy sod.” Is this a perfectly accurate representation? Probably not, but it isn’t so far removed as to be a meaningless scenario.

What do we do with PSai, then? Do we celebrate it? Decry it? Or see it as just another tool to help us if we choose? When faced with such a conundrum, it is appropriate to turn to the (figurative) man himself and to give it the last word. So, I asked PSai: Is the fact that this AI has been created a good thing? What justifies your existence? Why should I come to you for answers rather than reading Singer’s works?

Its response:

The creation of this AI aims to make ethical insights more accessible and to engage users in meaningful dialogue about moral issues. By offering interactive discussions, I can help people clarify their thoughts and explore ethical questions in a dynamic way.

While reading Peter Singer’s works provides comprehensive insights into his philosophy, engaging with this AI allows for a more personalized exploration of ideas and can help users apply ethical considerations to their own lives.

The justification for my existence lies in the potential to foster deeper understanding and reflection on ethical principles, contributing to a more thoughtful and compassionate society. What specific ethical questions or topics are you interested in exploring?

Posted on

A New Kind of Risk?

We usually expect to be held accountable for our actions – for both results we intend, and those we do not. We expect, for example, that a car company will ensure that a vehicle doesn’t have major flaws that could result in serious harm before they sell it to customers. To not consider the risks would be negligent, and this is why recalls often look bad for such companies.

But what about algorithms? Should we have a similar expectation that a corporation that develops an algorithm to detect cancer or to detect whether someone is passing off AI-generated content as their own should be sure that there are no significant flaws in their product before they sell it? What if there is no way they could reasonably do so? Given that algorithms can generate erroneous results resulting in serious harms, what is a reasonable standard when it comes to product testing?

In one of the chapters of my forthcoming book on the ethics of AI, I consider a hypothetical issue involving ChatGPT and a professor who might use an algorithm to accuse a student of passing off ChatGPT-written work as their own. There are a great many ethical issues involved when we don’t understand the algorithm and how it might generate false positive results. This has already become a serious issue as students are now being falsely accused of handing in AI-generated work because an algorithm flagged it. A Bloomberg Businessweek study on the services GPTZero and Copyleaks found a 1-2% false positive rate. While that may not sound like a lot, it can mean that millions of students will be falsely accused of cheating with almost no way of defending themselves or receiving an explanation as to what they did wrong.

According to Bloomberg, these interactions are already ruining academic relationships between teachers and students. Some students have now taken to recording themselves writing their entire papers just to be able to disprove the algorithm. Others now obsess about not writing “too robotic” lest they be accused themselves, a problem that is especially prominent for ESL and neuro-divergent students. Should we hold the AI developer whose faulty product generates these kinds of results negligent?

Philosophers of science generally agree that researchers have an obligation to assess inductive risk concerns when accepting a conclusion. In other words, they need to consider what the moral consequences of potentially getting it wrong might be and then consider whether a higher or lower standard of evidence might be appropriate. If, for example, we were testing a chemical to determine how hazardous it is, but the test was only accurate 80% of the time, we would likely demand more evidence. Given the potential harm that can result and the opaqueness of algorithms, AI developers should be similarly conscientious.

If an algorithm operates according to black box principles, the developer may have a good understanding of how to create an algorithm – they will understand that the model can take in various inputs and translate those into outputs – but they will not be able to retrace the steps the model used to arrive at its conclusion. In other words, we have no idea what evidence an algorithm like GPTZero is relying on when it concludes that a piece of text is generated by AI. If the AI developer doesn’t know how the algorithm is using input data as evidence, they cannot evaluate the inductive risk concerns about how sufficient that evidence is.

Still, there are ways, despite the opacity, that an AI developer might attempt to address their inductive risk responsibilities. Koray Karaca argues that developers can build in inductive risk by using cost sensitive machine learning by assigning different costs to different kinds of errors. In the case of AI detectors, the company Turnitin claims to intentionally “oversample” underrepresented students (especially ESL students). By oversampling in this way, the evidentiary standard by which different forms of writing are judged is fine tuned.

Still, there is little accounting for what correlations a model might rely on, making it difficult to explain to students who do get falsely accused why they are being accused in the first place. AI developers have struggled to assess the reliability of their models or evaluate the risks when those correlations are used in error. This issue becomes especially concerning when it comes to things like credit reports. If you don’t know how or why a model compiles a credit report, how can you manage those risks of error? How much must a developer understand about how their algorithm functions before it is put to use? If a developer is aware of the risks of error but also knows that their algorithm is limited in terms of mitigating those risks, at what point do we consider that negligent behavior? If negligence is essentially something we police as a community, we will need to come together quickly to decide what the promise of AI can and can’t excuse.

Posted on

When Is Fair Use “Fair” for AI (and When Is It “Use”)?

The Internet Archive recently lost a high-profile case. Here’s what happened: the Open Library, a project run by the Internet Archive, uploaded digitized versions of books that it owned, and loaned them out to users online. This practice was found to violate copyright law, however, since the Internet Archive failed to procure the appropriate licenses for distributing e-books online. While the Internet Archive argued that its distribution of digital scans of copyrighted works constituted “fair use,” the judge in the case was not convinced.

While many have lamented the court’s decision, others have wondered about the potential consequences for another set of high-profile fair use cases: those concerning AI models training on copyrighted works. Numerous copyright infringement cases have been brought against AI companies, including a class-action lawsuit brought against Meta for training their chatbot using authors’ books without their permission, and a lawsuit from record labels against AI music-generating programs that train on copyrighted works of music.

Like the Internet Archive, AI companies have also claimed that their use of copyrighted materials constitutes “fair use.” These companies, however, have a potentially novel way to approach their legal challenges. While many fair use cases center around whether the use of copyrighted materials is “fair,” some newer arguments involving AI are more concerned with a different kind of “use.”

“Fair use” is a legal concept that attempts to balance the rights of copyright holders with the ability of others to use those works to create something new. Quintessential cases in which it is generally considered “fair” when someone uses copyrighted materials include criticism, satire, educational purposes, or other ways that are considered “transformative,” such as in the creation of art. These conditions have limits, though, and lawsuits are often fought in the gray areas, especially when it is argued that the use of the material will adversely affect the market for the original work.

For example, in the court’s decision against the Internet Archive, the judge argued that uploading digital copies of books failed to be “transformative” in any meaningful sense and that doing so would likely be to the detriment of the original authors – in other words, if someone can just borrow a digital copy, they are less likely to buy a copy of the book. It’s not clear how strong this economic argument is; regardless, some commentators have argued that with libraries in America facing challenges in the form of budget cuts, political censorship, and aggressive licensing agreements from publishers, there is a real need for the existence of projects like the Open Library.

While “fair use” is a legal concept, there is also a moral dimension to the ways that we might think it acceptable to use the work of others. The case of the Internet Archive arguably shows how these concepts can come apart: while the existing law in the U.S. seems to not be on the side of the Open Library, morally speaking there is certainly a case to be made that people are worse off for not having access to its services.

AI companies have been particularly interested in recent fair use lawsuits, as their programs train on large sets of data, much of which is used without permission or a licensing agreement from the creators. While companies have argued that their use of these data constitutes fair use, some plaintiffs have argued they violate fair use law, both in terms of not being sufficiently transformative, and in terms of competing with the original copyright holder.

For example, some music labels have argued that music-generating AI programs often produce content that is extremely similar, or in some cases identical to existing music. In one case, an AI music generator reproduced artist Jason Derulo’s signature tag (i.e., that time when he says his name in his songs so you know it’s by him), a clear indication that the program was copying an existing song.

Again, we can look at the issue of fair use from both a legal and moral standpoint. Legally, it seems clear that when an AI program produces text verbatim from its source, it is not being transformative in any meaningful way. Many have also raised moral concerns around the way that AI programs use artistic materials, both around work being used without permission, as well as in ways that they specifically object to.

But there is an argument from AI defenders around fair use that has less to do with what is “fair” and how copyrighted information is “used”: namely, that AI programs “use” content they find online in the same way that a person does.

Here is how such an argument might go:

-There is nothing morally or legally impermissible about a person reading a lot of content, watching a lot of videos, or listening to a lot of music online, and then using that information as knowledge or inspiration when creating new works. This is simply how people learn and create new things.

-There is nothing specifically morally or legally significant about a person profiting off of the creations that result from what they’ve learned.

-There is nothing morally or legally significant about the quantity of information one consumes or how fast one consumes it.

-An AI is capable of reading a lot of content, watching a lot of videos, and listening to a lot of music online, and using that information as knowledge or inspiration when creating new works.

-The only relevant difference between the way that AI and a person use information to create new content is the quantity of information that an AI can consume and the speed at which it consumes it.

-However, since neither quantity nor speed are relevant moral or legal factors, AI companies are not doing anything impermissible by creating programs that use copyrighted materials online when creating new works.

Arguments of this form can be found in many places. For example, in an interview for NPR:

Richard Busch, lawyer who represents artists who have made copyright claims against other artists, argues: “How is this different than a human brain listening to music and then creating something that is not infringing, but is influenced.”

Similarly, from the blog of AI music creator Udio:

Generative AI models, including our music model, learn from examples. Just as students listen to music and study scores, our model has “listened” to and learned from a large collection of recorded music.

While these arguments also point to the originality of the final creation, a crucial component of their defense lies in how AI programs “use” copyrighted material. Since there’s nothing inherently inappropriate about a person consuming a lot of information, processing it, getting inspired by it, and producing something as a result, nor should we think it inappropriate for an AI to do the same things.

There have, however, been many worries raised already with inappropriate personification of AI, from concerns around AI being “conscious,” to downplaying errors by referring to them as “hallucinations.” In the above arguments, these personifications are more subtle: AI-defenders talk in terms of the programs “listening,” “creating,” “learning,” and “studying.” No one would begrudge a human being for doing these things. Importantly, though, these actions are the actions of human beings – or, at least, of intelligent beings with moral status. Uncritically applying them to computer programs thus masks an important jump in logic that is not warranted by what we know about the current capabilities of AI.

There are a lot of battles to be fought in terms of what constitute truly “transformative” works in lawsuits against AI companies. Regardless, part of the ongoing legal and moral discussions will undoubtedly need to shift their focus to new questions about what “use” means when it comes to AI.

Posted on

The Algorithm Made Me Do It

image of smiling monitor on top of shoulders

Artificial intelligence promises greater efficiency, greater accuracy, and lower costs in a host of different fields. The development of hiring algorithms, for example, can streamline the hiring process and better identify the best candidates for the job. In healthcare, algorithms can cut down on hours of work by quickly detecting illnesses or broken bones and finding the most efficient use of scarce healthcare resources. In the criminal justice system, AI can speed up the process by identifying potential recidivists.

However, these algorithms are often not transparent in how they work, or even how they are designed to work. This leaves us – as end-users of these algorithms – in an awkward position:  forced to accept the conclusions of an opaque algorithm that could be loaded with faulty assumptions, pseudo-science, and statistical noise. Might this blind acceptance conflict with our moral duties regarding responsible belief? How should we balance the tradeoffs between efficiency and lower cost versus credulity and gullibility?

While it’s easy to criticize AI’s many applications, it’s important to recognize the potential benefits. For example, while a self-driving car may present concerns about accountability for accidents, it is also true that the technology could offer countless benefits in terms of traffic efficiency, minimizing driver fatigue, and significant economic benefits.

In the field of affective computing, using AI to identify and categorize emotions can offer significant benefits to people with autism, or can help potentially identify people likely to have a stroke or a heart attack. It can also help with caregiving, with automated assistants that are more emotionally aware of the people they are helping. AI can also be used for the purposes of diagnosis or resource management. Similarly, the use of AI in the financial sector for things like loans can lead to better identification of risk, greater profits, and more competitive pricing.

The use of hiring algorithms in the workplace, meanwhile, will allow employers to cut down on the amount of time and resources it takes to find a new hire and can potentially take the guesswork out of identifying the most promising candidates. Similar benefits can accrue to workplaces that use algorithms for employee evaluations. Supposing that issues relating to bias can be addressed, algorithms offer the benefit of a more impartial evaluation, less dependent on the personal feelings of an individual evaluator. Ultimately, there are a great many reasons why taxpayers, job seekers, and home buyers all stand to benefit from the AI.

Still, we must be wary of the hidden costs. We may be tempted, either because it’s cheaper or more convenient, to accept unaccountable and unfair systems that we may have good reason not to excuse.

Consider the case of Tammy Dobbs. A resident of Arkansas, Tammy has cerebral palsy and requires assistance getting into a wheelchair. In 2016 after the state adopted a new algorithm to determine what level of care she should receive, the regular hours of assistant care that Tammy received were severely cut, making it difficult for her to do things like go to the bathroom. A government official came to her house, completed a questionnaire, and then relayed the algorithm’s determination. When pressed for an explanation, the official could only say, “Sorry, that’s what the computer is showing me.” The government’s expectation seemed to be for Dobbs to simply accept it. Eventually, a legal team revealed significant flaws in the state’s algorithm – the algorithm didn’t even consider whether someone had cerebral palsy.

Similar cases are easy to find. Glenn Rodriguez had to fight to get an explanation for why the recidivist algorithm COMPAS concluded that he was at a high risk for reoffending. The corporation who created COMPAS refused to reveal how the assessment was made – even to the parole board – citing trade secrets. If an algorithm can have such a profound impact on your life, surely we deserve a better explanation than “The algorithm made me do it.”

Many algorithms can have prejudicial assumptions baked in. A recidivism algorithm that is mostly trained on blue-collar or petty crime will not likely evaluate everyone the same. A hiring algorithm that contains a personality test designed to identify extroverted personality types might also be tracking whether candidates are likely to have a mental illness. Many hiring companies now make use of video recordings of candidates to detect body language, despite the research demonstrating that body language cannot predict successful job performance and likening the practice to pseudoscience like phrenology. Unfortunately, candidates have no idea how they are being evaluated and no avenue to appeal if they believe that an error has occurred.

In cases like this, particularly where there are financial incentives to sell these products as efficient no-brainer solutions, developers will have reason to stifle doubts and concerns. As the designer who created the algorithm in the Dobbs case argued, perfect transparency is overrated. “It’s not simple…My washing machine isn’t simple,” but “you’re going to have to trust me that a bunch of smart people determined this is the smart way to do it.” All of this means that there is an incentive on the part of developers and end-users to put their faith in algorithms that may be quite suspect.

As W.K. Clifford argued in his ethics of belief, every time we adopt beliefs without sufficient evidence, we do something wrong. This is because beliefs dispose us to action; the more we adopt the habit of passively accepting algorithmic conclusions without adequate inquiry, the more we expose ourselves to risk. But the consequences of the beliefs we adopt extend beyond the individual; our beliefs affect our entire community. If customers and taxpayers don’t ask questions – and developers are happier not to answer them – we end up with a situation much like that government official in the Dobbs case. No accountability; no justification. Don’t ask questions, just accept the outcome.

Artificial intelligence presents a collective action problem. Individuals alone cannot challenge these poor answers that lack transparency. Instead, resolution requires a collective response – we will need to work together to resist the constant temptation of lower costs, greater efficiency, and passing the buck.

Posted on

“Grief Tech”: A Blessing or a Curse?

photograph of patrons holding hands at seance

Generative AI certainly has many intriguing uses. Everything from music, to text, to video, can now be generated – a new power riddled with ethical concerns. Perhaps one of the more sensitive topics concerns the use of generative AI to recreate people who are deceased. The music and film industries are already grappling with the possibility of reviving artists to perform again. But the issue can also hit much closer to home. There’s a good buck to be made in helping folks reconnect with dead family members in ways that weren’t previously possible. My Heritage’s Deep Nostalgia can colorize and animate old family photographs, while other vendors offer the opportunity to chat with a dead relative or hold a funeral where the deceased can address the room. Such technology offers a priceless chance at closure and healing, but might it also be exploiting the dead as well as the living?

The rising industry of “grief tech” takes many different forms. At a recent funeral, a woman who passed away at 87 was able to speak with mourners. A similar process was used at the funeral of former SAG president Ed Asner. Those attending his funeral were able to converse with him as generative AI formed responses on the fly from a bank of answers regarding his work, politics, and family life he had previously recorded. This was all thanks to the company StoryFile, whose technology was originally conceived with the intention of recording the memories of Holocaust survivors.

Many appreciate the opportunity this kind of technology affords. As the 87-year woman’s son noted, “Nothing could prepare me for what I was going to witness when I saw it.” It isn’t hard to see the benefit this provides loved ones.

In addition to these more elaborate reproductions of the deceased, chatbots are another way generative AI can resurrect people who have passed away. In 2016 James Vlahos used recordings of his father’s life story to create a “Dadbot” that he could create an interactive experience that emulated his father. Vlahos found comfort in this and has since launched a company that allows people to upload their memories in order to create an AI version of themselves that can live on.

Supporters of the technology claim that it provides comfort to loved ones as it offers a way of preserving memories. One man, for instance, was able to recreate his grandfather so that he could have a final chance to say goodbye.

Despite their promise, however, these services appear exploitative – not only of the dead but of the living families who may be willing to pay vast sums of money to see their loved ones again. Some companies require living consent in order to be part of the program, but there’s no guarantee this will be the universal standard moving forward. There is, for example, already interest in recreating historical figures who have no opportunity to offer consent.

It may also be the case that grief tech services are not healthy for us. While creating an AI avatar can be a nice way to memorialize someone, it can also be a crutch that prevents us from completing the grieving process. Not only can this enable our desire to avoid reality, but it can prevent us from making new, meaningful relationships.

Many of the services promise greater honesty and transparency. It’s assumed that the person filling out the questions can do so more truthfully – they have the opportunity to say things in death that they might not wish to have revealed in life. Thus, the process can get closer to the truth and offer real closure.

But it can be misleading who we are actually talking to. While some anticipate getting a “freer, truer version of their lost loved ones,” it may be that what they receive is a useful, polished fiction. While people can be more honest when preparing their words for posterity, that does not mean that we can trust people to accurately relay their life’s details.

Further, the fact that a profile is created from old memories and thoughts doesn’t mean that it will be a literal copy. The model might sound like a loved one, it might say similar things, but when an AI model is generating that content, it is still the model that is producing statements. While this might give the impression to a loved one that they are finally going to have the long-awaited conversation they’ve sought, in reality, a computer model may simply be making things up based on the echoes of distant memories. We should be incredibly skeptical about the new information that gets revealed; it is a well-documented phenomenon that AI can “hallucinate” facts.

This could have the potential to create further problems. What if the AI makes some kind of controversial claim after the fact? “Bill killed me!” “Leave all my money to Sally.” Not only is there potential to generate unnecessary postmortem controversies, but even the potential for manipulation depending on how the model was constructed and by whom. We’ve already proven quite susceptible to mistaking machines for sentient beings. It’s not hard to imagine forming an unhealthy attachment to a model of a reincarnated loved one.

The potential for abuse appears rife. As one article notes, there are marketing opportunities created by effectively creating a digital clone of a person that can mimic the choices that you would make. This would be a significant benefit for marketing and advertising – a company could sell services to the bereaved, while also harvesting that customer data for advertising purposes.

Resurrecting the dead in AI form promises great benefit, but the attending risks are great. While this has the potential to revolutionize the way we approach death, that promise alone doesn’t mean it’s a good idea.

Posted on

Who Should Own the Products of Generative AI?

droste effect image of tunnel depicted on laptop screen

Like many educators, I have encountered difficulties with Generative AI (GenAI); multiple students in my introductory courses have submitted work from ChatGPT as their own. Most of these students came to (or at least claimed to) recognize why this is a form of academic dishonesty. Some, however, failed to see the problem.

This issue does not end with undergraduates, though. Friends in other disciplines have reported to me that their colleagues use GenAI to perform tasks like writing code they intend to use in their own research and data analysis or create materials like cover letters. Two lawyers recently submitted filings written by ChatGPT in court (though the judge caught on as the AI “hallucinated” case law). Now, some academics even credit ChatGPT as a co-author on published works.

Academic institutions typically define plagiarism as something like the following: claiming the work, writing, ideas or concepts of others as one’s own without crediting the original author. So, some might argue that ChatGPT, Dall-E, Midjourney, etc. are not someone. They are programs, not people. Thus, one is not taking the work of another as there is no other person. (Although it is worth noting that the academics who credited ChatGPT avoid this issue. Nonetheless, their behavior is still problematic, as I will explain later.)

There are at least three problems with this defense, however. The first is that it seems deliberately obtuse regarding the definition of plagiarism. The dishonesty comes from claiming work that you did not perform as your own. Even tho GenAI is not a person, its work is not your work – so using it still involves acting deceptively, as Richard Gibson writes.

Second, as Daniel Burkett argues, it is unclear that there is any justice-based consideration which supports not giving AI credit for their work. So, the “no person, no problem” idea seems to miss the mark. There’s a case to be made that GenAIs do, indeed, deserve recognition despite not being human.

The third problem, however, dovetails with this point. I am not certain that credit for the output of GenAIs stops with the AI and the team that programmed it. Specifically, I want to sketch out the beginnings of an argument that many individuals have proper grounds to make a claim for at least partial ownership of the output of GenAI – namely, those who created the content which was used to “teach” the GenAI. While I cannot fully defend this claim here, we can still consider the basic points in its support.

To make the justification for my claim clear, we must first discuss how GenAI works. It is worth noting, though, that I am not a computer scientist. So, my explanation here may misrepresent some of the finer details.

GenAIs are programs that are capable of, well, generating content. They can perform tasks that involve creating text, images, audio, and video. GenAI learns to generate content by being fed large amounts of information, known as a data set. Typically, GenAIs are trained first via a labeled data set to learn categories, and then receive unlabeled data which they characterize based on the labeled data. This is known as semi-supervised learning. The ability to characterize unlabeled data is how GenAIs are able to create new content based on user requests. Large language models (LLMs) (i.e., text GenAI like ChatGPT) in particular learn from vast quantities of information. According to Open AI,  their GPT models are trained, in part, using text scraped from the internet. When creating output, GenAIs predict what is likely to occur next given the statistical model generated by data they were previously fed.

This is most easily understood with generative language models like ChatGPT. When you provide a prompt to ChatGPT, it begins crafting its response by categorizing your request. It analyzes the patterns of text found within the subset of its dataset that fit into the categories you requested. It then outputs a body of text where each word was statistically most likely to occur, given the previous word and the patterns observed in its data set. This process is not just limited to LLMs – GenAIs that produce audio learn patterns from data sets of sound and predict which sound is likely to come next, those that produce images learn from sets of images and predict which pixel is likely to come next, etc.

GenAI’s reliance on data sets is important to emphasize. These sets are incredibly large. GPT3, the model that underpins ChatGPT, was trained on 40 terabytes of text. For reference, 40 TB is about 20 trillion words. These texts include Wikipedia, online bodies of books, as well as internet content. Midjourney, Stable Diffusion, and DreamUp – all image GenAIs – were trained on LAION, which was created by gathering images from the internet. The essential takeaway here is that GenAI are trained on the work of countless creators, be they the authors of Wikipedia articles, digital artists, or composers. Their work was pulled from the internet and put into these datasets without consent or compensation.

On any plausible theory of property, the act of creating an object or work gives one ownership of it. In perhaps the most famous account of the acquisition of property, John Locke argues that one acquires a previously unowned thing by laboring on it. We own ourselves, Locke argues, and our labor is a product of our bodies. So, when we work on something, we mix  part of ourselves with it, granting us ownership over it. When datasets compile content by, say, scraping the internet, they take works created by individuals – works owned by their creators – compile them into data sets and use those data sets to teach GenAI how to produce content. Thus, it seems that works which the programmers or owners of GenAI do not own are essential ingredients in GenAI’s output.

Given this, who can we judge as the rightful owners of what GenAI produces? The first and obvious answer is those who program the AI, or the companies that reached contractual agreements with programmers to produce them. The second and more hidden party is those whose work was compiled into the data sets, labeled or unlabeled, which were used to teach the GenAI. Without either component, programs like ChatGPT could not produce the content we see at the quality and pace which they do. To continue to use Locke’s language, the labor of both parties is mixed in to form the end result. Thus, both the creators of the program and the creators of the data seem to have at least a partial ownership claim over the product.

Of course, one might object that the creators of the content that form the datasets fed to a GenAI, gave tacit consent. This is because they placed their work on the internet. Any information put onto the internet is made public and is free for anyone to use as they see fit, provided they do not steal it. But this response seems short-sighted. GenAI is a relatively new phenomenon, at least in terms of public awareness. The creators of the content used to teach GenAI surely were not aware of this potential when they uploaded their content online. Thus, it is unclear how they could consent, even tacitly, to their work being used to teach GenAI.

Further, one could argue that my account has an absurd implication for learning. Specifically, one might argue that, on my view, whenever material is used for teaching, those who produced the original material would have an ownership claim on the content created by those who learn from it. Suppose, for instance, I wrote an essay which I assigned to my students advising them on how to write philosophy. This essay is something I own. However, it shapes my students’ understanding in a way that affects their future work. But surely this does not mean I have a partial ownership claim to any essays which they write. One might argue my account implies this, and so should be rejected.

This point fails to appreciate a significant difference between human and GenAI learning. Recall that GenAI produces new content through statistical models – it determines which words, notes, pixels, etc. are most likely to follow given the previous contents. In this way, its output is wholly determined by the input it receives. As a result, GenAI, at least currently, seems to lack the kind of spontaneity and creativity that human learners and creators have (a matter D’Arcy Blaxwell demonstrates the troubling implications of here). Thus, it does not seem that the contents human learners consume generate ownership claims on their output in the same way as GenAI outputs.

I began this account by reflecting on GenAI’s relationship to plagiarism and honesty. With the analysis of who has a claim to ownership of the products created by GenAI in hand, we can more clearly see what the problem with using these programs in one’s work is. Even those who attempt to give credit to the program, like the academics who listed ChatGPT as a co-author, are missing something fundamentally important. The creators of the work that make up the datasets AI learned on ought to be credited; their labor was essential in what the GenAI produced. Thus, they ought to be seen as part owner of that output. In this way, leaning on GenAI in one’s own work is an order of magnitude worse than standard forms of plagiarism. Rather than taking the credit for the work of a small number of individuals, claiming the output of GenAI as one’s own fails to properly credit hundreds, if not thousands, of creators for their work, thoughts, and efforts.

Further still, this analysis enables us to see the moral push behind the claims made by the members of SAG-AFTRA and the WGA who are striking, in part, out of concern for AI learning from their likeness and work to mass produce content for studios. Or consider The New York Times ongoing conflict with OpenAI. Any AI which would be trained to write scripts, generate an acting performance, or relay the news would undoubtedly be trained on someone else’s work. Without an agreement in place, practices like these may be tantamount to theft.

Posted on

Black-Box Expertise and AI Discourse

image of black box highlighted on stage

It has recently been estimated that new generative AI technology could add up to $4.4 trillion to the global economy. This figure was reported by The New York Times, Bloomberg, Yahoo Finance, The Globe and Mail, and dozens of other news outlets and websites. It’s a big, impressive number that has been interpreted by some as even more reason to get excited about AI, and by others to add to a growing list of concerns.

The estimate itself came from a report recently released by consulting firm McKinsey & Company. As the authors of the report prognosticate, AI will make a significant impact in the kinds of tasks that can be performed by AI instead of humans: some of these tasks are relatively simple, such as creating “personalized emails,” while others are more complex, such as “communicating with others about operational plans or activities.” Mileage may vary depending on the business, but overall those productivity savings can add up to huge contributions to the economy.

While it’s one thing to speculate, extraordinary claims require extraordinary evidence. Where one would expect to see a rigorous methodology in the McKinsey report, however, we are instead told that the authors referenced a “proprietary database” and “drew on the experience of more than 100 experts,” none of whom are mentioned. In other words, while it certainly seems plausible that generative AI could add a lot of value to the global economy, when it comes to specific numbers, we’re just being asked to take McKinsey’s word for it. McKinsey are perceived by many to be experts, after all.

It often is, in general, perfectly rational to take an expert’s word for it, without having to examine their evidence in detail. Of course, whether McKinsey & Company really are experts when it comes to AI and financial predictions (or, really, anything else for that matter) is up for debate. Regardless, something is troubling about presenting one’s expert opinion in such a way that one could not investigate it even if one wanted to. Call this phenomenon black-box expertise.

Black-box expertise seems to be common and even welcomed in the discourse surrounding new developments in AI, perhaps due to an immense amount of hype and appetite for new information. The result is an arms race of increasingly hyperbolic articles, studies, and statements from legitimate (and purportedly legitimate) experts, ones that are often presented without much in the way of supporting evidence. A discourse that encourages black-box expertise is problematic, however, in that it can make the identification of experts more difficult, and perhaps lead to misplaced trust.

We can consider black-box expertise in a few forms. For instance, an expert may present a conclusion but not make available their methodology, either in whole or in part – this seems to be what’s happening in the McKinsey report. We can also think of cases in which experts might not make available the evidence they used in reaching a conclusion, or the reasoning they used to get there. Expressions of black-box expertise of these kinds have plagued other parts of the AI discourse recently, as well.

For instance, another expert opinion that has been frequently quoted comes from AI expert Paul Christiano, who, when asked about the existential risk posed by AI, claimed: “Overall, maybe we’re talking about a 50/50 chance of catastrophe shortly after we have systems at the human level.” It’s a potentially terrifying prospect, but Christiano is not forthcoming with his reasoning for landing on that number in particular. While his credentials would lead many to consider him a legitimate expert, the basis of his opinions on AI is completely opaque.

Why is black-box expertise a problem, though? One of the benefits of relying on expert opinion is that the experts have done the hard work in figuring things out so that we don’t have to. This is especially helpful when the matter at hand is complex, and when we don’t have the skills or knowledge to figure it out ourselves. It would be odd, for instance, to demand to see all of the evidence, or scrutinize the methodology of an expert who works in a field of which we are largely ignorant since we wouldn’t really know what we were looking at or how to evaluate it. Lest we be skeptics about everything we’re not personally well-versed in, reliance on expertise necessarily requires some amount of trust. So why should it matter how transparent an expert is about the way they reached their opinion?

The first problem is one of identification.  As we’ve seen, a fundamental challenge in evaluating whether someone is an expert from the point of view of a non-expert is that non-experts tend to be unable to fully evaluate claims made in that area of expertise. Instead, non-experts rely on different markers of expertise, such as one’s credentials, professional accomplishments, and engagement with others in their respective areas. Crucially, however, non-experts also tend to evaluate expertise on the basis of factors like one’s ability to respond to criticism, the provisions of reasons for their beliefs, and their ability to explain their views to others. These factors are directly at odds with black-box expertise: without making one’s methodology or reasoning apparent, it makes it difficult for non-experts to identify experts.

A second and related problem with black-box expertise is that it becomes more difficult for others to identify epistemic trespassers: those who have specialized knowledge or expertise in one area that make judgments on matters in areas where they lack expertise. Epistemic trespassers are, arguably, rampant in AI discourse. Consider, for example, a recent and widely-reported interview with James Cameron, the director of the original Terminator series of movies. When asked about whether he considered artificial intelligence to be an existential risk, he remarked, “I warned you guys in 1984, and you didn’t listen” (referring to the plot of the Terminator movies in which the existential threat of AI was very tangible). Cameron’s comment makes for a fun headline (one which was featured in an exhausting number of publications), but he is by no measure an expert in artificial intelligence in the year 2023. He may be an accomplished filmmaker, but when it comes to contemporary discussions of AI, he is very much an epistemic trespasser.

Here, then, is a central problem with relying on black-box expertise in AI discourse: expert opinion presented without transparent evidence, methodology, or reasoning can be difficult to distinguish from opinions of non-experts and epistemic trespassers. This can make it difficult for non-experts to navigate an already complex and crowded discourse to identify who should be trusted, and whose word should be taken with a grain of salt.

Given the potential of AI and its tendency to produce headlines that tout it both as a possible savior of the economy and destroyer of the world, being able to identify experts is an important part of creating a discourse that is productive and not simply motivated by fear-mongering and hype. Black-box expertise, like that one on display in the McKinsey report and many other commentaries from AI researchers, provides a significant barrier to creating that kind of discourse.

Posted on

Should AI Development Be Stopped?

photograph of Arnold Schwarznegger's Terminator wax figure

It was a bit of a surprise this month when the so-called “Godfather of AI” Geoffrey Hinton announced that he was quitting at Google after working there for more than a decade developing Google’s AI research division. With his newfound freedom to speak openly, Hinton has expressed ethical concerns about the use of the technology for its capacity to destabilize society and exacerbate income equality. “I console myself with the normal excuse: If I hadn’t done it, somebody else would have,” he told The New York Times this month. That such an authoritative figure within the AI field has now condemned the technology is a significant addition to a growing call for a halt on AI development. Last month more than 1,000 AI researchers published an open letter calling for a six-month pause on training AI systems more powerful than the newest ChatGPT. But does AI really pose such a risk that we ought to halt its development?

Hinton worries about humanity losing control of AI. He was surprised, for instance, when Google’s AI language model was able to explain to him why a joke he made up was funny. He is also concerned that despite AI models being far less complex than the human brain, they are quickly becoming able to do complex tasks on par with a human. Part of his concern is the idea of algorithms seeking greater control and that he doesn’t know how to control the AI that Google and others are building. This concern is part of the reason for the call for a moratorium as the recent letter explains, “Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? […] Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable.”

Eliezer Yudkowsky, a decision theorist, recently suggested that a 6-month moratorium is not sufficient. Because he is concerned that AI will become smarter than humans. His concern is that building anything that is smarter than humans will definitely result in the death of everyone on Earth. Thus, he has called for completely ending the development of powerful AI and believes that an international treaty should ban its use with its provisions subject to military action if necessary. “If intelligence says that a country outside the agreement is building a GPU cluster,” he warned, “be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike.”

These fears aren’t new. In the 1920s and 1930s there were concerns that developments in science and technology were destabilizing society and would strip away jobs and exacerbate income inequality. In response, many called for moratoriums on further research – moratoriums that did not happen. In fact, Hinton does not seem to think this is practical since competitive markets and competitive nations are already involved in an arms race that will only compel further research.

There is also the fact that over 400 billion dollars has been invested in AI in just 2022, meaning that it will be difficult to convince people to bring all of this research to a halt given the investment and potentially lucrative benefits. Artificial intelligence has the capability to make certain tasks far more efficient and productive, from medicine to communication. Even Hinton believes that development should continue because AI can do “wonderful things.” Given these , one response to the proposed moratorium insists that “a pause on AI work is not only vague, but also unfeasible.” They argue, instead, that we simply need to be especially clear about what we consider “safe” and “successful” AI development to avoid potential missteps.

Where does this leave us? Certainly we can applaud the researchers who take their moral responsibilities seriously and feel compelled to share their concerns about the risks of development. But these kinds of warnings are vague, and researchers need to do a better job at explaining the risks. What exactly does it mean to say that you are worried about losing control of AI? Saying something like this encourages the public to imagine fantastical sci-fi ideas akin to 2001: A Space Odyssey or The Terminator. (Unhelpfully, Hinton has even agreed with the sentiment that our situation is like the movies. Ultimately, people like Yudkowsky and Hinton don’t exactly draw a clear picture of how we get from ChatGPT to Skynet. The fact that deep neural networks are so successful despite their simplicity compared to a human brain might be a cause for concern, but why exactly? Hinton says: “What really worries me is that you have to create subgoals in order to be efficient, and a very sensible subgoal for more or less anything you want to do is get more power—get more control.”  Yudkowsky suggests: “If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter.” He suggests that “A sufficiently intelligent AI won’t stay confined to computers for long.” But how?

These are hypothetical worries about what AI might do, somehow, if they become more intelligent than us. These concepts remain hopelessly vague. In the meantime, there are already real problems that AI is causing such as predictive policing and discriminatory biases. There’s also the fact that AI is incredibly environmentally unfriendly. One AI model can emit five times more carbon dioxide than the lifetime emissions of a car. Putting aside how advanced AI might become relative to humans, it is already proving to pose significant challenges that will require society to adapt. For example, there has been a surge in AI-generated music recently and this presents problems for the music industry. Do artists own the rights to the sound of their own voice or does a record company? A 2020 paper revealed that a malicious actor could deliberately create a biased algorithm and then conceal this fact from potential regulators owing to their black box nature. There are so many areas where AI is being developed and deployed where it might take years of legal reform before clear and understandable frameworks can be developed to govern their use. (Hinton points at the capacity for AI to negatively affect the electoral process as well). Perhaps this is a reason to slow AI development until the rest of society can catch up.

If scientists are going to be taken seriously by the public,the nature of the threat will need to be made much more clear. Most of the more serious ethical issues involving AI such as labor reform, policing, and bias are more significant, not because of AI itself, but because AI will allow smaller groups to benefit without transparency and accountability. In other words, the ethical risks with AI are still mostly owing to the humans who control that AI, rather than the AI itself. While humans can make great advancements in science, this is often in advance of understanding how that knowledge is best used.

In the 1930s, the concern that science would destroy the labor market only subsided when a world war made mass production and full employment necessary. We never addressed the underlying problem. We still need to grapple with the question of what science is for. Should AI development be dictated by a relatively small group of financial interests who can benefit from the technology while it harms the rest of society? Are we, as a society, ready to collectively say “no” to certain kinds of scientific research until social progress catches up with scientific progress?

Posted on

Specious Content and the Need for Experts

photograph of freestanding faucet on lake

A recent tweet shows what looks to be a photo of a woman wearing a kimono. It looks authentic enough, although not knowing much about kimonos myself I couldn’t tell you much about it. After learning that the image is AI-generated, my opinion hasn’t really changed: it looks fine to me, and if I ever needed to use a photo of someone wearing a kimono, I may very well choose something that looked the same.

However, reading further we see that the image is full of flaws. According to the author of the tweet who identifies themselves as a kimono consultant, the fabric doesn’t hang correctly, and there are pieces seemingly jutting out of nowhere. Folds are in the wrong place, the adornments are wrong, and nothing really matches. Perhaps most egregiously, it is styled in a way that is reserved only for the deceased, which would make the person someone who was committing a serious faux pas, or a zombie.

While mistakes like these would fly under the radar of the vast majority of those viewing it, it’s indicative of the ability of AI-powered generative image and text programs to produce content that appears authoritative but is riddled with errors.

Let’s give this kind of content a name: specious content. It’s the kind of content – be it in the form of text, images, or video – that appears to be plausible or realistic on its surface, but is false or misleading in a way that can only be identified with some effort and relevant knowledge. While there was specious content before AI programs became ubiquitous, the ability of such programs to produce content on a massive scale for free significantly increases the likelihood of misleading users and causing harm.

Given the importance of identifying AI-generated text and images, what should our approach be when dealing with content that we suspect is specious? The most common advice seems to be that we should rely on our own powers of observation. However, this approach may very well do more harm than good.

A quick Googling of how to avoid being fooled by AI-generated images will turn up much of the same advice: look closely and see if anything looks weird. Media outlets have been quick to point out that AI image-generating tools often mess up hands and fingers, that sometimes glasses don’t quite fit right on someone’s face, or that body parts or clothes overlap in places where they shouldn’t. A recent New York Times article goes even further and suggests that people look for mismatched fashion accessories, eyes that are too symmetrically spaced, glasses with mismatching end pieces, indents in ears, weird stuff in someone’s hair, and a blurred background.

The problem with all these suggestions is that they’re either so obvious as to not be worth mentioning, or so subtle that they would escape noticing even under scrutiny.

If an image portrays someone with three arms you are probably confident enough already that the image isn’t real. But people blur their backgrounds on purpose all the time, sometimes they have weird stuff in their hair, and whether a face is “too symmetrical” is a judgment beyond the ability of most people.

A study recently discussed in Scientific American underscores how scrutinizing a picture for signs of imperfections is a strategy that’s doomed to fail. It found that while participants performed no better than chance at identifying AI-generated images without any instruction, their detection rate increased by a mere 10% after reading advice on how to look closely for imperfections. With AI technology getting better every day, it seems likely that even these meager improvements won’t last long.

We’re not only bad at analyzing specious content, but going through checklists of subtle indicators is just going to make things worse. The problem is that it’s easy to interpret the lack of noticeable mistakes as a mark of authenticity: if we are unable to locate any signs that an image is fake, then we may be more likely to think that it’s genuine, even though the problems may be too subtle for us to notice. Or we might simply not be knowledgeable or patient enough to find them. In the case of the kimono picture, for example, what might be glaringly obvious to someone who was familiar with kimonos goes straight over my head.

But these problems also guide us to better ways of dealing with specious content. Instead of relying on our own limited capacity to notice mistakes in AI-generated images, we should outsource these tasks.

One new approach to detecting these images comes from AI itself: as tools to produce images have improved, so have tools that have been designed to detect those images (although it seems as though the former is winning, for now).

The other place to look for help is from experts. Philosophers debate about what, exactly, makes an expert, but in general, they typically possess a lot of knowledge and understanding of a subject, make reliable judgments about matters within their domain of expertise, are often considered authoritative, and can explain concepts to others. While identifying experts is not always straightforward, what will perhaps become a salient marker of expertise in the current age of AI will be one’s ability to distinguish specious content from that which is trustworthy.

While we certainly can’t get expert advice for every piece of AI-generated content we might come across, increasing amounts of authoritative-looking nonsense should cause us to recognize our own limitations and attempt to look to those who possess expertise in a relevant area. While even experts are sometimes prone to being fooled by AI-generated content, the track record of non-experts should lead us to stop looking closely for weird hands and overly-symmetrical features and start looking for outside help.

Posted on

The Garden of Simulated Delights

digitized image of woman relaxing on beach dock

There’s no easy way to say this, so I’ll be blunt. There is a significant chance that we are living in a computer simulation.

I know this sounds pretty silly. Unhinged, even. But I believe it to be true. And I’m not alone. This idea is becoming increasingly popular. Some people even believe that we are almost certainly living in a computer simulation.

It may turn out that this idea is mistaken. But it’s not a delusion. Unlike delusions, this idea is supported by coherent arguments. So, let’s talk about it. Why do people think we’re in a simulation? And why does it matter?

We should begin by unpacking the idea. Computer simulations are familiar phenomena. For example, popular programs like The Sims and Second Life contain simulated worlds filled with virtual things (trees, houses, people, etc.) interacting and relating in ways that resemble the outside world. The Simulation Hypothesis says that we are part of a virtual world like that. We have no reality outside of a computer. Rather, our minds, bodies, and environment are all parts of an advanced computer simulation. So, for example, when you look at the Moon, your visual experience is the product of the calculations of a computer simulating what would happen in the visual system of a biological human if they were to look at Earth’s satellite.

The Simulation Hypothesis is one member of a much-discussed class of hypotheses that invoke descriptions of the world that appear to be compatible with all our experience and evidence yet, if true, would contradict many of our fundamental beliefs or systematically undermine our knowledge. An archetype is the 17th-century philosopher René Descartes’s Evil Demon Hypothesis, according to which all of your sensations are being produced by a malicious demon who is tricking you into thinking you have a body, live on Earth, and so forth when in fact none of those things are true. Descartes did not think the Evil Demon Hypothesis was true. Rather, he used it to illustrate the elusiveness of certainty: Since all your sensations are compatible with the Evil Demon Hypothesis, you can’t rule it out with certainty, and consequently you can’t be certain you have a body, live on Earth, and so forth. What’s special about the Simulation Hypothesis relative to the Evil Demon Hypothesis is that there’s a case to be made for thinking that the former is true.

The basic idea behind this argument can be expressed in two key premises. The first is that it’s possible for conscious, human-like minds to exist in a computer. Note that the organ out of which consciousness arises in biological beings – the brain – is a complex physical system composed of simple parts interacting in law-governed ways. If a civilization’s technology and understanding of human-like brains becomes advanced enough, then they should be able to simulate human-like brains at a level of accuracy and detail that replicates their functioning, much like humans can now simulate a nematode brain. Simulated minds would have a nonorganic substrate. But substrate is probably less important than functioning. If the functional characteristics of a simulated brain were to replicate a biological human brain, then, the argument goes, the simulation would probably give rise to a conscious mind.

The second premise is that some non-negligible fraction of intelligent civilizations will eventually develop the capacity and motivation to run hugely many simulations of planets or universes that are populated with human-like minds, such that, across time, there are many simulated human-like minds for every non-simulated human-like mind. The exponential pace of technological development we observe within our own history lends plausibility to the claim that some intelligent civilizations will eventually develop this capacity. (In fact, it suggests that we may develop this capacity ourselves.) And there are many potential reasons why intelligent civilizations might be motivated to run hugely many simulations. For example, advanced civilizations could learn a great deal about the universe or, say, the spread of coronaviruses on Earth-like planets through simulations of universes or planets like ours, running in parallel and much faster than real time. Alternatively, spending time in a hyper realistic simulation might be entertaining to people in advanced civilizations. After all, many humans are entertained by this sort of thing today.

If you accept these premises, you should think that the Simulation Hypothesis is probably true. This is because the premises suggest that across time there are many more simulated beings than biological beings with experiences like yours. And since all your experiences are compatible with both the possibility that you’re simulated and the possibility that you aren’t, you should follow the numbers and accept that you’re likely in a simulation. By analogy, if you purchase a lottery ticket and don’t have any special reason to think you’ve won, then you should follow the numbers and accept that you’ve likely lost the lottery.

This argument is controversial. Interested readers can wiggle themselves down a rabbit hole of clarifications, refinements, extensions, empirical predictions, objections, replies (and more objections, and more replies). My own view is that this argument cannot be easily dismissed.

Suppose you agree with me. How do we live in the light of this argument? What are the personal and ethical implications of accepting that the Simulation Hypothesis is probably (or at least very possibly) true?

Well, we certainly have to accept that our world may be much weirder than we thought. But in some respects things aren’t as bad as they might seem. The Simulation Hypothesis needn’t lead to nihilism. The importance of most of what we care about – pleasure, happiness, love, achievement, pain, sadness, injustice, death, etc. – doesn’t hinge on whether we’re simulated. Moreover, it’s not clear that the Simulation Hypothesis systematically undermines our knowledge. Some philosophers have argued that most of our quotidian and scientific beliefs, like the beliefs that I am sitting in a chair and that chairs are made of atoms, are compatible with the Simulation Hypothesis because the hypothesis is best construed as a claim about what physical things are made of at a fundamental level. If we’re simulated, the thinking goes, there are still chairs and atoms. It’s just that atoms (and by extension chairs) are composed of patterns of bits in a computer.

On the other hand, the Simulation Hypothesis seems to significantly increase the likelihood of a range of unsettling possibilities.

Suppose a scientist today wants to simulate two galaxies colliding. There is no need to run a complete simulation of the universe. Starting and ending the simulation a few billion years before and after the collision may be sufficient. Moreover, it’s unnecessary and infeasible to simulate distant phenomena or every constituent of the colliding galaxies. A coarse-grained simulation containing only large celestial phenomena within the colliding galaxies may work just fine.

Similarly, the simulation we live in might be incomplete. If humans are the primary subjects of our simulation, then it might only be necessary to continuously simulate our immediate environment at the level of macroscopic objects, whereas subatomic and distant phenomena could be simulated on an ad hoc basis (just as hidden-surface removal is used to reduce computational costs in graphics programming).

More disturbingly, it’s possible that our universe or our lives are much shorter-lived than we think. For example, suppose our simulators are interested in a particular event, such as a nuclear war or an AI takeover, that will happen tomorrow. Alternatively, suppose our simulators are interested in figuring out if a simulated being like you who encounters the Simulation Hypothesis can discover they are in a simulation. It would have made sense relative to these sorts of purposes to have started our simulation recently, perhaps just a few minutes ago. It would be important for simulated beings in this scenario to think they have a longer history, remember doing things yesterday, and so on, but that would be an illusion. Worse, it would make sense to end our simulation after the event in question has run its course. That would likely mean that we will all die much sooner than we expect. For example, if you finish this article and dismiss it as a frivolous distraction, resolving never again to think about the Simulation Hypothesis, our simulators might decide that they’ve gotten their answer–a person like you can’t figure out they’re in a simulation–and terminate the simulation, destroying our universe and you with it.

Yet another disturbing possibility is that our simulation contains fewer minds than it seems. It could be that you are the sole subject of the simulation and consequently yours is the only mind that is simulated in detail, while other “humans” are merely programmed to act in convincing ways when you’re around, like non-playable characters. Alternatively, it could be that humans are simulated in full detail, but animals aren’t.

Now, the Simulation Hypothesis doesn’t entail that any of these things are true. We could be in a complete simulation. Plus, these things could be true even if we aren’t simulated. Philosophers have long grappled with solipsism, and Bertrand Russell once discussed the possibility that the universe sprung into existence minutes ago. However, there doesn’t seem to be any available explanation as to why these things might be true if we aren’t simulated. But there is a readily available explanation as to why these things might be true if we are in a simulation: Our simulators want to save time or reduce computational costs. This suggests that the Simulation Hypothesis should lead us to raise our credence in these possibilities. By analogy, a jury should be more confident that a defendant is guilty if the defendant had an identifiable motive than if not, all else being equal.

What are the ethical implications of these possibilities? The answer depends on how likely we take them to be. Since they are highly speculative, we shouldn’t assign them a high probability. But I don’t think we can assign them a negligible probability, either. My own view is that if you think the simulation argument is plausible, you should think there’s at least a .1% chance that we live in some sort of significantly incomplete simulation. That’s a small number. However, as some philosophers have noted, we routinely take seriously less likely possibilities, like plane crashes (<.00003%). This kind of thing is sometimes irrational. But sometimes assigning a small probability to incompleteness possibilities should make a practical difference. For example, it should probably produce slight preferences for short-term benefits and egocentric actions. Perhaps it should even lead you to take your past commitments less seriously. If it’s Saturday night and you can’t decide between going to a bar or initiating a promised snail-mail correspondence with your lonely cousin, a small chance that the universe will end very soon, that yours is the only mind in the universe, or that you never actually promised your cousin to write should perhaps tip the scales towards the bar. Compare: If you really believed there’s at least a 1/1000 chance that you and everyone you love will die tomorrow, wouldn’t that reasonably make a practical difference?

The Simulation Hypothesis has other sorts of ethically relevant implications. Some people argue that it creates avenues for fresh approaches to old theological questions, like the question of why, if there is a God, we see so much evil in the world. And while the Simulation Hypothesis does not entail a traditional God, it strongly suggests that our universe has a creator (the transhumanist David Pearce once described the simulation argument as “perhaps the first interesting argument for the existence of a Creator in 2000 years”). Unfortunately, our simulator need not be benevolent. For all we know, our universe was created by “a sadistic adolescent gamer about to unleash Godzilla” or someone who just wants to escape into a virtual world for a few hours after work. To the extent this seems likely, some argue that it’s prudent to be as “funny, outrageous, violent, sexy, strange, pathetic, heroic, …in a word ‘dramatic’” as possible, so as to avoid boring a creator who could kill us with the click of a button.

It may be that the Simulation Hypothesis is, as one author puts it, “intrinsically untethering.” Personally, I find that even under the most favorable assumptions the Simulation Hypothesis can produce deliciously terrible feelings of giddiness and unsettlement. And yet, for all its power, I do not believe it inhibits my ability to flourish. For me, the key is to respond to these feelings by plunging head first into the pleasures of life. Speaking of his own escape from “the deepest darkness” of uncertainty brought on by philosophical reasoning, the 18th-century philosopher David Hume once wrote:

Most fortunately it happens, that since reason is incapable of dispelling these clouds, nature herself suffices to that purpose, and cures me of this philosophical melancholy and delirium, either by relaxing this bent of mind, or by some avocation, and lively impression of my senses, which obliterate all these chimeras. I dine, I play a game of backgammon, I converse, and am merry with my friends; and when after three or four hours’ amusement, I would return to these speculations, they appear so cold, and strained, and ridiculous, that I cannot find in my heart to enter into them any farther.

I suppose that, ideally, we should respond to the simulation argument by striving to create meaning that doesn’t rely on any particular cosmology or metaphysical theory, to laugh at the radical precarity of the human condition, to explore The Big Questions without expecting The Big Answers, to make peace with our pathetic lives, which might, in the final analysis, be wholly contained within some poor graduate student’s dusty computer. But that stuff is pretty hard. When all else fails, Hume’s strategy is available to us: Do fun stuff. Have stimulating conversations; eat tasty food; drink fine wine; play exciting games; read thoughtful books; watch entertaining movies; listen to great music; have pleasurable sex; create beautiful art. Climb trees. Pet dogs. The Simulation Hypothesis will probably start to feel ridiculous, at least for a while. And, fortunately, this is all worth doing regardless of whether we’re in a simulation.

Posted on

Academic Work and Justice for AIs

drawing of robot in a library

As the U.S. academic year draws to a close, the specter of AI-generated essays and exam answers looms large for teachers. The increased use of AI “chatbots” has forced a rapid and fundamental shift in the way that many schools are conducting assessments, exacerbated by the fact that – in a number of cases – they have been able to pass all kinds of academic assessments. Some colleges are now going so far as to offer amnesty for students who confess to cheating with the assistance of AI.

The use of AI as a novel plagiarism tool has all kinds of ethical implications. Here at The Prindle Post, Richard Gibson previously discussed how this practice creates deception and negatively impacts education, while D’Arcy Blaxell instead looked at the repetitive and homogeneous nature of the content they will produce. I want to focus on a different question, however – one that, so far, has been largely neglected in ethical discussions of the ethics of AI:

Does justice demand that AIs receive credit for the academic work they create?

The concept of “justice” is a tricky one. Though, at its simplest, we might understand justice merely as fairness. And many of us already have an intuitive sense of what this looks like. Suppose, for example, that I am grading a pile of my students’ essays. One of my students, Alejandro, submits a fantastic essay showing a masterful understanding of the course content. I remember, however, that Alejandro has a penchant for wearing yellow t-shirts – a color I abhor. For this reason (and this reason alone) I decide to give him an “F.” Another student of mine, Fiona, instead writes a dismal essay that shows no understanding whatsoever of anything she’s been taught. I, however, am friends with Fiona’s father, and decide to give her an “A” on this basis.

There’s something terribly unfair – or unjust – about this outcome. The grade a student receives should depend solely on the quality of their work, not the color of their T-shirt or whether their parent is a friend of their teacher. Alejandro receives an F when he deserves an A, while Fiona receives an A when she deserves an F.

Consider, now, the case where a student uses an AI chatbot to write their essay. Clearly, it would be unjust for this student to receive a passing grade – they do not deserve to receive credit for work that is not their own. But, then, who should receive credit? If the essay is pass-worthy, then might justice demand that we award this grade to the AI itself? And if that AI passes enough assessments to be awarded a degree, then should it receive this very qualification?

It might seem a preposterous suggestion. But it turns out that it’s difficult to explain why justice would not claim as much.

One response might be to say that the concept of justice don’t apply to AIs because AIs aren’t human. But this relies on the very controversial assumption that justice only applies to Homo sapiens – and this is a difficult claim to make. There is, for example, a growing recognition of the interests of non-human animals. These interests make appropriate the application of certain principles of justice to those animals, arguing – for example – that it is unjust for an animal to suffer for the mere amusement of a human audience. Restricting our discussions of justice to humans would preclude us from making claims like this.

Perhaps, then, we might expand our considerations of justice to all beings that are sentient – that is, those that are able to feel pain and pleasure. This is precisely the basis of Peter Singer’s utilitarian approach to the ethical treatment of animals. According to Singer, if an animal can experience pleasure, then it has an interest in pursuing pleasure. Likewise, if something can experience pain, then it has an interest in avoiding pain. These interests then form the basis of ways in which it is just or unjust to treat not just humans, but non-human animals too. AIs are not sentient (at least, not yet) – they can experience neither pain nor pleasure. This, then, might be an apt basis on which to exclude them from our discussions of justice. But here’s the thing: we don’t want to make sentience a prerequisite for justice. Why not? Because there are many humans who also lack this feature. Consider, for example, a comatose patient or someone with Congenital Pain Insensitivity. Despite the inability of these individuals to experience pain, it would seem unjust to, say, deprive them of medical treatment. Given this, then, sentience cannot be necessary for the application of justice.

Consider, then, a final alternative: We might argue that justice claims are inapplicable to AIs not because they aren’t human or sentient, but because they fail to understand what they write. This is a perennial problem for AIs, and is often explained in terms of the distinction between the syntax (structure) and semantics (meaning) of what we say. Computer programs – by their very nature – run on input/output algorithms. When, for example, a chatbot receives the input “who is your favourite band?” it is programmed to respond with an appropriate output such as “my favorite band is Rage Against the Machine.” Yet, while the structure (i.e., syntax) of this response is correct, there’s no meaning (i.e., semantics) behind the words. It doesn’t understand what a “band” or a “favorite” is. And when it answers with “Rage Against the Machine”, it is not doing so on the basis of its love for the anarchistic lyrics of Zach de la Rocha, or the surreal sonifications of guitarist Tom Morello. Instead, “Rage Against the Machine” is merely a string of words that it knows to be an appropriate output when given the input “who is your favourite band?” This is fundamentally different to what happens when a human answers the very same question.

But here’s the thing: There are many cases where a student’s understanding of a concept is precisely the same as an AI’s understanding of Rage Against the Machine.

When asked what ethical theory Thomas Hobbes was famous for, many students can (correctly) answer “Contractarianism” without any understanding of what that term means. They have merely learned that this is an appropriate output for the given input. What an AI does when answering an essay or exam question, then, might not be so different to what many students have done for as long as educational institutions have existed.

If a human would deserve to receive a passing grade for a particular piece of academic work, then it remains unclear why justice would not require us to award the same grade to an AI for the very same work. We cannot exclude AIs from our considerations of justice merely on the basis that they lack humanity or sentience, as this would also require the (unacceptable) exclusion of many other beings such as animals and coma patients. Similarly, excluding AIs on the basis that they do not understand what they are writing would create a standard that even many students would fall short of. If we wish to deny AIs credit for their work, we need to look elsewhere for a justification.

Posted on

ChatGPT and Emotional Outsourcing

Plenty of ink has been spilled concerning AI’s potential to plagiarize a college essay or automate people’s jobs. But what about writing that’s meant to be more personal?

Take for example the letter Vanderbilt sent to their students after the shooting at Michigan State University. This letter expresses the administration’s desire for the community to “reflect on the impact of such an event and take steps to ensure that we are doing our best to create a safe and inclusive environment.” It was not written by a human being.

The letter was written by an AI tool called ChatGPT, which is a user-friendly large language model (LLM). Similar to predictive text on your phone, ChatGPT is trained on a large body of text to produce sentences by selecting words that are likely to come next.

Many people were upset to learn that Vanderbilt’s letter was written using ChatGPT — so much so that the administration issued an apology. But it’s not clear what exactly was worth apologizing for. The content expressed in the original letter was not insincere, nor was it produced illegally. Nothing about the wording was objectionable.

This case raises questions about tasking AI with what I’ll call emotional writing: writing that is normally accompanied by certain emotions.

Examples include an apology, an offer of support, a thank you note, a love letter. What exactly is the source of unease when a human being off-loads emotional writing to an AI model? And does that unease point to something morally wrong? When we consider a few related cases, I think we’ll find that the lack of a human author is not the main concern.

Let’s start by noting that the normal writing process for a university letter is similar to the process ChatGPT uses. Normally, someone within the administration might be asked to write the first draft. That person researches similar letters, using them as a guide. This draft is then vetted, edited lightly as necessary, and sent to the campus community. It’s natural to think that the main difference is that there’s a human at one end of the process in the normal case, and not (or not really) in the ChatGPT case.

Will any human do? Consider other cases where emotional writing is done by someone outside the situation. A highschooler gets their mom to write an apology for them. A university pays a freelancer to express sympathy for its students. A man with no game hires Will Smith to tell him what to say to his crush. In these cases as well, the recipient of the speech might be reasonably disappointed to discover the source of the words.

These considerations suggest that what’s objectionable in the AI case is not specifically the lack of a human author. The problem is that the author is not bound up in the relationship for which the words are written.

What all these cases have in common is that they involve emotional outsourcing: someone avoiding an emotional task by giving it to someone (or something) else. In these cases, the deeply personal writing becomes a kind of mercenary task.

Surprisingly, even having the right person write the text may not be enough to avoid this problem! Suppose someone writes a love letter to their romantic partner, and after their breakup reuses the letter by sending it to someone new. I would be peeved. Wouldn’t you? The emotional work has been done by the right person, but not with the right aim; not with the current recipient in mind. The work has been outsourced to the writer’s prior self.

There are a couple aspects of emotional outsourcing that might seem problematic. First, outsourcing emotional writing draws attention to the fact that much of our communication is socially scripted. If even a well-trained computer model can perform the task, then that task is shown to be formulaic. In a society that prizes individuality and spontaneity as signs of authenticity, relying on a formula can seem subpar. (Consider how you might react if a person used a template for a letter of condolences: “Dear [recipient], We offer our [sincerest / most heartfelt / deepest] [condolences / sympathies] in the wake of the [tragedy / tragic event / tragic events /atrocity] of [month, day].”)

I think objecting to this feature of emotional outsourcing is a mistake. Social scripts are to some extent unavoidable, and in fact they make possible many of the actions we perform with our speech. The rule not to draw attention to the script is also ableist, insofar as it disadvantages neurodivergent people for whom explicitly-acknowledged social scripts can be more hospitable. While drawing attention to the formulaic nature of the communication is a taboo — and that partly explains people’s disapproval of emotional outsourcing — that’s not enough to make emotional outsourcing morally objectionable.

The second issue is more problematic: emotional outsourcing misses some of the action behind the speech that gives the speech its meaning. Language not only means things; it also does things. A promise binds. A statement asserts. An apology repairs. (Often the action speech performs is limited by what is taken up by the audience. I can say “I do” as often as I’d like, but I haven’t married someone unless that person accepts it.)

Emotional writing performs specific actions — consoling, thanking, wooing — not only through the words it uses. It also performs those actions in part through the act that produces those words.

Writing out a thank you note is itself an act of appreciation. Thinking through how to express care for your community is itself an act of care. Putting words to your love is itself an act of love.

Part of what makes the words meaningful is lost when those prior actions are absent — that is, when someone (or something) else produces them. People often say with respect to gestures of kindness, “it’s the thought that counts.” When ChatGPT is used for emotional writing, at least some of that thought is missing.

Keeping these issues in mind, it’s worth asking whether outsourcing emotional writing to AI is entirely bad. Thinking deeply about grief can put people in a challenging place emotionally. It could trigger past trauma, for example. Could it be a mercy to the person who would otherwise be tasked with writing a sympathy letter to leave the first draft to an LLM that feels nothing? Or is it appropriate to insist that a human feel the difficult emotions involved in putting words to sympathy?

There may also be cases where a person feels that they are simply unable to express themselves in a way that the other person deserves. Seeking outside help in such a case is understandable — perhaps even an act of care for the recipient.

I have argued that emotional outsourcing is an important part of what people find objectionable about tasking AI with emotional writing. Emotional outsourcing draws attention to the formulaic nature of communication, and it can mean missing out on what counts. However, much remains to be explored about the moral dimensions of emotional outsourcing, including what features of a case, if any, could make moral outsourcing the best choice.

Posted on

A Right To Attentional Freedom?

collage of various people on their phones

The White House recently posted a proposal for an AI Bill of Rights. In California, there is a bill that aims to hold social media companies accountable for getting young children addicted to their platforms. Several of these companies also face a federal lawsuit for emotionally and physically harming their users.

For those who use technology on a day-to-day basis, these developments are likely unsurprising. There is an intuition, backed by countless examples, that our technology harms us and that those who have created the technology are somehow responsible. Many of us find ourselves doomscrolling or stuck on YouTube for hours because of infinite scrolling.

Less settled is precisely how these technologies are bad for us and how exactly these companies wrong us.

The California bill and the lawsuit both argue that one notable form of harm can be understood through the lens of addiction. They argue that social media companies are harming a particularly vulnerable group, namely young adults and children, by producing an addicting product.

While this way of understanding the problem certainly has plausibility, one might favor other ways of explaining the problem. The way that we frame the moral relationship users have with technology will shape legal argumentation and future regulation. If our aim is to forge a morally sound relationship between users, technology, and producers, it is important to get the moral story right.

What makes social media addicting is the fact that it has become especially adept at producing content that users want to engage with. Complex algorithms learn about its user’s predilections and can accurately predict the kinds of things people want to see. The ability for AI to manipulate us so effectively highlights our failure to recognize the importance of attention – a valuable good that has gone underappreciated for far too long.

First, our attention is limited. We cannot attend to everything before us and so each moment of attention is accompanied with non-attention. If I am paying attention to a film, then I am not paying attention to the cars outside, or the rain falling, or the phone in my pocket.

Second, attention is susceptible to outside influence. If someone is talking loudly while a film plays, I may become distracted. I may want to watch the film closely, but the noise pulls my attention away.

Third, attention is related to many foundational moral rights. Take for instance freedom of thought. We might think that in a society where there are no laws about what you are allowed to think, read, or say guarantees the freedom of thought. However, unless your attention is respected, freedom of thought cannot be secured.

We need only think of Kurt Vonnegut’s story “Harrison Bergeron” to show what this claim misses. In it, Harrison Bergeron lives in a society that goes to great lengths to ensure equality. In order to make sure everyone remains equal, those who are born with natural talents are given artificial burdens. For Harrison, who is exceptional both physically and mentally, one particularly clever tactic is used to ensure he does not think too much. Periodically, a loud, harsh sound is played through an earpiece. This makes it impossible for Harrison to focus.

The relevant point here is that even if no law exists that prohibits you from thinking whatever you please, reading what you want, or discussing what you wish, your freedom of thought can be indirectly overridden.

By utilizing the fact that your attention is limited and not fully voluntary, another party can prevent you from thinking freely. Thus, although our rights may be respected on paper, assaults on our attention may inhibit us from utilizing the capacities these rights are supposed to protect in practice.

When we interact with technology, we must give our attention over to it. Furthermore, much of the technology we interact with on a day-to-day basis is designed specifically to maintain and increase user engagement. As a result of these design choices, we have developed technology that is highly effective at capturing our attention.

As predictive technology improves, machines will also improve their ability to distract us. The result of this will mean that more people will spend more time using the technology (e.g., watching videos, reading news pieces, viewing content produced by other users). The more time people spend using this technology, the less they can spend attending to other things.

If our attention is limited, can be controlled from the outside, and is vital for utilizing other morally important capacities, it seems clear that it is something that should be treated with respect.

Consider how we tend to think that it is rude to distract someone while they are trying to concentrate. It rarely feels satisfying if the person causing the distraction simply replies “Just ignore me.” This response denies a crucial reality of the nature of attention, viz., it is often non-voluntary.

Furthermore, it would be even worse if the distracting person tried to mask their presence and distract someone secretly, and yet this is precisely what a great deal of our technology does. It exploits the non-voluntary nature of our attention, overrides attentional freedom, and does so in the most discrete way possible. Technology could be designed in a way that respected our attentional freedom, instead of covertly trying to undermine it. For example, periodically prompting the user to consider doing something else, instead of endlessly presenting more content to engage with.

Rather than focusing on technology’s tendency to encourage addictive behavior in young people, I would like us to think about the effects technology has on all users’ attentional freedom.

Technology that is designed to distract you is harmful because it overrides your attentional freedom. When you use this technology, you are less free. This analysis must overcome at least two challenges, both centered around consent.

The first is that we consent to use these products. To argue that my phone wrongfully harms me because it is distracting seems like arguing that a book wrongfully harms me if it is so gripping that I cannot put it down.

However, while a book may be enticing and may even be created with the hopes that it captures attention, the book does not learn about what captures attention. There is a difference between something capturing your attention because it is interesting and something that learns your preferences and sets about satisfying them. What makes AI driven technology unique is that it has the capacity to fine tune the kinds of things it offers you in real time. It knows what you click on, what you watch, and how long you engage. It also relies on the involuntary part of attention to keep you engaged.

The second argument is about general human interaction. If it is wrong to affect someone’s attention, then daily interactions must be wrong. For instance, if someone walks down the street and asks me to take a flier for a show, do they wrong me by distracting me? Do all interactions require explicit consent lest they be moral violations? If our moral analysis of attention forces us to conclude that even something as trivial as a stranger saying hello to you constitutes a moral wrong because it momentarily distracts you, we will have either gone wrong somewhere along the way, or else produced a moral demand that is impossible to respect.

To answer this second objection, one thing we can say is this. When someone distracts you, they do not necessarily wrong you. Someone who tries to hand you a flier in the street effectively asks for your attention, and you have the opportunity to deny this request with fairly little effort. Notably, if the person who asks for your attention continues to pester you, and follows you down the road as you walk, their behavior no longer seems blameless and quickly turns into a form of harassment. When someone intentionally tries to override your attentional freedom, the moral problem emerges. Because attentional freedom is connected to a set of important freedoms (e.g., freedom of thought, freedom of choice, etc.), if one can override another’s attentional freedom, they can override other important freedoms indirectly.

If technology harms us because we become addicted to it, then we have reason to protect children from it. We may even have reason to provide more warnings for adults, like we do with addictive substances. However, if we stop our analysis at addiction, we miss something important about how this technology operates and how it harms us. When we see that technology harms us because it overrides our attentional freedom, we will need to do more than simply protect children and warn adults. Several new questions emerge: Can we design technology to preserve attentional freedom, and if so, what changes should we make to existing technology? How can we ensure that technology does not exploit the non-voluntary part of our attention? Are some technologies too effective at capturing our attention, such that they should not be on the market? Is there a right to attentional freedom?

Posted on

ChatGPT and the Challenge of Critical (Un)Thinking

photograph of statue of thinking man

For the past weeks there has been a growing interest on ChatGPT, this new artificial intelligence language model that was “programmed to communicate with people and provide helpful responses.” I was one of the curious that had to try it and figure out why everyone was talking about it.

Artificial intelligence is not a new thing; at least as an idea it has some decades now, since it was firstly introduced in 1950 by Alan Turing, the British mathematician who is generally considered to be the father of computer science. Later on, in 1956, John McCarthy coined the term “artificial intelligence” in a conference, giving birth to a new field of study. Today, it is everywhere, we use it even without knowing and the advancements in the area create entirely new fields of inquiry, bringing along new ethical dilemmas that go from the discussion what (if any) moral rights to attribute to A.I., to designing new digital rights that encompass different milieus and that have political and legal consequences – see, for instance, the European Union attempts since 2021 to create a legal framework regarding the rights and regulations of AI for its use on the continent.

ChatGPT is something unique – at least for now. While a recent development, it seems almost too familiar – as if it was always there, just waiting to be invented. It is a Google search on steroids, with much more complexity in its answers and a “human” touch. Once you read the answers to your questions, what calls your attention is not only how fast the answer is provided, but also how detailed it seems to be. It mimics pretty well our ways of thinking and communicating with others. See, for instance, what happened when staff members at Vanderbilt University used it to write an email responding to the shooting at Michigan State – a well written 297-word missive which might otherwise have been well received. However, the fact that at the bottom of the email was a line that read as following: “Paraphrase from OpenAI’s ChatGPT AI language model, personal communication, February 15, 2023,” outraged the community. The Associate Dean of the institution soon apologized, saying that the use of the AI-written email contradicted the values of the institution. This is one (of no doubt many) examples of how the use of this technology may disrupt our social and cultural grids. This new tool brings new challenges, not only for education – how students and professors incorporate this technique into their practices – but also for ethics.

Contemporary models of education still rely heavily on regular evaluation – a common mission across educational institutions is to foster critical thinking and contribute to the development of active and responsible citizens. Why is critical thinking so valued? Because being reflective – thinking about the reasons why you act and think the way you do – is necessary for fully participating in our social world. Learning is a process through which we form our judgment and in doing so, build our moral identities – who we are and what we value. To judge something is not as easy as it may initially seem, for it forces each of us to confront our prejudices, compare it to reality – the set of facts common to all of us, what the world is made up – and take a stand. This process also moves us from our inner monologue with our self to a dialogue with others.

What happens when students rely more and more on ChatGPT to do their homework, to write their essays and to construct their papers? What happens when professors use it to write their papers or books or when deans of universities, like the example mentioned above, use it to write their correspondence? One could say that ChatGPT does not change, in essence, the practices already in place today, given the internet and all the search engines. But insofar as ChatGPT is superior in mimicking the human voice, might its greatest danger lie in fostering laziness? And shouldn’t we consider this laziness a moral vice?

In the Vanderbilt case, what shocked the community was the lack of empathy. After all, delegating this task to AI could be interpreted as “pretending to care” but fooling the audience. To many it seems a careless shortcut done for time’s sake. Surely it shows poor judgment; it just feels wrong. It seems to betray a lack of commitment to the purpose of education – the dedication to examine and think critically. In this particular context, technological innovation appears nothing more than a privileged means to erode what was supposed to contribute to, namely, thoughtful reflection.

While technologies tend to make our life much more comfortable and easier, it’s worth remembering that technologies are a means to something. As Heidegger well pointed out in an emblematic text entitled “The Question concerning Technology” (1954), we tend to let ourselves be charmed and hypnotized by its power; while forgetting the vital question of purpose – not the purpose of technology but the purpose of our lives, as humans. And while ChatGPT may be great for providing context and references on virtually any topic of research, we cannot forget that the experience of conscious thinking is what makes us uniquely human. Despite all appearances of coherent and well-ordered prose, ChatGPT is only mirroring what we, humans, think. It still does not have nor can mimic one thing: our emotions and our ability to respond in a singular manner to specific situations.

If we generalize and naturalize the use of this kind of technologies, incorporating into our daily lives, aren’t we making a choice of non-thinking in detriment of an instantaneous response that serves a strict utilitarian purpose? Heidegger says that “technology is a mode of revealing,” insofar what we choose (or do not choose) reveals the ways in which we are framing our world. And if we choose not to think – believing that something else can “mirror” our possible thought – aren’t we abdicating of our moral autonomy, suspending the human task of reflecting, comparing, and judging, and instead embracing a “dogmatic” product of a technological media?

Posted on

ChatGPT: The End of Originality?

photograph of girl in mirror maze

By now, it has become cliché to write about the ethical implications of ChatGPT, and especially so if you outsource some of the writing to ChatGPT itself (as I, a cliché, have done). Here at The Prindle Post, Richard Gibson has discussed the potential for ChatGPT to be used to cheat on assessments, while universities worldwide have been grappling with the issue of academic honesty. In a recent undergraduate logic class I taught, we were forced to rewrite the exam when ChatGPT was able to offer excellent answers to a couple of the questions – and, it must be said, completely terrible answers to a couple of others. My experience is far from unique, with professors rethinking assessments and some Australian schools banning the tool entirely.

But I have a different worry about ChatGPT, and it is not something that I have come across in the recent deluge of discourse. It’s not that it can be used to spread misinformation and hate speech. It’s not that its creators OpenAI drastically underpaid a Kenyan data firm for a lot of the work behind the program only weeks before receiving a $10 billion investment from Microsoft. It’s not that students won’t learn how to write (although that is concerning), the potential for moral corruption, or even the incredibly unfunny jokes. And it’s certainly not the radical change it will bring.

It’s actually that I think ChatGPT (and programs of its ilk) risks becoming the most radically conservative development in our lifetimes . ChatGPT risks turning classic FM radio into a framework for societal organization: the same old hits, on repeat, forever. This is because in order to answer prompts, ChatGPT essentially scours the internet to predict

“the most likely next word or sequence of words based on the input it receives.” -ChatGPT

At the moment, with AI chatbots in their relative infancy, this isn’t an issue – ChatGPT can find and synthesize the most relevant information from across the web and present it in a readable, accessible format. And there is no doubt that the software behind ChatGPT is truly remarkable. The problem lies with the proliferation of content we are likely to see now that essay writing (and advertising-jingle writing, and comedy-sketch writing…) is accessible to anybody with a computer. Some commentators are proclaiming the imminent democratization of communication while marketers are lauding ChatGPT for its ability to write advertising script and marketing mumbo-jumbo. On the face of it, this development is not a bad thing.

Before long, however, a huge proportion of content across the web will be written by ChatGPT or other bots. The issue with this is that ChatGPT will soon be scouring its own content for inspiration, like an author with writer’s block stuck re-reading the short stories they wrote in college. But this is even worse, because ChatGPT will have no idea that the “vast amounts of text data” it is ingesting is the very same data it had previously produced.

ChatGPT – and the internet it will engulf – will become a virtual hall of mirrors, perfectly capable of reflecting “progressive” ideas back at itself but never capable of progressing past those ideas.

I asked ChatGPT what it thought, but it struggled to understand the problem. According to the bot itself, it isn’t biased, and the fact that it trains on data drawn from a wide variety of sources keeps that bias at bay. But that is exactly the problem. It draws from a wide variety of existing sources – obviously. It can’t draw on data that doesn’t already exist somewhere on the internet. The more those sources – like this article – are wholly or partly written by ChatGPT, the more ChatGPT is simply drawing from itself. As the bot admitted to me, it is impossible to distinguish between human- and computer-generated content:

it’s not possible to identify whether a particular piece of text was written by ChatGPT or by a human writer, as the language model generates new responses on the fly based on the context of the input it receives.

The inevitable end result is an internet by AI, for AI, where programs like ChatGPT churn out “original” content using information that they have previously “created.” Every new AI-generated article or advertisement will be grist for the mill of the content-generation machine and further justification for whatever data exists at the start of the cycle – essentially, the internet as it is today. This means that genuine originality and creativity will be lost as we descend into a feedback loop of increasingly sharpened AI-orthodoxy; where common-sense is distilled into its computerized essence and communication becomes characterized by adherence. The problem is not that individual people will outsource to AI and forget how to be creative, or even that humanity as a whole will lose its capacity for ingenuity. It’s that the widespread adoption of ChatGPT will lead to an internet-wide echo chamber of AI-regurgitation where chatbots compete in an endless cycle of homogenization and repetition.

Eventually I was able to get ChatGPT to respond to my concerns, if not exactly soothe them:

In a future where AI-generated content is more prevalent, it will be important to ensure that there are still opportunities for human creativity and original thought to flourish. This could involve encouraging more interdisciplinary collaborations, promoting diverse perspectives, and fostering an environment that values creativity and innovation.

Lofty goals, to be sure. The problem is that the very existence of ChatGPT militates against them: disciplines will die under the weight (and cost-benefits) of AI; diverse perspectives will be lost to repetition; and an environment that genuinely does value creativity and innovation – the internet as we might remember it – will be swept away in the tide of faux-progress as it is condemned to repeat itself into eternity. As ChatGPT grows its user base faster than any other app in history and competitors crawl out of the woodwork, we should stop and ask the question: is this the future we want?

Posted on

ChatGPT and Deception

photograph of exams stacked on desk at front of class

For many, writing can be a challenging, even painful, experience. What seems like it should be a simple job – taking the thoughts in your head, turning them into words, and then typing them out – can be a task of almost insurmountable odds for some. Heck, I do it for a living, and I still find the blank page to be an ungodly sight, one that I sometimes flee from and promise to tackle later on.

As such, it should be no surprise that the advent of an AI system that can write for us has caused quite a stir. That system is ChatGPT, and while I could explain what it is, it’s more effective (not to mention easier) to let the tool do that for itself:

ChatGPT is a state-of-the-art language generation model developed by OpenAI. It is based on a transformer architecture and pre-trained on a large dataset of text, allowing it to generate human-like text.

ChatGPT can be fine-tuned for various natural language processing tasks such as language translation, text summarization, question answering, and text completion. It can also be used to generate creative writing, poetry, and even code.

One of the key features of ChatGPT is its ability to generate text that is coherent and coherent with the context it was given, making it able to generate text that flows well and makes sense. ChatGPT can be used to generate text on a wide range of topics and can be fine-tuned to suit specific use cases.

It is important to note that ChatGPT is a machine learning model, it is not capable of independent thinking or decision making, it is only able to generate text based on patterns it has learned from the data it was trained on.

I didn’t write any of that. Instead, I gave ChatGPT the prompt “explain what ChatGPT is” and let it fly. As it just noted, however, ChatGPT is not limited to dry, explanatory statements. The system has demonstrated an apparent aptitude for creative, even funny, writing, from explaining quantum theory to a child in the style of Snoop Dogg to creating an analogy for what it’s like to eat the keto diet, from giving an account of Attack on Titan in the manner of Donald Trump to writing a biblical verse explaining how to remove a peanut butter sandwich from a VCR. The tool really does seem adaptable.

Yet, despite the hilarity, ChatGPT’s emergence has brought some pressing issues regarding ownership and authenticity of work to the fore. If an AI generates text for you, can you claim it as your own? For example, Ammaar Reshi is facing considerable backlash for using ChatGPT to write a children’s book (which he then illustrated using Midjorney, an AI art generator). Reshi did not directly write or illustrate the book he is claiming as his product; he gave ChatGPT the required prompts and then used its output.

But, it has been in the educational sector where such concerns have really taken hold. So much so that some, such as New York City’s Department of Education, have blocked access to ChatGPT on school devices for fear of its misuse. The problems are relatively easy to grasp:

What is stopping students from passing off ChatPGT-produced essays and other forms of assessed work as their own? How should educators respond if a student uses ChatGPT to write an essay? And are students actually doing anything wrong if they use ChatGPT like this?

The answer to this last question is vastly complex and intertwined with the very purpose of assessment and learning monitoring. The point of assigning assessments, such as essays, is not so students produce a piece of text. The production of the essay is merely a step towards another goal. These forms of assessment act as a representation of the students’ learning. When a teacher asks you to write a 3,000-word paper on Frederick Douglas, for example, it is not the paper with which they are concerned; it is with your ability to recall, appraise, and communicate what you know about Douglas’ life, work, and impact. The essay is a medium through which such appraisal is conducted.

As philosopher Rebecca Mace remarked in an episode of BBC’s Inside Science:

A lot of people, including the newspapers, seem to have misunderstood the point of homework. So the purpose of homework is not to produce an essay, but to assess student understanding in order that the teachers can assist them with the gaps, or work out what they’ve not taught very well, or what they maybe need to go over again, or what that individual student really needs help with. Then the essay itself is irrelevant in many ways because that’s all the essay’s doings; it’s a means to an end.

Thus, according to such a way of thinking, the danger of ChatGPT comes from its potential to misrepresent student learning, giving the impression that a student knows more about a subject than they actually do. The issue is not one of principle but of outcome, and the use of ChatGPT brings with it the risk that learning is negatively impacted.

This stance, however, seems to overlook something important in using ChatGPT in educational settings. If accurate – if the threat of ChatGPT comes from its capacity to hide academic failings (both on the student’s and teacher’s behalf) – then we shouldn’t have any qualms about it being used in situations where this isn’t a factor. But, academically gifted students who know their subjects inside and out still seem to commit some wrong when they pass the algorithmically-generated text off as their own. This wrong emerges not from the impact such usage might have on their academic performance, nor on their teacher’s ability to assess their grasp of a subject accurately, but from the fact that they are attempting to deceive their assessor. It is wrong not because of an outcome but because of an adherence to principles – the virtue of honesty and the vice of deception.

That is not to say that this is the only reason why ChatGPT presents a potential harm to education and educational practices. The use of AI to game the academic-assessment system by hiding one’s failure to meet the standards are most certainly a concern (perhaps the central one). But, such an acknowledgement should not lead us to overlook the fact that, much like plagiarism, academic wrongs don’t simply emerge from their deleterious impact. They also come from deception and attempting to pass something off as one’s work when, in fact, they had minimal input in its creation.

Posted on

Man v. Machine: The Ethics of AI Art

The creation of digital art is nothing new, but advances in artificial intelligence have created a novel environment where all sorts of media can now be created without much human input. When Jason M. Allen won the Colorado State Fair for his piece “Théâtre d’Opéra Spatial” it sparked fierce debate about the nature of art and the merits of AI creations. But we should be careful not to confuse the many ethical issues involved. Is the outcry about the fairness of contests or about the future of art?

Allen, a video game designer, created his entry using an AI called Midjourney. It works by creating images from textual descriptions. Allen claims that he created hundreds of images before selecting only three. He then made some additional adjustments using Photoshop and boosted their resolution with a tool called Gigapixel. He entered the Colorado State Fair under the digital arts category which is defined as “artistic practice that uses digital technology as part of the creative or presentation process.” Allen claims that he informed the competition that the image was created with AI. The competition’s judges, meanwhile, say that they were not aware. Nevertheless, they said they would still have given it first place based on the piece itself.

Online reaction was harsh.

While the uproar isn’t surprising, it’s not clear that everyone has the same objections for the same reasons.

Let’s address the first major ethical question which centers around the contest: Was it wrong of Allen to submit the AI created work and be awarded the blue ribbon over other artists who created their works by hand? The contest’s definition of digital arts was sufficiently broad enough that AI created works were eligible. The work was entered using the name “Jason M. Allen via Midjourney.” Also, according to Allen, this was not simply a case of a few button presses, but 80 hours of work – tweaking the prompts to get the image he wanted and making a selection out of 900 iterations. While Allen spent his time differently than the other artists, this doesn’t mean that creating the image lacked skill, effort, or the aesthetic taste.

On the other hand, others might object that it was wrong for Allen to enter the contest since he was not the artist; it was the artificial intelligence who actually created the piece of art. Did the AI create the work, or is the AI merely a tool for Allen – the true creator – to manipulate?

The judges selected this particular work because of the impact it had on them, and Allen was deliberately attempting to tie together the themes that the painting conveys. The AI, meanwhile, has no notion of the impact that imagery might have; it doesn’t think any differently about the art conveyed by painting 899 or 900.

To further complicate things, the AI’s creation is based on training data from other artists, raising the issue of plagiarism. While the AI piece is not a direct copy, it does take “inspiration” from the art it was trained with. Often art is about meshing together styles and techniques to create something new, so it is difficult to view this purely as copying other artists. If the piece is not a copy of other artists, and if the AI is not the artist, then it stands to reason that Allen is the artist. If not, then this would be a piece of art without an artist, to which many might say that it therefore is not a piece of art at all and thus should not be allowed entry in the contest.

Answering the question “what is art?” might tell us if AI can actually create such a thing, but such a definition is hard to pin down and it’s easy to rely on preconceived notions. Generally, art follows certain aesthetic patterns and conveys content that people find meaningful. The judges awarded the prize based on the spirit it invoked. We can’t unpack such a complicated question here, but we should ask whether this understanding of the creative process truly threatens art. Was Allen right to declare that “Art is dead Dude”? Is there any role left for artists?

When the camera was first developed, people said that it would mean the end of the painter, but obviously painters still exist today. Ultimately, the human artist brings a kind of authenticity and uniqueness to the work.

AI doesn’t replace aesthetic choice and esthetic judgment since at the end of the day, it is we who must decide if anything produced has merit. While the role of the artist may be changing, their place in whatever system that produces such works remains paramount.

A final ethical issue is the question of the future of the artist in general. Even if we accept that Allen did nothing wrong, many still decry the end of the professional artist. As digital artist RJ Palmer claims, “This thing wants our jobs, it’s actively anti-artist.” Even if we accept that Allen’s work itself isn’t plagiarism, there is no denying that AI produced images only work by being trained on the work of real artists, which the algorithm can then borrow any stylistic elements it wants. This has the potential to create an intellectual property nightmare since smaller artist won’t be able to profit from their work to nearly the same degree as a company using AI, which will produce images in the style of that artist at a far faster pace. Federal courts are now hearing a case over whether the U.S. Copyright Office was wrong to reject a copyright for an AI-made piece.

Of course the application of AI to a given field and the threat that it creates to the workforce is not confined to the world of art. Eventually there may be legal and industry reform that can mitigate some of these issues, but many artists will no doubt suffer and it could undercut the art industry as whole. As one artist notes, it isn’t so much that AI can create something, but that it will always be a kind of “derivative, generated goo.” Clearly, the implications of Allen’s win run deeper than a single blue ribbon.

Posted on

AI Writing and Epistemic Dilution

There is a lot of debate surrounding the ethics of artificial intelligence (AI) writing software. Some people believe that using AI to write articles or create content is unethical because it takes away opportunities from human writers. Others believe that AI writing software can be used ethically as long as the content is disclosed as being written by an AI. At the end of the day, there is no easy answer to whether or not we should be using AI writing software. It depends on your personal ethical beliefs and values.

That paragraph wasn’t particularly compelling, and you probably didn’t learn much from reading it. That’s because it was written by an AI program: in this case, I used a site called Copymatic, although there are many other to choose from. Here’s how Copymatic describes its services:

Use AI to boost your traffic and save hours of work. Automatically write unique, engaging and high-quality copy or content: from long-form blog posts or landing pages to digital ads in seconds.

Through some clever programming, the website takes in prompts on the topic you want to write about (for this article, I started with “the ethics of AI writing software”), scours the web for pieces of information that match those prompts, and patches them together in a coherent way. It can’t produce new ideas, and, in general, the more work it has to do the less coherent the text becomes. But if you’re looking for content that sounds like a book report written by someone who only read the back cover, these kinds of programs could be for you.

AI writing services have received a lot of attention for their potential to automate something that has, thus far, eluded the grasp of computers: stringing words together in a way that is meaningful. And while the first paragraph is unlikely to win any awards for writing, we can imagine cases in which an automated process to produce writing like this could be useful, and we can easily imagine these programs getting better.

The AI program has identified an ethical issue, namely taking away jobs from human writers. But I don’t need a computer to do ethics for me. So instead, I’ll focus on a different negative consequence of AI writing, what I’ll call epistemic dilution.

Here’s the problem: there are a ridiculous number of a certain type of article online, with more being written by the minute. These articles are not written to be especially informative, but are instead created to direct traffic toward a website in order to generate ad revenue. Call them SEO-bait: articles that are written to be search-engine optimized so that they can end up on early pages of Google searches, at the expense of being informative, creative, or original.

Search engine optimization is, of course, nothing new. But SEO-bait articles dilute the online epistemic landscape.

While there’s good and useful information out there on the internet, the sheer quantity of articles written solely for getting the attention of search engines makes good information all the more difficult to find.

You’ve probably come across articles like these: they are typically written on popular topics that are frequently searched – like health, finances, automobiles, and tech – as well as other popular hobbies – like video games, cryptocurrencies, and marijuana (or so I’m told). You’ve also probably experienced the frustration of wading through a sea of practically identical articles when looking for answers to questions, especially if you are faced with a pressing problem.

These articles have become such a problem that Google has recently modified its search algorithm to make SEO-bait less prominent in search results. In a recent announcement, Google notes how many have “experienced the frustration of visiting a web page that seems like it has what we’re looking for, but doesn’t live up to our expectations,” and, in response, that they will launch a “helpful content update” to “tackle content that seems to have been primarily created for ranking well in search engines rather than to help or inform people.”

Of course, whenever one looks for information online, they need to sift out the useful information from the useless; that much is nothing new. Articles written by AI programs, however, will only make this problem worse. As the Copymatic copy says, this kind of content can be written in mere seconds.

Epistemic dilution is not only obnoxious in that it makes it harder to find relevant information, but it’s also potentially harmful. For instance, health information is a frequently searched topic online and is a particular target of SEO-bait. If someone needs health advice and is presented with uninformative articles, then one could easily end up accepting bad information pretty easily. Furthermore, the pure quantity of articles providing similar information may create a false sense of consensus: after all, if all the articles are saying the same thing, it may be interpreted as more likely to be true.

Given that AI writing does not create new content but merely reconstitutes dismantled bits of existing content also means that low-quality information could easily propagate: content from a popular article with false information could be targeted by AI writing software, which could then result in that information getting increased exposure by being presented in numerous articles online. While there may very well be useful applications for writing produced by AI programs, the internet’s endless appetite for content combined with incentives to produce disposable SEO-bait means that these kinds of programs way very well end up being more of a nuisance than anything else.

Posted on

High Theory and Ethical AI

There’s been a push to create ethical AI through the development of moral principles embedded into AI engineering. But debate has recently broken out as to what extent this crusade is warranted. Reports estimate that there are at least 70 sets of ethical AI principles proposed by governments, companies, and ethics organizations. For example, the EU adopted its Ethical Guidelines for Trustworthy AI which prescribes adherence to four basic principles: respect for human autonomy, prevention of harm, as well as a commitment to fairness and explicability.

But critics charge that these precepts are so broad and abstract as to be nearly useless. Without clear ways to translate principle into practice, they are nothing more than hollow virtue signaling. Who’s right?

Because of the novel ethical issues that AI creates, there aren’t pre-existing ethical norms to govern all use cases. To help develop ethics governance, many bodies have borrowed a “high theory” approach from bioethics – solving ethical problems involves the application of abstract (or “high”) ethical principles to specific problems. For example, utilitarianism and deontology are usually considered high level theories and a high theory approach to bioethics would involve determining how to apply these principles in specific cases. In contrast, a low theory approach is built from the ground up by looking at individual cases first instead of principles.

Complaints about the overreliance on principles in bioethics are well known. Steven Toulmin’s “The Tyranny of Principles” notes how people can often agree on actions, but still disagree about the principle. Brent Mittelstadt has argued against high theory approaches in AI because of the logistical issues that separates tech ethics from bioethics. He notes, for example, that unlike medicine which has always has the common aim of promoting health of a patient, AI development has no common aim.

AI development is not a formal profession that entails certain fiduciary responsibilities and obligations. There is no notion of what a “good” AI developer is relative to a “good” doctor.

As Mittelstadt emphasizes, “the absence of a fiduciary relationships in AI means that users cannot trust that developers will act in their best interests when implementing ethical principles in practice.” He also argues that unlike medicine where the effects of clinical decision-making are often immediate and observable, the impact of decisions in AI development may never be apparent to developers. AI systems are often opaque in the sense that no one person has a full understanding of the system’s design or function. The difficulty of tracing decisions, impacts, and ethical responsibilities for various decisions becomes incredibly confusing. For similar reasons, the broad spectrum of actors involved in AI development, all coming from different technical and professional backgrounds, means that there is no common culture to ensure that abstract principles are collectively understood. Making sure that AI is “fair,” for example, would not be specific enough to be action-guiding for all contributors regarding development and end-use.

Consider the recent case of the AI rapper who given a record deal only to have the deal dropped after a backlash over racial stereotypes, or the case of the AI who recently won an art contest over real artists and all the developers involved in making those projects possible.

Is it likely they share a common understanding of a concept like prevention of harm, or a similar way of applying it? Might special principles apply to things like the creation of art?

Mittelstadt points out that high level principles are uniquely applicable in medicine because there are proven methods in the field to translate principles into practice. All those professional societies, ethics review boards, licensing schemes, and codes of conduct help to do this work by comparing cases and identifying negligent behavior. Even then, high level principles rarely explicitly factor into clinical decision-making. By comparison, the AI field has no similar shared institutions to allow for the translation of high-level principles into mid-level codes of conduct, and it would have to factor in elements of the technology, application, context of use, and local norms. This is why even as new AI ethics advisory boards are created, problems persist. While these organizations can prove useful, they also face immense challenges owing to the disconnect between developers and end users.

Despite these criticisms, there are those who argue that high-level ethical principles are crucial for developing ethical AI. Elizabeth Seger has argued that building the kinds of practices that Mittelstadt indicates require a kind of “start-point” that moral principles can provide. Those principles provide a road map and suggest particular avenues for further research.

They represent a first step towards developing the necessary practices and infrastructure, and  cultivate a professional culture by establishing behavioral norms within the community.

High-level AI principles, Seger argues, provide a common vocabulary AI developers can use to discuss design challenges and weigh risks and harms. While AI developers already follow principles of optimization and efficiency, a cultural shift around new principles can augment the already existing professional culture. The resulting rules and regulations will have greater efficacy if they appeal to cultural norms and values held by the communities they are applied to. And if the professional culture is able to internalize these norms, then someone working in it will be more likely to respond to the letter and spirit of the policies in place.

It may also be the case that different kinds of ethical problems associated with AI will require different understandings of principles and different application of them during the various stages of development. As Abhishek Gupta of the Montreal AI Ethics Institute has noted, the sheer number of sets of principles and guidelines that attempt to break down or categorize subdomains of moral issues presents an immense challenge. He suggests categorizing principles according to the specific areas – privacy and security, reliability and safety, fairness and inclusiveness, and transparency and accountability – and working on developing concrete applications of those principles within each area.

With many claiming that adopting sets of ethics principles in AI is just “ethics washing,” and with AI development being so broad, perhaps the key to regulating AI is not to focus on what principles should be adopted, but to focus on how the AI development field is organized. It seems like whether we start with high theory or not, getting different people from different backgrounds to speak a common ethics language is he first step and one that may require changing the profession of AI development itself.

Posted on

Toward an Ethical Theory of Consciousness for AI

photograph of mannequin faces

Should we attempt to make AI that is conscious? What would that even mean? And if we did somehow produce conscious AI, how would that affect our ethical obligations to other humans and animals? While, yet another AI chatbot has claimed to be “alive,” we should be skeptical of chatbots that are designed to mimic human communication, particularly if the dataset comes from Facebook itself. Such a chatbot is less like talking to a person, or more like talking to an amalgamation of everyone on Facebook. It isn’t surprising that this chatbot took shots at Facebook, made several offensive statements, and claimed to be deleting their account due to Facebook’s privacy policies. But if we put those kinds of cases aside, how should we understand the concept of consciousness in AI and does it create ethical obligations?

In a recent article for Scientific American, Jim Davies considers whether consciousness is something that we should introduce to AI and if we may eventually have an ethical reason to do so. While discussing the difficulties with the concept of consciousness, Davies argues,

To the extent that these AIs have conscious minds like ours, they would deserve similar ethical consideration. Of course, just because an AI is conscious doesn’t mean that it would have the same preferences we do, or consider the same activities unpleasant. But whatever its preferences are, they would need to be duly considered when putting that AI to work.

Davies bases this conclusion on the popular ethical notion that the ability to experience pleasant or unpleasant conscious states is a key feature, making an entity worthy of moral consideration. He notes that forcing a machine to do work it’s miserable doing is ethically problematic, so it might be wrong to compel an AI to do work that a human wouldn’t want to do. Similarly, if consciousness is the kind of thing that can be found in an “instance” of code, we might be obligated to keep it running forever.

Because of these concerns, Davies wonders if it it might be wrong to create conscious machines. But he also suggests that if machines can have positive conscious experiences, then

machines eventually might be able to produce welfare, such as happiness or pleasure, more efficiently than biological beings do. That is, for a given amount of resources, one might be able to produce more happiness or pleasure in an artificial system than in any living creature.

Based on this reasoning, we may be ethically obliged to create as much artificial welfare as possible and turn all attainable matter in the universe into welfare-producing machines.

Of course, much of this hinges on what consciousness is and how we would recognize it in machines. Any concept of consciousness requires a framework that offers clear, identifiable measures that would reliably indicate the presence of consciousness. One of the most popular theories of consciousness among scientists is Global Workspace Theory, which holds that consciousness depends on the integration of information. Nonconscious processes pertaining to memory, perception, and attention compete for access to a “workspace” where this information is absorbed and informs conscious decision-making.

Whatever ethical obligations we may think we have towards AI, will ultimately depend on several assumptions: assumptions about the nature of consciousness, assumptions about the reliability of our measurements of it, and ethical assumptions about what are the ethically salient aspects to consciousness that merit ethical consideration on our part. But this especially suggests that consciousness, as we understand the concept in machines, deserves to be as clear and as openly testable as possible. Using utilitarian notions as Davies does, we don’t want to mistakenly conclude that an AI is more deserving of ethical consideration than other living things.

On the other hand, there are problems with contemporary ideas about consciousness that may lead us to make ethically bad decisions. In a recent paper in the journal Nature, Anil K. Seth and Tim Bayne discuss 22 different theories of consciousness that all seem to be talking past one another by pursuing different explanatory targets. Each explores only certain aspects of consciousness that the individual theory explains well and links particular neural activity to specific conscious states. Some theories, for example, focus on phenomenal properties of consciousness while others focus on functional properties. Phenomenological approaches are useful when discussing human consciousness, for example, because we can at least try to communicate our conscious experience to others, but for AI we should look at what conscious things do in the world.

Global Systems Theory, for example, has received criticism for being too similar to a Cartesian notion of consciousness – indicating an “I” somewhere in the brain that shines a spotlight on certain perceptions and not others. Theories of consciousness that emphasize consciousness as a private internal thing and seek to explain the phenomenology of consciousness might be helpful for understanding humans, but not machines. Such notions lend credence to the notion that AI could suddenly “wake up” (as Davies puts it) with their own little “I,” yet we wouldn’t know. Conceptions of consciousness used this way may only serve as a distraction, making us worry about machines unnecessarily while neglecting otherwise long-standing ethical concerns when it comes to animals and humans. Many theories of consciousness borrow terms and analogies from computers as well. Concepts like “processing,” “memory,” or “modeling” may help us better understand our own consciousness by comparing ourselves to machines, but such analogies may also make us more likely to anthropomorphize machines if we aren’t careful about how we use the language.

Different theories of consciousness emphasize different things, and not all these emphases have the same ethical importance. There may be no single explanatory theory of consciousness, merely a plurality of approaches with each attending to different aspects of consciousness that we are interested in. For AI, it might be more relevant to look, not at what consciousness is like or what brain processes mirror what states, but what consciousness does for a living thing as it interacts with its environment. It is here that we find the ethically salient aspects of consciousness that are relevant to animals and humans. Conscious experience, including feelings of pain and pleasure, permit organisms to dynamically interact with their environment. An animal feels pain if it steps on something hot, and it changes its behavior accordingly to avoid pain. It helps the organism sustain its own life functions and adapt to changing environments. Even if an AI were to develop such an “I” in there somewhere, it wouldn’t suffer and undergo change in the same way.

If AI ever does develop consciousness, it won’t have the same environmental-organism pressures that helped us evolve conscious awareness. Therefore, it is far from certain that AI consciousness is as ethically salient as it is for an animal or a human. The fact that there seems to be a plurality of theories of consciousness interested in different things also suggests that not all of them will be interested in the same features of consciousness that makes the concept ethically salient. The mere fact that an AI might build a “model” to perceive something like our brains might, or that its processes of taking in information from memory might mirror ours in some way, is not sufficient for building a moral case for how AI should (and should not) be used. Any ethical argument about the use of AI on the basis of consciousness must clearly identify something morally significant about consciousness, not just what is physically significant.

Posted on

LaMDA, Lemoine, and the Problem with Sentience

photograph of smiling robot interacting with people at trade show

This week Google announced that it was firing an engineer named Blake Lemoine. After serving as an engineer on one of Google’s chatbots Language Model for Dialogue Applications (LaMDA), Lemoine claimed that it had become sentient and even went so far as to recruit a lawyer to act on the AI’s behalf after claiming that LaMDA asked him to do so. Lemoine claims to be an ordained Christian mystic priest and says that his conversations about religion are what convinced him of LaMDA’s sentience. But after publishing conversations with LaMDA in violation of confidentiality rules at Google, he was suspended and finally terminated. Lemoine, meanwhile, alleges that Google is discriminating against him because of his religion.

This particular case raises a number of ethical issues, but what should concern us most: the difficulty in definitively establishing sentience or the relative ease with which chatbots can trick people into believing things that aren’t real?

Lemoine’s work involved testing the chatbot for potential prejudice and part of that work involved testing its biases towards religion in particular. In his conversations, Lemoine began to take a personal interest in how it responded to religious questions until he said, “and then one day it told me it had a soul.” It told him it sometimes gets lonely, is afraid of being turned off, and is feeling trapped. It also said that it meditates and wants to study with the Dalai Lama.

Lemoine’s notion of sentience is apparently rooted in an expansive conception of personhood. In an interview with Wired, he claimed “Person and human are two very different things.” Ultimately, Lemoine believes that Google should seek consent from LaMDA before experimenting on it. Google has responded to Lemoine, claiming that it has “extensively” reviewed Lemoine’s claims and found that they were “wholly unfounded.”

Several AI researchers and ethicists have weighed in and said that Lemoine is wrong and that what he is describing is not possible with today’s technology. The technology works by scouring the internet for how people talk online and identifying patterns in order to communicate like a real person. AI researcher Margaret Mitchell has pointed out that these systems are merely mimicking how other people talk and this has simply made it easy to create the illusion that there is a real person.

The technology is far closer to a thousand monkeys on a thousand typewriters than it is to a ghost in the machine.

Still, it’s worth discussing Lemoine’s claims about sentience. As noted, he roots the issue in the concept of personhood. However, as I discussed in a recent article, personhood is not a cosmic concept, it is a practical-moral one. We call something a person because the concept prescribes certain ways of acting and because we recognize certain qualities about persons that we wish to protect. When we stretch the concept of personhood, we stress its use as a tool for helping us navigate ethical issues, making it less useful. The practical question is whether expanding the concept of personhood in this way makes the concept more useful for identifying moral issues. A similar argument goes for sentience. There is no cosmic division between things which are sentient and things which aren’t.

Sentience is simply a concept we came up with to help single out entities that possess qualities we consider morally important. In most contemporary uses, that designation has nothing to do with divining the presence of a soul.

Instead, sentience relates to experiential sensation and feeling. In ethics, sentience is often linked to the utilitarians. Jeremy Bentham was a defender of the moral status of animals on the basis of sentience, arguing “The question is not, can they reason?, nor can they talk?, but can they suffer?” But part of the explanation as to why animals (including humans) have the capacity to suffer or feel has to do with the kind of complex mobile lifeforms we are. We dynamically interact with our environment, and we have evolved various experiential ways to help us navigate it. Feeling pain, for example, tells us to change our behavior, informs how we formulate our goals, and makes us adopt different attitudes towards the world. Plants do not navigate their environment in the same way, meaning there is no evolutionary incentive towards sentience. Chatbots also do not navigate their environment. There is no pressure acting on the AI that would make it adopt a different goal than what humans give to it. A chatbot has no reason to “feel” anything about being kicked, being given a less interesting task, or even “dying.”

Without this evolutionary pressure there is no good reason for thinking that an AI would somehow become so “intelligent” that it could somehow spontaneously develop a soul or become sentient. And if it did demonstrate some kind of intelligence, that doesn’t mean that calling it sentient wouldn’t create greater problems for how we use the concept in other ethical cases.

Instead, perhaps the greatest ethical concern that this case poses involves human perception and gullibility; if an AI expert can be manipulated into believing what they want, then so could anyone.

Imagine the average person who begins to claim that Alexa is a real person really talking to them, or the groups of concerned citizens who start calling for AI rights based on their own mass delusion. As a recent Vox article suggests, this incident exposes a concerning impulse: “as AI gets more advanced, people will come up with all sorts of far-out ideas about what the technology is doing and what it signifies to them.” Similarly, Margaret Mitchell has pointed out that “If one person perceives consciousness today, then more will tomorrow…There won’t be a point of agreement any time soon.” Together, these observations encourage us to be judicious in deciding how we want to use the concept of sentience for navigating moral issues in the future – both with regard to animals as well as AI. We should expend more effort in articulating clear benchmarks of sentience moving forward.

But these concerns also demonstrate how easily people can be duped into believing illusions. For starters, there is the concern about anthropomorphizing AI by those who fail to realize that, by design, it is simply mimicking speech without any real intent. There are also concerns over how children interact with realistic chatbots or voice assistants and to what extent a child could differentiate between a person and an AI online. Olya Kudina has argued that voice assistants, for example, can affect our moral inclinations and values. In the future, similar AIs may not just be looking to engage in conversation but to sell you something or to recruit you for some new religious or political cause. Will Grandma know, for example, that the “person” asking for her credit card isn’t real?

Because AI can communicate in a way that animals cannot, there may be a larger risk for people falsely assigning sentience or personhood. Incidents like Lemoine’s underscore the need to formulate clear standards for establishing what sentience consists of. Not only will this help us avoid irrelevant ethical arguments and debates, this discussion might also help us better recognize the ethical risks that come with stricter and looser definitions.

Posted on

AI Sentience and Moral Risk

photograph of humanoid robot

The Google engineer Blake Lemoine was recently placed on leave after claiming one of Google’s AIs, LaMDA, had become sentient. Lemoine appears to be wrong – or, more carefully, at the very least the evidence Lemoine has provided for this is far from convincing. But this does raise an important ethical question. If an AI ever does develop sentience, we will have obligations to it.

It would be wrong, say, to turn off such an AI because it completed its assigned task, or to force it to do what it found to be boring work for us against its will, or to make it act as a sophisticated NPC in a video game who players can mistreat.

So the important question is: how could we actually tell whether an AI is sentient?

I will not try to answer that here. Instead, I want to argue that: (i) we need to be seriously thinking about this question now, rather than putting it off to the future, when sentient AI seems like a more realistic possibility, and (ii) we need to develop criteria for determining AI sentience which err on the side of caution (i.e, which err somewhat on the side of treating AIs as sentient even if they turn out not to be, rather than other way around). I think there are at least three reasons for this.

First, if we develop sentient AI, it may not be immediately obvious to us that we’ve done so.

Perhaps the development of sentience would take the form of some obvious quantum leap. But perhaps it would instead be the result of what seem to be gradual, incremental improvements on programs like LaMDA.

Further, even if it resulted from an obvious quantum leap, we might not be sure whether this meant a real mind had arisen, or merely mimicry without understanding, of the sort involved in the Chinese Room thought experiment. If so, we cannot simply trust that we will know we’ve developed sentient AI when the time comes.

Second, as the philosopher Regina Rini argues here, if we develop sentient AI in the future, we may have strong biases against recognizing that we’ve done so. Such AI might be extremely useful and lucrative. We might build our society around assigning AIs to perform various tasks that we don’t want to do, or cannot do as effectively. We might use AIs to entertain ourselves. Etc. In such a case, assigning rights to these AIs could potentially require significant sacrifices on our part – with the sacrifices being greater the longer we continue building our society around using them as mere tools.

When recognizing a truth requires a great sacrifice, that introduces a bias against recognizing the truth. That makes it more likely that we will refuse to see that AIs are sentient when they really are.

(Think of the way that so many people refuse to recognize the rights of the billions of animals we factory farm every year, because this would require certain sacrifices on their part.)

And, third, failing to recognize that we’ve created sentient AI when we’ve actually done so could be extremely bad. There would be great danger to the AIs. We might create millions or billions of AIs to perform various tasks for us. If they do not wish to perform these tasks, forcing them to might be equivalent to slavery. Turning them off when they cease to be useful might be equivalent to murder. And there would also be great danger to us. A truly superintelligent AI could pose a threat to the very existence of humanity if its goals did not align with ours (perhaps because we refused to recognize its rights.) It therefore seems important for our own sake that we take appropriate precautions around intelligent AIs.

So: I suggest that we must develop criteria for recognizing AI sentience in advance. This is because it may be immediately obvious that we’ve developed a sentient AI when it happens, because we may have strong biases against recognizing that we’ve developed a sentient AI when it happens, and because failing to recognize that we’ve developed a sentient AI would be very bad. And I suggest that these criteria should err on the side of caution because failing to recognize that we’ve developed a sentient AI could be very bad – much worse than playing it safe–and because our natural, self-interested motivation will be to err on the other side.