automation Archives - Prindle Institute

March 20, 2023March 20, 2023

What’s Wrong with AI Therapy Bots?

I have a distinct memory from my childhood: I was on a school trip, at what I think was the Ontario Science Centre, and my classmates and I were messing around with a computer terminal. As this was the early-to-mid 90s the computer itself was a beige slab with a faded keyboard, letters dulled from the hunt-and-pecking of hundreds of previous children on school trips of their own. There were no graphics, just white text on a black screen, and a flashing rectangle indicating where you were supposed to type.

The program was meant to be an “electronic psychotherapist,” either some version of ELIZA – one of the earliest attempts at what we would now classify as a chatbot – or some equivalent Canadian substitute (“Eh-LIZA”?). After starting up the program there was a welcome message, after which it would ask questions – something like “How are you feeling today?” or “What seems to be bothering you?” The rectangle would flash expectantly, store the value of the user’s input in a variable, and then spit it back out, often inelegantly, in a way that was meant to mimic the conversation of a therapist and patient. I remember my classmate typing “I think I’m Napoleon” (the best expression of our understanding of mental illness at the time) and the computer replying: “How long have you had the problem I think I’m Napoleon?”

30-ish years later, I receive a notification on my phone: “Hey Ken, do you want to see something adorable?” It’s from an app called WoeBot, and I’ve been ignoring it. WoeBot is one of several new chatbot therapists that tout that they are “driven by AI”: this particular app claims to sit at the intersections of several different types of therapy – cognitive behavioral therapy, interpersonal psychotherapy, and dialectical behavior therapy, according to their website – and AI that is powered by natural language processing. At the moment, it’s trying to cheer me up by showing me a gif of a kitten.

Inspired by (or worried they’ll get left behind by) programs like ChatGPT, tech companies have been chomping at the bit to create their own AI programs that produce natural-sounding text. The lucrative world of self-help and mental well-being seems like a natural fit for such products, and many claim to solve a longstanding problem in the world of mental healthcare: namely, that while human therapists are expensive and busy, AI therapists are cheap and available whenever you need them. In addition to WoeBot, there’s Wysa – also installed on my phone, and also trying to get my attention – Youper, Fingerprint for Success, and Koko, which recently got into hot water by failing to disclose to its userbase that they were not, in fact, chatting with a human therapist.

Despite having read reports that people have found AI therapy bots to be genuinely helpful, I was skeptical. But I attempted to keep an open mind, and downloaded both WoeBot and Wysa to see what all the fuss was about. After using them for a month, I’ve found them to be very similar: they both “check in” at prescribed times throughout the day, attempt to start up a conversation about any issues that I’ve previously said I wanted to address, and recommend various exercises that will be familiar to those who have ever done any cognitive behavioral therapy. They both offer the option to connect to real therapists (for a price, of course), and perhaps in response to the Koko debacle, neither hides the fact that they are programs (often annoying so: WoeBot is constantly talking about how its friends are other electronics, a schtick that got tired almost immediately).

It’s been an odd experience. The apps send me messages saying that they’re proud of me for doing good work, that they’re sorry if I didn’t find a session to be particularly useful, and that they know that keeping up with therapy can be difficult. But, of course, they’re not proud of me, or sorry, and they don’t know anything. At times their messages are difficult to distinguish from those of a real therapist; at others, they don’t properly parse my input, and respond with messages not unlike “How long have you had the problem I think I’m Napoleon?” If there is any therapeutic value in the suspension of disbelief then it often does not last long.

But apart from a sense of weirdness and the occasional annoyances, are there any ethical concerns surrounding the use of AI therapy chatbots?

There is clearly potential for them to be beneficial: your stock model AI therapist is free, and the therapies that they draw their exercises from are often well-tested in the offline world. A little program that reminds you to take deep breaths when you’re feeling stressed out seems all well and good, so long as it’s obvious that it’s not a real person on the other side.

Whether you think the hype about new AI technology is warranted or not will likely impact your feelings about the new therapy chatbots. Techno-optimists will emphasize the benefit of expanding care to many more people than could be reached through other means. Those who are skeptical of the hype, however, are likely to think that spending so much money on unproven tech is a poor use of resources: instead of sinking billions into competing chatbots, maybe that money could be spent on helping a wider range of people access traditional mental health resources.

There are also concerns about the ability of AI-driven text generators to go off the rails. Microsoft’s recent experiment with their new AI-powered Bing search had an inauspicious debut, occasionally spouting nonsense and even threatening users. It’s not hard to imagine the harm such unpredictable outputs could cause for someone who relied heavily on their AI therapy bot. Of course, true believers in the new AI revolution will dismiss these worries as growing pains that inevitably come along with the use of any new tech.

What is perhaps troubling is that the apps themselves walk a tightrope between trying to be a sympathetic ear, and reminding you that they’re just bots. The makers of WoeBot recently released research results that suggest that users feel a “bond” with the app, similar to the kind of bond they might feel with a human therapist. This is clearly an intentional choice on the part of the creators, but it brings with it some potential pitfalls.

For example, although the apps I’ve tried have never threatened me, they have occasionally come off as cold and uninterested. During a recent check-in, Wysa asked me to tell it what was bothering me that morning. It turned out to be a lot (the previous few days hadn’t been great). But after typing it all out and sending it along, Wysa quickly cut the conversation short, saying that it seemed like I didn’t want to engage at the moment. I felt rejected. And then I felt stupid that I felt rejected, because there was nothing that was actually rejecting me. Instead of feeling better by letting it all out, I felt worse.

In using the apps I’m reminded of a thought experiment from philosopher Hilary Putnam. He asks us to consider an ant on a beach who, through its search for food and random wanderings, happens to trace out what looks to be a line drawing of Winston Churchill. It is not, however, a picture of Churchill, and the ant did not draw it, at least in the way that you or I might. However, at the end of the day a portrait of Winston Churchill consists of a series of marks on a page (or on a beach), so what, asks Putnam, is the relevant difference between those made by the ant and those made by a person?

His answer is that only the latter are made intentionally, and it is the underlying intention which gives the marks their meaning. WoeBot and Wysa and other AI-powered programs often string together words in ways that look indistinguishable from those that might be written down by a human being on the other side. But there is no intentionality, and without intentionality there is no genuine empathy or concern or encouragement behind the words. They are just marks on a screen that happen to have the same shape as something meaningful.

There is, of course, a necessary kind of disingenuousness that must exist for these bots to have any effect at all. No one is going to feel encouraged to engage with a program that explicitly reminds you that it does not care about you because it does not have the capacity to care. AI therapy requires that you play along. But I quickly got tired of playing make believe with my therapy bots, and it’s overall become increasingly difficult for me to find the value in this kind of ersatz therapy.

I can report one concrete instance in which using an AI therapy bot did seem genuinely helpful. It was guiding me through an exercise, the culmination of which was to get me to pretend as though I were evaluating my own situation as that of a friend, and to consider what I would say to them. It’s an exercise that is frequently used in cognitive behavioral therapy, but one that’s easy to forget to do. In this way, the app checking-in did, in fact, help: I wouldn’t have been as sympathetic to myself had it not reminded me to. But I can’t help but think that if that’s where the benefits of these apps lie – in presenting tried-and-tested exercises from various therapies and reminding you to do them – then the whole thing is over-engineered. If it can’t talk or understand or empathize like a human, then there seems to be little point in there being any artificial intelligence in there at all.

AI therapy bots are still new, and so it remains to be seen whether they will have a lasting impact or just be a flash in the pan. Whatever does end up happening, though, it’s worth considering whether we would even want the promise of AI-powered therapy to come true.

May 5, 2022December 14, 2022

Virtual Work and the Ethics of Outsourcing

Like a lot of people over the past two years, I’ve been conducting most of my work virtually. Interactions with colleagues, researchers, and other people I’ve talked to have taken place almost exclusively via Zoom, and I even have some colleagues I’ve yet to meet in person. There are pros and cons to the arrangement, and much has been written about how to make the most out of virtual working.

A recent event involving Canadian outlets of restraint chain Freshii, however, has raised some ethical questions about a certain kind of virtual working arrangement, namely the use of virtual cashiers called “Percy.” Here’s how it works: instead of an in-the-flesh cashier to help you with your purchase, a screen will show you a person working remotely, ostensibly adding a personal touch to what might otherwise feel like an impersonal dining experience. The company that created Percy explains their business model as follows:

Unlike a kiosk or a pre-ordering app, which removes human jobs entirely, Percy allows for the face-to-face customer experience, that restaurant owners and operators want to provide their guests, by mobilizing a global and eager workforce.

It is exactly this “global and eager workforce” that has landed Freshii in hot water: it has recently been reported that Freshii is using workers who are living in Nicaragua and are paid a mere $3.75 an hour. In Canada, several ministers and labor critics have harshly criticized the practice, with some calling for new legislation to prevent other companies from doing the same thing.

Of course, outsourcing is nothing new: for years, companies have hired overseas contractors to do work that can be done remotely, and at a fraction of the cost of domestic workers. At least in Canada, companies are not obligated to pay outsourced employees a wage that meets the minimum standards of Canadian wage laws; indeed, the company that produces Percy has maintained that they are not doing anything illegal.

There are many worries one could have with the practice of outsourcing in general, primarily among them: that they take away job opportunities for domestic employees, and that they treat foreign employees unfairly by paying them below minimum wage (at least by the standards of the country where the business is located).

There are also some arguments in favor of the practice: in an op-ed written in response to the controversy, the argument is made that while $3.75 is very little to those living in Canada and the U.S., it is more significant for many people living in Nicaragua. What’s more, with automation risking many jobs regardless, wouldn’t it be better to at least pay someone for this work, as opposed to just giving it to a robot? Of course, this argument risks presenting a false dichotomy – one could, after all, choose to pay workers in Nicaragua a fair wage by Canadian or U.S. standards. But the point is still that such jobs provide income for people who need it.

If arguments about outsourcing are old news, then why all the new outrage? There does seem to be something particularly odd about the virtual cashier. Is it simply that we don’t want to be faced with a controversial issue that we know exists, but would rather ignore, or is there something more going on?

I think discomfort is definitely part of the problem – it is easier to ignore potentially problematic business practices when we are not staring them in the virtual face. But there is perhaps an additional part of the explanation, one that raises metaphysical questions about the nature of virtual work: when you work virtually, where are you?

There is a sense in which the answer to this question is obvious: you are wherever your physical body is. If I’m working remotely and on a Zoom call, the place I am would be in Toronto (seeing as that’s where I live) while my colleagues will be in whatever province or country they happen to be physically present in at the time.

When we are all occupying the same Zoom call, however, we are also in another sense in the same space. Consider the following. In this time of transition between COVID and (hopefully) post-COVID times, many in-person events have become hybrid affairs: some people will attend in-person, and some people will appear virtually on a screen. For instance, many conferences are being held in hybrid formats, as are government hearings, trials, etc.

Let’s say that I give a presentation at such a conference, that I’m one of these virtual attendees, and that I participate while sitting in the comfort of my own apartment. I am physically located in one place, but also attending the conference: I might not be able to be there in person, but there’s a sense in which I am still there, if only virtually.

It’s this virtual there-ness that I think makes a case like Percy feel more troubling. Although a Canadian cashier who worked at Freshii would occupy the physical space of a Freshii restaurant in Canada, a virtual cashier would do much of the same work, interact with the same customers, and see and hear most of the same things. In some sense, they are occupying the same space: the only relevant thing that differentiates them from their local counterpart is that they are not occupying it physically.

What virtual work has taught us, though, is that one’s physical presence really isn’t an important factor in a lot of jobs (excluding jobs that require physical labor, in-person contact, and work that is location-specific, of course). If the work of a Freshii cashier does not require physical presence, then it hardly seems fair that one be compensated at a much lower rate than one’s colleagues for simply not being there. After all, if two employees were physically in the same space, working the same job, we would think they should be compensated the same. Why, then, should it matter if one is there physically, and the other virtually?

Again, this kind of unfairness is present in many different kinds of outsourced work, and whether physical distance has ever been a justification for different rates of pay is up for debate. But with physical presence feeling less and less necessary for so many jobs, new working possibilities call into question the ethics of past practices.

March 27, 2018December 14, 2022

Who Should Program the Morality of Self-Driving Cars?

On Sunday, March 18, Elaine Herzberg died after being hit by Uber’s self-driving car on the road in Tempe, Arizona. Out for a test-run, the video of the collisions suggests that there was a failure of the self-driving technology as well as the in-car driver meant to supervise the testing of the test drive. Uber has removed its self-driving cars from the road while cooperating with investigations, and discussions of the quickly advancing future of driverless vehicles have once again been stirred up in the press.

Continue reading “Who Should Program the Morality of Self-Driving Cars?”

November 11, 2015December 14, 2022

Digital Decisions in the World of Automated Cars

We’re constantly looking towards the future of technology and gaining excitement for every new innovation that makes our lives easier in some way. Our phones, laptops, tablets, and now even our cars are becoming increasingly smarter. Most new cars on the market today are equipped with GPS navigation, cruise control, and even with some intelligent parallel parking programs. Now, self-driving cars have made their way to the forefront of the automotive revolution.

Continue reading “Digital Decisions in the World of Automated Cars”