← Return to search results
Back to Prindle Institute

pornography

Posted on

Deepfake Porn and the Pervert’s Dilemma

blurred image of woman on bed

This past week Representative Alexandra Ocasio-Cortez spoke of an incident where she was realistically depicted by a computer engaged in a sexual act. She recounted the harm and difficulty of being depicted in this manner. The age of AI-generated pornography is upon us and so-called deepfakes are becoming less visually distinguishable from real life every day. Emerging technology could allow people to generate true-to-life images and videos of their most forbidden fantasies.

What happened with Representative Ocasio-Cortez raises issues well beyond making pornography with AI of course. Deepfake pornographic images are not just used for personal satisfaction, they are used to bully, harass, and demean. Clearly, these uses are problematic, but what about the actual creation of the customized pornography itself? Is that unethical?

To think this through Carl Öhman articulates the “pervert’s dilemma”: We might think that any sexual fantasy conceived — but not enacted — in the privacy of our home and our own head is permissible. If we do find this ethical, then why exactly do we find it objectionable if a computer generates those images, also in the privacy of one’s home? (For the record, Öhman believes they have a way out of this dilemma.)

The underlying case for letting a thousand AI-generated pornographic flowers bloom is rooted in the famous Harm Principle of John Stuart Mill. His thought was that in a society which values individual liberty, behaviors should generally not be restricted unless they cause harm to others. Following from this, as long as no one is harmed in the generation of the pornographic image, the action should be permissible. We might find it gross or indecent. We might even find the behaviors depicted unethical or abhorrent. But if nobody is being hurt, then creating the image in private via AI is not itself unethical, or at least not something that should be forbidden.

Moreover, for pornography in which some of the worst ethical harms occur in the production process (the most extreme example being child pornography), AI-generated alternatives would be far preferable. (If it turns out that being able to generate such images increases the likelihood of the corresponding real-world behaviors, then that’s a different matter entirely.) Even if no actual sexual abuse is involved in the production of pornography, there have been general worries about the working conditions within the adult entertainment industry that AI-generated content could alleviate. Although, alternatively, just like in other areas, we may worry that AI-generated pornography undermines jobs in adult entertainment, depressing wages and replacing actors and editors with computers.

None of this disputes that AI-generated pornography can’t be put to bad ends, as the case of Representative Ocasio-Cortez clearly illustrates. And she is far from the only one to be targeted in this way (also see The Prindle Post discussion on revenge porn). The Harm Principle defender would argue that while this is obviously terrible, it is these uses of pornography that are the problem, and not simply the existence of customizable AI-generated pornography. From this perspective, society should target the use of deepfakes as a form of bullying or harassment, and not deepfakes themselves.

Crucially, though, this defense requires that AI-generated pornography be adequately contained. If we allow people to generate whatever images they want as long as they pinky-promise that they are over 18 and won’t use them to do anything nefarious, it could create an enforcement nightmare. Providing more restrictions on what can be generated may be the only way to meaningfully prevent the images from being distributed or weaponized even if, in theory, we believe that strictly private consumption squeaks by as ethically permissible.

Of course, pornography itself is far from uncontroversial, with longstanding concerns that it is demeaning, misogynistic, addictive, and encourages harmful attitudes and behaviors. Philosophers Jonathan Yang and Aaron Yarmel raise the worry that by providing additional creative control to the pornography consumer, AI turns these problematic features of pornography up to 11.  The argument, both in response to AI-generated pornography and pornography generally, depends on a data-driven understanding of the actual behavioral and societal effects of pornography — something which has so far eluded a decisive answer. While the Harm Principle is quite permissive about harm to oneself, as a society we may also find that the individual harms of endless customizable pornographic content are too much to bear even if there is no systematic impact.

Very broadly speaking, if the harms of pornography we are most worried about relate to its production, then AI pornography might be a godsend. If the harms we are most worried about relate to the images themselves and their consumption, then it’s a nightmare. Additional particularities are going to arise about labor, distribution, source images, copyright, real-world likeness, and much else besides as pornography and AI collide. Like everything sexual, openness and communication will be key as society navigates the emergence of a transformative technology in an already fraught ethical space.

Posted on

Fascism, Book Banning, and Non-Violent Direct Action

photograph of book burning in flames

This article has a set of discussion questions tailored for classroom use. Click here to download them. To see a full list of articles with discussion questions and other resources, visit our “Educational Resources” page.

Throughout the summer, armed Idaho citizens showed up at library board meetings at a small library in Bonners Ferry to demand that a list of 400 books be taken off of the shelves. The books in question were not, in fact, books that this particular library carried. In response to the ongoing threats against the library, its insurance company declined to continue to cover them, citing increased risk of violence or harm that might take place in the building. The director of the library, Kimber Glidden, resigned her position in response to the situation, citing personal threats and angry armed protestors showing up at her private home demanding that she remove the “pornography” from the shelves of her library.

This behavior is far from limited to the state of Idaho. In Oklahoma, Summer Boismier, an English teacher at Norman High School was put on leave because she told her students about UnBanned — a program out of the Brooklyn Country Library which allows people from anywhere in the country to access e-book versions of books that have been banned. The program was designed to fight back against censorship and to advocate for the “rights of teens nationwide to read what they like, discover themselves, and form their own opinions.” Boismier was put on leave after a parent protested that she had violated state law HB1775 which, among other things, prohibits the teaching of books or other material that might make one race feel that they are worse than another race. Despite the egalitarian sounding language, the legislation was passed in part to defend a conception of a mythic past and to prevent students from reading about the ways in which the United States’ history of racism continues to have significant consequences to this day. Boismier resigned in protest.

Many states have passed laws banning books with certain content, and that content often involves race, feminism, sexual orientation, and gender identity. And prosecutors in states like Wyoming have considered bringing criminal charges against librarians who continue to carry books that their legislatures have outlawed.

Laws like these capitalize on in-group/out-group dynamics and xenophobia, often putting marginalized groups at further risk of violence, anxiety, depression, and suicide.

In heated board meetings across the country, there appear to be at least two sides to this issue. First, there are the parents and community members who are opposed to censorship or who believe that noise over “pornographic” literature targeting children in libraries is tilting at windmills; in other words, the content the protestors are concerned about simply doesn’t exist. On the other side of the debate, there are parents who are concerned that their children are being exposed to material that is developmentally inappropriate and might actually significantly harm them.

Granting that some of these books contain material that genuinely concerns parents, it simply doesn’t follow that the material to which they object really is bad for the children involved.

The fact that a person is a parent does not make that person an expert on what is best for developing minds and parents do not own the minds of their children.

For instance, one of the most commonly challenged books in the country is The Hate U Give, which is, in part, a book about a teenage girl’s response to the racially motivated killing of her friend by a police officer. Some want this book banned because they don’t want their children internalizing anti-police sentiment. However, reading this kind of a book might increase a child’s critical thinking skills when it comes to how they perceive authority, while also contributing to compassion for historically marginalized groups.

Consider also perhaps the most controversial books — books that have some sexual content.

These books may not be appropriate for children under a certain age, but reading stories in which teenagers are going through common teenage struggles and experiences has the potential to help young readers understand that their thoughts and experiences are totally normal and even healthy.

Parents who want this content banned might be wrong about what is best for their children. Some of these parents might instead be trying to control their children. In any case, the fact that some parents don’t want their children to have access to a particular book does not mean that all young people should be prohibited from reading the books in question. Some parents don’t believe in censorship and trust their children to be discerning and reflective when they read.

That said, the apparent two-sidedness of this debate may well be illusory. After all, there is a simple way to determine whether the library carries books with the kind of content that parents are concerned about — simply check the database. The librarians in question claim that the books about which these parents are complaining are not books that the library carries. What’s more, these “debates” – though sometimes well-intentioned – are more troubling than they may initially appear. More and more people across the country appear to be succumbing to the kind of conspiratorial thinking that tills fertile ground for fascism. These are trends that are reminiscent of moral panics over comic books in the ’50s and ’60s and video games in the ’80s and ’90s.

Perhaps the most pressing question confronting our culture today is not whether libraries should continue to carry pornographic or racist materials (since they don’t) but, instead, what we should do about the looming threat of fascism.

Philosopher Hannah Arendt wrote about her concern that fascist demagogues, who behave as if facts themselves are up for debate, destroy the social fabric of reason on which we all rely. This creates communities of “people for whom the distinction between fact and fiction…and the distinction between true and false…no longer exist.” Novelists have explored these themes countless times, and the restriction of reading material is a common theme in dystopian novels. In particular, readers see these themes explored in 1984, Fahrenheit 451, and The Handmaid’s Tale. In 1984, Orwell describes a “Ministry of Truth” that is responsible for changing history books so that they say all and only what the authoritarian regime wants them to say. In Fahrenheit 451, Bradberry describes an authoritarian regime that disallows reading altogether. In The Handmaid’s Tale, reading is permitted, but only by the powerful, and in this case the only powerful community members are men.

All of these dystopian tales emphasize the importance of language, writing, reading, and freedom of expression both for healthy societies and for healthy individuals.

Ironically, all three of these books are frequently found on banned or challenged books lists.

Rising aggression on the part of those who call for the banning of books has motivated some to respond in purposeful but peaceful ways. In Idaho, for instance, a small group of a couple of dozen concerned citizens – composed of both liberals and conservatives – met in a grove of apple trees to hold a read-in in support of public libraries and against censorship.

These approaches respond to violence with non-violence, an activist approach favored by Martin Luther King Jr. That said, King was also explicit about the fact that the movement he facilitated was a movement of non-violent direct action, which meant more than simply showing up and being peaceful. The strategies King employed involved disrupting society, but non-violently. The Montgomery Bus Boycott, for instance, involved refusing to contribute to the economy of the city by paying for bus fare until it gave up its policy of forcing Black riders to give up their seats to white passengers and to sit at the back of the bus. The non-violent protests in Birmingham in 1963 were intended to affect the profits of merchants in the area so that there would be palpable motivation to end segregation. Non-violent direct action was never intended to be “polite” in the sense that it didn’t provide reasons for frustration. In his “Letter from a Birmingham Jail,” King says

My citing the creation of tension as part of the work of the nonviolent resister may sound rather shocking. But I must confess that I am not afraid of the word “tension.” I have earnestly opposed violent tension, but there is a type of constructive, nonviolent tension which is necessary for growth.

So, while there is certainly nothing wrong with reading a book under an apple tree, if people want to roll back the wave of censorship and anti-intellectualism – both trends that are part and parcel with fascism – action that is more than “polite” but less than violent may be warranted.

Posted on

Revenge Porn, Public Interest, and Free Speech

image of camera lens blended with an image of an eye

After resigning from office in 2019, former California Representative Katie Hill has been dragged back in the news. Hill’s lawsuits against the Daily Mail and RedState for publishing nonconsensual pornography were recently dismissed, and, to add insult to injury, she was ordered to pay more than $200,000 in defendants’ legal fees. Not only was Hill denied the possibility of being made whole, but was also forced to pay for the “privilege of them publishing nude photos … obtained from an abuser.”

How did we get here? Why is Katie Hill continuing to foot the bill for being made the target of “revenge porn”? How could the distribution of others’ private sexual images without their consent ever be protected by the First Amendment? Shouldn’t respect for individual privacy trump others’ interest in publicizing such intimate information?

In 2019, news broke that Hill and her then-husband had been involved in an improper relationship with a junior employee. Texts surfaced suggesting one of Hill’s campaign workers was not only engaged in a sexual relationship, but that she might also have been abused. It was further alleged that Hill was romantically involved with a congressional aide. RedState and the Daily Mail added fuel to the fire by publishing a number of suggestive and salacious photos. With rumors of a media trove containing hundreds of additional pictures and texts and a congressional probe just beginning, Hill resigned.

These revelations were no small thing; they suggested more than a mere lapse in judgment. The asymmetric power relation between the would-be congresswoman and a subordinate demonstrated a failure of responsibility and constituted an abuse of power. These charges must be taken seriously.

But the question of whether Hill’s relationship with a campaign staffer was improper (it was) should be kept separate from the question of whether the Daily Mail and RedState were acting in the public interest by choosing to publish nonconsensual pornography.

Judge Yolanda Orozco, who dismissed Hill’s case, claimed that the media outlets’ circulation of those nude photographs served a compelling public interest in questioning Hill’s “character, judgment and qualifications for her congressional position.” The electorate, she argued, deserved to bear witness to their representative’s apparent vices, and these news organizations were merely serving a democratic function in satisfying that need. The photos “allegedly depicted [Hill] with a campaign staffer whom she was alleged to have had a sexual affair with and appeared to show [Hill] using a then-illegal drug and displaying a tattoo that was controversial because it resembled a white supremacy symbol.” These, Orozco insists, are important details that the public has a right to know, or, at least, that news organizations have an overriding interest in disseminating.

This reasoning, however, appears to provide an incredibly broad read of what qualifies as a “matter of public concern.” Indeed, it seems to offer a blank check to any gossip or tabloid journalism that people might be able to put to good (political) use. (Consider, for example, the recently leaked video of New York City Council candidate Zack Weiner.) This justification does more than simply make the relaying of private information an important social good. Instead, Orozco’s position suggests that it is people’s feelings about the information, not the legal relevance of the information itself, which should determine the permissibility of sharing revenge porn. Whatever distaste or revulsion an image might provoke is enough to warrant overturning an individual’s right to legal protection against this kind of invasion of privacy, harassment, and sexual violence.

Further, according to Judge Orozco’s ruling, supplying a written description of the photos’ content instead of posting the photos themselves would fail to adequately capture or sufficiently communicate the level of depravity exhibited by the actual image. Instead, “the public should be permitted to determine the importance or relevance … for itself.” Again, this suggests that the ambiguity attending any likeness (inevitably lacking context) is just as important as (if not more important than) the cold, hard facts. A picture is worth a thousand words, and it’s meaning should lie with the beholder.

Note, however, that the photos’ publication is defended on the grounds that they provide evidence of Hill lacking moral fiber, not proof of the allegations leveled against her. The photograph which includes a staffer depicts a previous relationship that occurred during the campaign — a consensual relationship that was surely unethical, but not illegal, and to which Hill admitted — while the public value of the other photos centers on Hill holding a bong and sporting a questionable tattoo.

The publication of these photos, then, does little else but invite the viewer to pass judgment according to arbitrary standards of decency and decorum that amounts to nothing more than moralizing and slut-shaming. (Consider, for example, commentators’ judgments that the photos are revealing, “bespeaking a kind of hedonism that at least some voters may view as a character defect.”)

But our recognition of this unique kind of harm is precisely why revenge porn laws exist in the first place. Their purpose is to restrict the freedom of bad actors in recording and distributing salacious materials against their victims’ will. The publicizing of nonconsensual pornography weaponizes information by bringing public opinion down on the head of victims and branding them with an unerasable social stigma. Given the limitations to genuine redress, the law must have sufficient teeth to act as a deterrent capable of discouraging other would-be attackers. The law operates with the understanding that there is no putting the genie back in the bottle once a leak occurs.

The language of “revenge porn” is often criticized for focusing our attention on the perpetrator’s mindset as opposed to the damage done to victims. Surely, we shouldn’t concentrate our efforts on divining whether spite was the overwhelming motivation behind a perpetrator choosing to distribute illicit images. What matters is the unique kind of sexual violence that is being threatened. But, in this particular case, the context might prove important. First, the photos were taken without Hill’s knowledge (or consent) and distributed by a jilted and “abusive” ex-husband in the midst of a messy divorce. But the leak’s publication also looks to be the work of a concerted effort by Hill’s political opponents. As Quinta Jurecic of Lawfare notes, this may be the first known instance where a “politically aligned publication has published an explicit photo of an opposition politician for apparent political gain.” In fact, Politico reported that the person responsible for posting the photos used the same platform to publicly advocate for the Republican running for Hill’s vacated seat.

Despite these troubling circumstances, commentators have suggested that Hill simply “needs to take the L and move on.” Anyone who chooses to thrust herself into the public eye sacrifices the right to keep any detail of her private life hidden from view. Everything she does becomes a public concern. Ultimately, ours is a society that prioritizes the community’s right to know over individuals’ right not to disclose, privileges one’s freedom to do rather than others’ right to impede, and chafes much more at government overreach than at the paparazzi’s prying eyes. Still, it seems backwards to conclude those occupying the spotlight are less in need of this protection rather than more.

Posted on

Bella Thorne and Celebrities Inhabiting Shared Spaces

photograph of bella thorne on red carpet with crowd behind her

The age of technology has brought many new things into modern life, but arguably one of the most influential and important is social media. A radically new world was created online where everyone around the globe can be connected within seconds no matter their location. One of the groups to take advantage of this instant connection was celebrities as social media and online platforms allow them to connect with their fans directly and give audiences glimpses into their private lives, without having to actually even meet in-person. This has given rise to the phenomena of celebrity culture where the public can know almost any aspect of a star’s life. Some have used this trend to help build their fame and monetize their brands. While celebrities have every right to use these platforms just like any other member of the public, they enter into these spaces with an unfair advantage. They have a following and a brand, which usually disrupts some of the communities that are made up of the public, who might depend on these platforms to make a living. There’s a fine line here for celebrities to watch, as their introduction to these spaces threatens to undermine these platforms, and perhaps eliminate, or at least adulterate, this communal space.

Recently, one platform in particular, OnlyFans, has taken over the pornography market by allowing individuals to have autonomy over what and when they create. This form of pornography can be highly personal with subscribers getting to know the performers whose bodies and lives they are consuming. With OnlyFans, as long as you gain a following, anyone can make money through this form of sex work, without having to find a studio, or work in the public space. A new creator on the platform, actress Bella Thorne who started her career with the popular Disney show “Shake it Off,” broke records within 24 hours of her appearance on the site. She announced her introduction to OnlyFans with Paper Magazine where she wanted to discuss “the politics behind female body shaming & sex.” Immediately, she made headlines with her addition, which inevitably began sparking conversations around sex work and female sexuality — the discussion that she hoped would be happening.

There are both advantages and disadvantages to a celebrity of Bella Thorne’s caliber joining OnlyFans. Sex work has historically been a job that is not seen as a valid form of work and is criminalized in most countries around the world. As a consequent of this criminalization there are specific dangers that sex workers face in their line of employment, which are usually ignored by politicians, police officers, and society as a whole. If celebrities begin to partake in creating this type of content, however, a normalization may begin, which could work to validate and decriminalize sex work, and possibly address those issues that sex workers face daily. This appears to be Bella Thorne’s intention behind her move to OnlyFans. But she gravely miscalculated the responsibility she had to ensure that she didn’t hurt the very community she was trying to help.

Sex workers who rely on their income from OnlyFans faced a crisis as the website suddenly changed their policies, limiting the freedom and ability of performers to make a living off the platform. The catalyst to these changes was directly after Thorne made her debut on the platform, however, OnlyFans claims the two weren’t connected. Thorne made $1 million dollars within her first day on the site and $2 million after the first week. She also caused massive refunds after people paid for a nude photo, which in reality was not nude, and therefore many of those subscribers were demanding their money back from OnlyFans. Shortly after, the platform set limits on how much creators could charge for their content and the amount that consumers could give in tips to performers. Additionally, they lengthened the time that performers would receive their income to 30 days. A company that was once a safe space for sex workers to earn their living is now catering to the effects of celebrities. They profit from the audience that these big names bring on to their site, all the while ignoring the concerns of everyday sex workers whose livelihoods depend on the platform. For Bella Thorne, joining the platform is a way to have fun with her sexuality and popularity without the censorship or judgement of platforms like Instagram. She does not depend on that money for rent or food. She experiences little to none of the stigma that sex workers face daily. Her actions did not help the sex worker community, but actually severely hurt a community that is already one of the most marginalized.

What responsibility does Thorne even have in starting these conversations over sexual politics and female sexuality? How should she use her celebrity status and the privilege of millions of followers listening and watching her? One cannot ignore the fact that a lot of this increasing legitimacy of sex work has centered around middle-upper class white women beginning to explore the realms of sex work, while women of color continue to experience the stigma and marginalization of sex work. While sex work may slowly begin to be seen as a proper line of employment, there seems to be an otherness appearing in it, in which it is acceptable for certain women, but deplorable for others to take part in. This normalization is beginning to look more like a gentrification in which white women profit off the work that other women have been doing for decades, which would of course only continue to hurt a large portion of the sex worker community. So, perhaps it was not even Thorne’s place to be the catalyst to start those conversations she wants to have. Her attempt to make that conversation was centered around herself and her own experiences. Instead of reaching out to women already experienced in the industry, she decided to see for herself the inner workings of the industry. But, it is impossible for a celebrity like her to experience sex work in a way that accurately represents the issues that sex workers deal with in reality.

Bella Thorne, however, is not the only celebrity to hop on this trend. The biggest name to recently join the platform is rapper Cardi B, although she won’t be creating sexual content, but rather exclusive content on her life and music. Some other celebrities like rapper Tyga, or YouTuber Tana Mangeau are deciding to follow in Thorne’s direction and make sexual content for their consumers. All of these celebrities can bring waves of fans to the site looking to buy subscriptions for the exclusive content. Whether their selling sex or exclusive updates on music, however, they will be entering a platform that already has plenty of competition for subscribers. Sex workers and musicians depend on their subscriptions from OnlyFans to continue paying rent or buying groceries, especially in the midst of a global pandemic, which are concerns that none of these celebrities would ever have to troubles themselves with. While the platform may be useful for them to promote their albums, have fun with their sexuality, or connect with fans, all their profits are solely pocket money for them. They could accomplish all of those things through Instagram pages with their millions of followers, or with a multitude of opportunities that are not open to the public. Celebrities need to recognize the havoc that they can wreak on the lives of everyday people when they decide to turn their livelihoods into fun experiments on social media.

Posted on

Ethical Considerations of Deepfakes

computer image of two identical face scans

In a recent interview for MIT Technology Review, art activist Barnaby Francis, creator of deepfake Instagram account @bill_posters_uk, mused that deepfake is “the perfect art form for these kinds of absurdist, almost surrealist times that we’re experiencing.” Francis’ use of deepfakes to mimic celebrities and political leaders on Instagram is aimed at raising awareness about the danger of deepfakes and the fact that “there’s a lot of people getting onto the bandwagon who are not really ethically or morally bothered about who their clients are, where this may appear, and in what form.” While deepfake technology has received alarmist media attention in the past few years, Francis is correct in his assertion that there are many researchers, businesses, and academics who are pining for the development of more realistic deepfakes.

Is deepfake technology ethical? If not, what makes it wrong? And who holds the responsibility to prevent the potential harms generated by deepfakes: developers or regulators?

Deepfakes are not new. The first mention of deepfake was by a reddit user in 2017, who began using the technology to create pornographic videos. However, the technology soon expanded to video games as a way to create images of people within a virtual universe. However, the deepfake trend suddenly turned toward more global agendas, with fake images and videos of public figures and political leaders being distributed en masse. One altered video of Joe Biden was so convincing that even President Trump fell for it. Last year, there was a deepfake video of Mark Zuckerberg talking about how happy he was to have thousands of people’s data. At the time, Facebook maintained that deepfake videos would stay up, as they did not violate their terms of agreement. Deepfakes have only increased since then. In fact, there exists an entire YouTube playlist with deepfake videos dedicated to President Trump.

In 2020, those who have contributed to deepfake technology are not only individuals in the far corners of the internet. Researchers at the University of Washington have also developed deepfakes using algorithms in order to combat their spread. Deepfake technology has been used to bring art to life, recreate the voices of historical figures, and to use celebrities’ likeness to communicate powerful public health messages. While the dangers of deepfakes have been described by some as dystopian, the methods behind their creation have been relatively transparent and accessible.

One problem with deepfakes are that they mimic a person’s likeness without their permission. The original Deepfakes, which used photos or videos of a person mixed with pornography uses a person’s likeness for sexual gratification. Such use of a person’s likeness might never personally affect them, but could still be considered wrong, since they are being used as a source of pleasure and entertainment, without consent. These examples might seem far-fetched, but in 2019 a now-defunct app called DeepNude, sought to do exactly that. Even worse than using someone’s likeness without their knowledge, is if the use of their likeness is intended to reach them and others, in order to humiliate or damage their reputation. One could see the possibility of a type of deepfake revenge-porn, where scorned partners attempt to humiliate their exes by creating deepfake pornography. This issue is incredibly pressing and might be more prevalent than the other potential harms of deepfakes. One study, for example, found that 96% of existing deepfakes take the form of pornography.

Despite this current reality, much of the moral concern over deepfakes is grounded in their potential to easily spread misinformation. Criticism around deepfakes in recent years has been mainly surrounding their potential for manipulating the public to achieve political ends. It is becoming increasingly easy to spread a fake video depicting a politician who is clearly incompetent or spreading a questionable message, which might detract from their base. On a more local level, deepfakes could be used to discredit individuals. One could imagine a world in which deepfakes are used to frame someone in order to damage their reputation, or even to suggest they have committed a crime. Video and photo evidence is commonly used in our civil and criminal justice system, and the ability to manipulate videos or images of a person, undetected, arguably poses a grave danger to a justice system which relies on our sense of sight and observation to establish objective fact. Perhaps even worse than framing the innocent could be failing to convict the guilty. In fact, a recent study in the journal Crime Science found that deepfakes pose a serious crime threat when it comes to audio and video impersonation and blackmail. What if a deepfake is used to replace a bad actor with a person who does not exist? Or gives plausible deniability to someone who claims that a video or image of them has been altered?

Deepfakes are also inherently dishonest. Two of the most popular social media networks, Instagram and TikTok, inherently rely upon visual media which could be subject to alteration by self-imposed deepfakes. Even if a person’s likeness is being manipulated with their consent and also could have positive consequences, it still might be considered wrong due to the dishonest nature of its content. Instagram in particular has been increasingly flooded with photoshopped images, as there is an entire app market that exists solely for editing photos of oneself, usually to appear more attractive. The morality of editing one’s photos has been hotly contested amongst users and between feminists. Deepfakes only stand to increase the amount of media that is self-edited and the moral debates that come along with putting altered media of oneself on the internet.

Proponents of deepfakes argue that their positive potential far outweighs the negative. Deepfake technology has been used to spark engagement with the arts and culture, and even to bring historical figures back to life, both for educational and entertainment purposes. Deepfakes also hold the potential to integrate AI into our lives in a more humanizing and personal manner. Others, who are aware of the possible negative consequences of deepfakes, argue that the development and research of this technology should not be impeded, as the advancement of the technology can also contribute to research methods of spotting it. And there is some evidence backing up this argument, as the development of deepfake progresses, so do the methods for detecting it. It is not the moral responsibility of those researching deepfake technology to stop, but rather the role of policymakers to ensure the types of harmful consequences mentioned above do not wreak havoc on the public. At the same time, proponents such as David Greene, of the Electronic Frontier Foundation, argue that too stringent limits on deepfake research and technology will “implicate the First Amendment.”

Perhaps then it is not the government nor deepfake creators who are responsible for their harmful consequences, but rather the platforms which make these consequences possible. Proponents might argue that the power of deepfakes is not necessarily from their ability to deceive one individual, but rather the media platforms on which they are allowed to spread. In an interview with Digital Trends, the creator of Ctrl Shift Face (a popular deepfake YouTube channel), contended that “If there ever will be a harmful deepfake, Facebook is the place where it will spread.” While this shift in responsibility might be appealing, detractors might ask how practical it truly is. Even websites that have tried to regulate deepfakes are having trouble doing so. Popular pornography website, PornHub, has banned deepfake videos, but still cannot fully regulate them. In 2019, a deepfake video of Ariana Grande was watched 9 million times before it was taken down.

In December, the first federal regulation pertaining to deepfakes passed through the House, the Senate, and was signed into law by President Trump. While increased government intervention to prevent the negative consequences of deepfakes will be celebrated by some, researchers and creators will undoubtedly push back on these efforts. Deepfakes are certainly not going anywhere for now, but it remains to be seen if the potentially responsible actors will work to ensure their consequences remain net-positive.

Posted on

On Tumblr, Adult Content is Banned – For Good?

Photo of a man speaking into a microphone, standing in front of a screen displaying a tumblr dashboard

In early December, blogging platform Tumblr announced that it would be banning adult content from its site starting mid-month. In a post explaining the decision, CEO Jeff D’Onofrio stated that removing such content would better allow Tumblr to be a “safe place for creative expression [and] self-discovery” and would result in “a place where more people feel comfortable expressing themselves.” D’Onofrio explained further that while he recognized that many users sought out Tumblr as a source of adult content, that “[t]here are no shortage of sites on the internet” that users could turn to. The content to be removed includes “images, videos, or GIFs that show real-life human genitals or female-presenting nipples—this includes content that is so photorealistic that it could be mistaken for featuring real-life humans (nice try, though)” although “certain types of artistic, educational, newsworthy, or political content featuring nudity” will be allowed to stay.

While creating safe places for expression and self-discovery are laudable goals, if a bit vague, recent problems with the platform are perhaps a better explanations of Tumblr’s decision. Notably, the fact that the Tumblr app was recently removed from the iOS App Store, ostensibly for the reason that Tumblr was not doing enough to ensure that illegal content – specifically in the form of child pornography – was being filtered out. Whether Tumblr’s decision to ban all adult content was based on genuine moral concern or simply concern for the bottom-line (it would no doubt be a major blow to Tumblr to be removed from the iOS store permanently), many online have speculated that the ban will spell the death of the platform. As Motherboard discovered, approximately a significant percentage of Tumblr blogs were based around providing adult content, with an even more significant percentage of users seeking out that content.

It is clear that Tumblr has a moral obligation to do as much as they can in order to prevent harmful material like child pornography from appearing on their platform. It also seems clear that Tumblr has, up until this point, failed to meet said obligation. The move to ban all adult content, then, might seem to be the straightforwardly right way to attempt to amend for their past transgressions, as well as to prevent further such harms in the future. And indeed, it may very well be the case that, overall, Tumblr’s ban on adult content will result in the prevention of many further harms (especially given many of the problems inherent to the type of pornography that tends to be propagated online).

Nevertheless, there has been a good amount of negative response to Tumblr’s decision, as well. In general, people appear to be expressing three main types of worries. The first is that Tumblr’s new filters are bad at distinguishing the kind of content they want to ban from content that they have deemed inoffensive, and thus that an attempted universal ban on adult content will potentially stifle legitimate creative expression; second, that Tumblr is being hypocritical in banning of adult content but not, for example, taking strides to address other problems on its platform, especially those involving hate speech; and third, that the ban on adult content disproportionately affects users from marginalized groups.

With regards to the first worry, many Tumblr users have already noticed that their filters are not great at separating inappropriate from appropriate content. As The Guardian reports, Tumblr has already flagged as inappropriate images of fully-clothed historical figures, artists, and ballet dancers, as well as a painting of Jesus in a loincloth. If Tumblr is meant to be a site that encourages self-expression, stifling said expression as a result of bad programming does not seem to be the best way to achieve this goal.

The second concern pertains to Tumblr’s policies generally about what kind of content is deemed acceptable on the platform. As The Washington Post reports, while searches involving sexually explicit terms on Tumblr gave no results, “racist and white supremacist content, including Nazi propaganda, was easily surfaced”, despite the fact that such posts violate Tumblr’s policies surrounding hate speech. It seems that just as Tumblr has an obligation to attempt to prevent illegal pornographic content, it similarly has obligations with respect to preventing users from promoting hate speech. It seems hypocritical, however, to focus only on one type of content and not another, especially given the widespread harms that can result from the dissemination of the type of racist and white supremacist content that is easily searchable on the platform.

The final worry pertains to the way that Tumblr’s new ban, combined with inefficient filtering technology, could potentially impact members of marginalized communities. Writing for BBC News, David Lee argues that,

“Unlike typical pornography sites, which overwhelmingly cater to men, and serve an often narrow definition of what is attractive, Tumblr has been a home for something else – content tailored at vibrant LGBT communities, or for those with tastes you might not necessarily share with all your friends.”

In a post on his blog, actor Wil Wheaton concurs:

“The reality is that for a lot of the LGTBQ+ community, particularly younger members still discovering themselves and members in extremely homophobic environments where most media sites were banned (but Tumblr wasn’t even considered important enough to be), this was a bastion of information and self-expression.”

Wheaton supported his view by performing an experiment, one in which he posted a series of images of “beautiful men kissing” on Tumblr. The post was flagged by as inappropriate, despite the images not being pornographic or in any clear violation of Tumblr’s newly stated policies. Wheaton laments that “it’s ludicrous and insulting that – especially in 2018 – this is flagged, either by some sort of badly-designed algorithm, or by shitty homophobic people”. Finally, Kaila Hale-Stern writing at The Mary Sue, argues that

“What D’Onofrio and his corporate overlords at Verizon’s Oath [who own Tumblr] don’t understand—or don’t care about—is that this sort of adult content is frequently generated by women, marginalized people, and all sorts of creatives struggling in our vicious ‘gig economy.’ They’re going to be hurt the most by the ban.”

Aside from stifling self-expression, then, Hale-Stern points to a more tangible harm in the form of creators of adult content losing their livelihoods as a result of Tumblr’s new ban.

What initially appeared as a seemingly straightforward fulfilment of Tumblr’s obligations to prevent harmful content from appearing on their platform is, on closer inspection, more complicated. As many have argued, there is perhaps room for a middle-ground: instead of issuing a universal ban on all adult content on the platform, Tumblr could have done a better job of implementing more effective algorithms to detect and filter out offensive content without removing the means for self-expression and livelihoods of many users (better filtering could also, hopefully, address concerns about hate speech, as well).

Posted on

Law Enforcement Surveillance and the Protection of Civil Liberties

In a sting operation conducted by the FBI in 2015, over 8,000 IP addresses in 120 countries were collected in an effort to take down the website Playpen and its users. Playpen was a communal website that operates on the Dark Web through the Tor browser. Essentially, the site was used to collect images related to child pornography and extreme child abuse. At its peak, Playpen had a community of around 215,000 members and more than 117,000 posts, with 11,000 unique visitors a week.

Continue reading “Law Enforcement Surveillance and the Protection of Civil Liberties”