← Return to search results
Back to Prindle Institute

Content Moderation and Emotional Trauma

image of wall of tvs each displaying something different

In the wake of the Russian invasion of Ukraine, which has been raging violently since February 24th of 2022, Facebook (now known as “Meta”) recently announced its decision to change some of its content-moderation rules. In particular, Meta will now allow for some calls for violence against “Russian invaders,” though Meta emphasized that credible death threats against specific individuals would still be banned.

“As a result of the Russian invasion of Ukraine we have temporarily made allowances for forms of political expression that would normally violate our rules like violent speech such as ‘death to the Russian invaders.’ We still won’t allow credible calls for violence against Russian civilians,” spokesman Andy Stone said.

This recent announcement has reignited a discussion of the rationale — or lack thereof — of content moderation rules. The Washington Post reported on the high-level discussion around social media content moderation guidelines: how these guidelines are often reactionary, inconsistently-applied, and not principle-based.

Facebook frequently changes its content moderation rules and has been criticized by its own independent Oversight Board for having rules that are inconsistent. The company, for example, created an exception to its hate speech rules for world leaders but was never clear which leaders got the exception or why.

Still, politicians, academics, and lobbyists continue to call for stricter content moderation. For example, take the “Health Misinformation Act of 2021”, introduced by Senators Amy Klobuchar (D-Minnesota) and Ben Ray Luján (D-New Mexico) in July of 2021. This bill, a response to online misinformation during the COVID-19 pandemic, would revoke certain legal protections for any interactive computer service, e.g., social media websites, that “promotes…health misinformation through an algorithm.” The purpose of this bill is to incentivize internet companies to take greater measures to combat the spread of misinformation by engaging in content-moderation measures.

What is often left out of these discussions, however, is the means by which content moderation happens. It is often assumed that such a monumental task must be left up to algorithms, which can scour through mind-numbing amounts of content at a breakneck speed. However, much of the labor of content-moderation is performed by humans. And in many cases, these human content-moderators are poor laborers working in developing nations for an extremely small salary. For example, employees at Sama, a Kenyan technology company that is the direct employer of Facebook’s Kenya-based content moderators, “remain some of Facebook’s lowest-paid workers anywhere in the world.” While U.S.-based moderators are typically paid a starting wage of $18/hour, Sama moderators make an average of $2.20/hour. And this low wage is their salary after a recent pay-increase, which happened a few weeks ago. Prior to that, Sama moderators made $1.50/hour.

Such low wages, especially for labor outsourced to poor or developing nations, is nothing new. However, content moderation can be a particularly harrowing — in some cases, traumatizing — line of work. In their paper “Corporeal Moderation: Digital Labour as Affective Good,” Dr. Rae Jereza interviews one content moderator named Olivia about her daily work, which includes identifying “non‐moving bod[ies]”, visible within a frame, “following an act of violence or traumatic experience that could reasonably result in death.” The purpose of this is so videos containing dead bodies can be flagged as containing disturbing content. This content moderator confesses to watching violent or otherwise disturbing content prior to her shift, in an effort to desensitize herself to the content she would have to pick through as part of her job. The content that she was asked to moderate ranged over many categories, including “hate speech, child exploitation imagery (CEI), adult nudity and more.”

Many kinds of jobs involve potentially traumatizing duties: military personnel, police, first responders, slaughterhouse and factory farm workers, and social workers all work jobs with high rates of trauma and other kinds of emotional/psychological distress. Some of these jobs are also compensated very poorly — for example, factory and industrial farms primarily hire immigrants (many undocumented) willing to work for pennies on the dollar in dangerous conditions. Poorly-compensated high-risk jobs tend to be filled by people in the most desperate conditions, and these workers often end up in dangerous employment situations that they are nevertheless unable or unwilling to leave. Such instances may constitute a case of exploitation: someone exploits someone else when they take unfair advantage of the other’s vulnerable state. But not all instances of exploitation leave the exploited person worse-off, all things considered. The philosopher Jason Brennan describes the following case of exploitation:

Drowning Man: Peter’s boat capsizes in the ocean. He will soon drown. Ed comes along in a boat. He says to Peter, “I’ll save you from drowning, but only if you provide me with 50% of your future earnings.” Peter angrily agrees.

In this example, the drowning man is made better-off even though his vulnerability was taken advantage of. Just like this case, certain unpleasant or dangerous lines of work may be exploitative, but may ultimately make the exploited employees better-off. After all, most people would prefer poor work conditions to life in extreme poverty. Still, there seems to be a clear moral difference between different instances of mutually-beneficial exploitation. Requiring interest on a loan given to a financially-desperate acquaintance may be exploitative to some extent, but is surely not as morally egregious as forcing someone to give up their child in exchange for saving their life. What we demand in exchange for the benefit morally matters. Can it even be permissible to demand emotional and mental vulnerability in exchange for a living wage (or possibly less)?

Additionally, there is something unique about content moderation in that the traumatic material moderators view on any given day is not a potential hazard of the job — it is the whole job. How should we think about the permissibility of hiring people to moderate content too disturbing for the eyes of the general public? How can we ask some people to weed out traumatizing, pornographic, racist, threatening posts, so that others don’t have to see it? Fixing the low compensation rates may help with some of the sticky ethical issues concerning this sort of work. Yet, it is unclear whether any amount of compensation can truly make hiring people for this line of work permissible. How can you put a price on mental well-being, on humane sensitivity to violence and hate?

On the other hand, the alternatives are similarly bleak. There seem to be few good options when it comes to cleaning up the dregs of virtual hate, abuse, and shock-material.

On the Just Treatment of Appalachian Coal Towns

A small town main street with a green mountain in the background and an American flag

Detroit had automobiles, Pittsburgh had steel, and Boston had textiles. Amongst these former capitals of industrialization in the United States is one that is seldom mentioned: the coal kingdom of Hazard, Kentucky. Hazard started booming as a coal town in 1912 when a railway along the North Fork of the Kentucky River was built with a station there. Soon thereafter, nearby coal mines started opening frequently, making for an abundance of jobs and a rapidly growing population. Between 1920 and 1930, the population of Hazard grew from 4,348 to over 7,000. The small city still hosts a Black Gold Festival every September to celebrate its industrial roots.

However, Hazard’s once rich history has now turned into severe poverty, unemployment, and addiction. The population is currently 5,600—shrinking over 1,800 since its peak in 1940 despite heavy national population growth. Additionally, Hazard’s median household income of $20,690 is almost $40,000 under the U.S. median, with about 31% of its small population living below the poverty line. Perhaps the darkest aspect of Hazard’s downturn is the opioid addiction crisis. Perry County, Kentucky (where Hazard is located) had the most opioid-related hospitalizations in the nation in 2017, with 6% of the county’s population having been hospitalized due to opioid abuse. Three counties surrounding Perry were also in the top ten in the nation.

Hazard, however, is not isolated in the challenges it faces. Towns all over Appalachia have experienced severe downturn characterized by these same issues. To illustrate, six of the 15 states with the highest opioid death rates are Appalachian states (West Virginia is first with 43.4 out of every 100,000 people dying of opioid overdose, followed by Ohio at third, Maryland at fourth, Kentucky at 10th, Pennsylvania at 12th, and Tennessee at 14th). Between the years 1999 and 2016, the number of opioid-related deaths in Pennsylvania jumped 736 percent, while those in Kentucky, Ohio, and West Virginia jumped over 1,000 percent. These rates are compared to a national increase of 528 percent. With the opioid epidemic, Appalachian states also deal with unusually high poverty and crumbling infrastructure that has led to the contamination of drinking water.

Why is it that these challenges are so inflated in Appalachia? Why is a region of the U.S. that once prospered and was overflowing with opportunities for lucrative employment is now rapidly decaying? The most obvious reason is the decline in demand for coal. U.S. coal consumption peaked as recently as 2007 but has dropped 44 percent since then, reaching its lowest level in almost 40 years. This number is expected to drop another eight percent this year. Additionally, of the 1,470 coal-fired generators in the U.S. in 2007, 540 have been retired as of September 2018. The decrease in demand has inevitably resulted in a decrease in coal production, and therefore a decrease in coal jobs. Coal mining jobs in the U.S. have dropped a whopping 71 percent since 1985. This effect is more pronounced in Appalachia. Coal production in Appalachia fell 45 percent from 2005 to 2015, resulting in a loss of about 33,500 mining jobs since 2011. In Hazard in particular, the local coal industry now produces approximately 4.1 million tons of coal annually, dropping about 13 million tons since 2008, and losing about 13,000 coal jobs in eastern Kentucky since 2011. This year marks the lowest number of coal jobs Kentucky has had since 1898.

People across the nation have noticed Appalachia’s suffering, and have searched for causes behind it. Donald Trump in particular ran on a platform that included the revival of the Appalachian coal industry. He puts most of the blame for its recent lack of success on environmental regulations imposed by Democratic politicians. In one tweet referring to the Obama administration’s regulations on carbon emissions, Trump wrote, “Obama’s coal regulations will destroy the coal industry, put Americans out of work, raise electricity prices, and lead to blackouts.” Trump continues to tweet into his presidency about how he has “ended the war on coal.” The coal town of Hazard seems to agree, as 77.2 percent of voters in Perry County, Kentucky voted for Trump in 2016. However, Trump’s assumption that environmental regulations are destroying the coal industry and, by association, the economy of Appalachia, is overly simplistic. In fact, environmental regulations only account for approximately 3.5 percent of the total decline in U.S. coal production. The reality is that the coal industry is dying primarily of natural causes.

There are two main reasons behind the Appalachian coal industry’s decline: decreasing productivity, and competition with other energy sources. The former is caused by the dwindling supply of easily-accessible coal deposits in the Appalachian Mountains. After almost a century and a half of mining, miners now have to dig deeper into the mountains for usable coal. To offset this increased difficulty in mining, coal corporations have turned towards mechanization, which puts even more miners out of work. Even with a mechanized workforce, however, Appalachian coal cannot keep up with coal production in western states such as Wyoming, Montana, and Texas, where deposits are closer to the surface and therefore require less time and money to mine.

Along with challenges the coal industry faces to keep up production, it must also deal with higher competition with other sources of energy, namely renewable energy (wind power, solar power, etc.) and natural gas. Renewable energy is beginning to compete with coal because of rising concern amongst Americans about the environment. Burning coal is one of most harmful sources of electricity in terms of carbon emissions, and while it is cheap, Americans are beginning to prioritize environmental harms caused by energy sources over the economic costs they incur. This gives renewable energy sources (which are typically more expensive) an advantage over coal. Most competitive with the coal industry, however, is natural gas. In 2016, natural gas surpassed coal as the U.S.’s leading source of electricity. This is largely due to falling natural gas prices after the fracking boom of the late 2000’s. Natural gas is also cheaper to produce than coal, and burning natural gas puts out significantly less carbon emissions than coal.

Because the Appalachian coal industry is already becoming obsolete at the hands of struggling productivity and a more competitive energy market, it would be foolish and reckless for politicians to attempt breathing life back into it. Rather, these efforts should be put into diversifying Appalachia’s economy. Some coal towns are already beginning to do this. The residents of Hinton, West Virginia, for example, have started the Appalachian Beekeeping Collective, a nonprofit aimed at training out-of-work coal miners in beekeeping. This nonprofit has grown to operate in 17 counties throughout the state of West Virginia in just two years since its creation, and helps ex-miners earn money in a more sustainable field of work. Additionally, the state of Pennsylvania, which has traditionally been a coal state and is still one of the biggest energy producers in the U.S., is moving its economy to subsist on nuclear power and natural gas while coal is on the decline. Even Hazard is turning the site of one of its former surface mines into the USA Drone Port—a research and testing facility for drone companies to use. To supplement this, the Hazard Community and Technical College has introduced courses in unmanned technology, drone flight, photography, and videography: all skills that will translate well into USA Drone Port’s workforce. The Technical College has already graduated 200 students in these fields, many of whom used to be coal miners.

Unfortunately, the coal industry does not appear to be a significant part of the future U.S. economy. However, adapting to the future is possible, and many places around Appalachia are showing it. Appalachia is a region full of innovation, grit, and most importantly, people who are ready to work for the betterment of their communities, with or without coal.