LOADING ...

Discord Explains How It Handles Harassment, Doxxing, And Threatening Behavior

Nathan Grayson Aug 14, 2019. 6 comments

Discord, the only way young people communicate via voice after we collectively agreed to stop answering phone calls several years ago, is trying to be more open about how it handles harassment, threats, doxxing, and other forms of abuse on its platform. In order to do this, it plans to release regular “transparency reports,” the first of which is available to peruse now.

The idea, says Discord, is to keep people clued into the decision-making underlying the 250-million-user chat service so that people can understand why things do and, in some cases, don’t get done.

“We believe that publishing a Transparency Report is an important part of our accountability to you—it’s kind of like a city’s crime statistics,” said the company. “We made it our goal not only to provide the numbers for you to see, but also to walk you through some obstacles that we face every day so that you can understand why we make the decisions we do and what the results of those decisions are.”

The numbers are fascinating, albeit not wholly unexpected if you display symptoms of being Too Online. First, a graph of reports received by Discord from the start of the year until April:

“Other” is the most reported category, but harassment comes in behind it at 17.3 percent. Significant pieces of the multi-colored pie also go to hacks/cheats, threatening behavior, NSFW content, exploitative content, doxxing, spamming, raiding, malware, and self-harm. Discord does not mince words in describing what these categories refer to. Exploitative content is defined as “A user discovers their intimate photos are being shared without their consent; two minors’ flirting leads to trading intimate images with each other” while one example of NSFW content is “A user joins a server and proceeds to DM bloody and graphic violent images (gore) to other server members.”

After Discord receives a report, the company’s trust and safety team “acts as detectives, looking through the available evidence and gathering as much information as possible.” Initially, they focus on reported messages, but investigations can expand into entire servers “dedicated to bad behavior” or historical patterns of rule-breaking. “We spend a lot of time here because we believe the context in which something is posted is important and can change the meaning entirely (like whether something’s said in jest, or is just plain harassment),” wrote Discord.

If there’s a violation, the team then takes action, which can mean anything from removing the content in question to removing a whole server from Discord. However, the percentage of reports that caused Discord to spring into action earlier this year was relatively small. Just 12.49 percent of harassment reports got actioned, for example. Other categories saw Discord intervene more often, but most percentages were still relatively small: 33.17 percent for threatening behavior, 6.86 percent for self-harm, 44.34 percent for cheats/hacks, 41.72 percent for doxxing, 60.15 percent for malware, 14.74 percent for exploitative content, 58.09 percent for NSFW content, 28.72 percent for raiding, and the big outlier, 95.09 percent for spam.

Discord explained that action percentages are lower than you might expect because many reports simply don’t pass muster. Some are false or malicious, with people taking words out of context or banding together to report innocent users. Others demand too much for too little. “We may receive a harassment report about a user who said to another user, ‘I hate your dog,’ and the reporter wants the other user banned,” said the company. Other reports that Discord doesn’t action might include information, but no concrete evidence. Lastly, users sometimes apply the wrong category to reports, but Discord says it still actions those, but it may not count toward action percentages.

From January to April, the biggest contributors to bans were spam and exploitative content. Spam accounted for 89 percent of account bans—a total of 119,244 accounts. On the exploitative content side of things, Discord banned 10,642 accounts, and it says it’s doing its best to squelch that issue—perhaps due to its well-documented troubles with child porn in the past.

“We’ve been spending significant resources on proactively handling Exploitative Content, which encompasses non-consensual pornography (revenge porn/’deep fakes’) and any content that falls under child safety (which includes child sexual abuse material, lolicon, minors accessing inappropriate material, and more),” the company wrote. “We think that it is important to take a very strong stance against this content, and while only some four thousand reports of this behavior were made to us, we took action on tens of thousands of users and thousands of servers that were involved in some form of this activity.”

Discord also removed thousands of servers during the first few months of the year, mostly focusing on hacks/cheats. However, servers focused on hate speech, harassment, and “dangerous ideologies”—again, an area where Discord has struggled in the past —are also a big focus. On a related note, the company also discussed its response to videos and memes born of the March 14 Christchurch shooting.

“At first, our primary goal was removing the graphic video as quickly as possible, wherever users may have shared it,” the company said. “Although we received fewer reports about the video in the following hours, we saw an increase in reports of ‘affiliated content.’ We took aggressive action to remove users glorifying the attack and impersonating the shooter; we took action on servers dedicated to dissecting the shooter’s manifesto, servers in support of the shooter’s agenda, and even memes that were made and distributed around the shooting... Over the course of the first ten days after this horrific event, we received a few hundred reports about content related to the shooting and issued 1,397 account bans alongside 159 server removals for related violations.”

Discord closed out the report by saying it believes that this sort of transparency should be “a norm” among tech companies, so that people can “better determine how platforms keep their users safe.” The Anti-Defamation League, an anti-hate organization dedicated to fighting bigotry and defending civil rights, agrees.

“Discord’s first transparency report is a meaningful step toward real tech platform transparency, which other platforms can learn from,” the organization wrote in a statement about Discord’s report. “We look forward to collaborating with them to further expand their transparency efforts, so that the public, government and civil society can better understand the nature and workings of the online platforms that are and will continue to shape our society.”

6 Comments

Other Nathan Grayson's posts

Halo: Reach, As Told By Steam Reviews Halo: Reach, As Told By Steam Reviews

SteamedSteamedSteamed is dedicated to all things in and around Valve’s PC gaming service.Prev NextView All Halo: Reach, Bungie’s final entry in its second (of three, so far) series about armored space knights, is finally on PC. People are clearly excited, so much so that they managed to propel it to the third most-played slot on Steam earlier this week....

The State Of PC Gaming In 2019 The State Of PC Gaming In 2019

Year In ReviewYear In ReviewWe look back at the highs, lows, surprises, and standouts in and around video games this year.Prev NextView All Sometimes, as borders erode, new fissures form in their wake. In 2019, more games flowed between PC and other platforms than ever before, but a single major division cracked the PC gaming landscape in half.This year’s...

Psychiatrist Brings Streamer To Tears During Confessional Stream Psychiatrist Brings Streamer To Tears During Confessional Stream

Everyone could benefit from therapy, especially streamers. While popular streamer Byron “Reckful” Bernstein’s on-stream conversation with Dr. Alok Kanojia was not an officially designated counseling session, it resembled one at times.Bernstein is a former top World of Warcraft player and current streamer who’s now working on his own online game, Everland. Over the years, he’s been fairly public about...

Twitch Briefly Bans One Of The Biggest Moderation Bots Twitch Briefly Bans One Of The Biggest Moderation Bots

If allowed to chatter and meme unchecked, Twitch chat can grow into an unstoppable avalanche of ugly behavior. Many streamers rely on at least a handful of volunteer or paid moderators, as well as chat bots that allow streamers to auto-block specific terms, stop spam before it starts, and quarantine questionable comments. Yesterday, Twitch accidentally banned one of the...

Suggested posts

Mario Maker 2's New Speedrunning Mode Looks A Lot Like A Popular Fan-Made Hack Mario Maker 2's New Speedrunning Mode Looks A Lot Like A Popular Fan-Made Hack

In addition to the ability to play as Link, Super Mario Maker 2’s latest update has added a new speedrunning mode that’s remarkably similar to one rom hacker’s mod of an SNES emulator called Quantum Mario. This has raised questions about Nintendo’s relationship with the Mario homebrew community and the designers that power its Mario Maker games.“In 2008 I...

What Are You Playing This Weekend? What Are You Playing This Weekend?

The weekend is for finally fixing my endlessly running toilet, a process internet videos say is very easy but has thusfar involved me opening the toilet tank, staring in confusion, and then closing the tank again. The weekend is also for playing video games while I avoid this task.I’m excited to check out Mosaic now that it’s on Steam....

The Latest Paper Mario-Style Steven Universe Game Keeps It Simple The Latest Paper Mario-Style Steven Universe Game Keeps It Simple

In 2015, Cartoon Network and developer Grumpyface Studios released Steven Universe: Attack the Light , a simple yet delightful turn-based mobile RPG with strong Paper Mario vibes. 2017’s console follow-up, Save the Light , beefed up the series with intricate combo attacks and free-roaming exploration. A new entry in the series recently released for Apple Arcade, Steven Universe: Unleash...

Cult Classic Game Where You Can Raise A Princess Of Hell Coming To Switch Cult Classic Game Where You Can Raise A Princess Of Hell Coming To Switch

Child raising simulator Princess Maker 3 is coming to Switch on December 23, and—no, wait, come back! I promise it’s cool.In Princess Maker 3, you schedule out your daughter’s time between various school and work activities to raise her skills. You can also take her on adventures, which are basic RPG sequences where your daughter can explore and fight...

Halo: Reach, As Told By Steam Reviews Halo: Reach, As Told By Steam Reviews

SteamedSteamedSteamed is dedicated to all things in and around Valve’s PC gaming service.Prev NextView All Halo: Reach, Bungie’s final entry in its second (of three, so far) series about armored space knights, is finally on PC. People are clearly excited, so much so that they managed to propel it to the third most-played slot on Steam earlier this week....

Goodbye To Cecilia D'Anastasio, Planeswalking Cyber Sleuth And Aspiring Palutena Main Goodbye To Cecilia D'Anastasio, Planeswalking Cyber Sleuth And Aspiring Palutena Main

Cecilia D’Anastasio, Kotaku Senior Reporter and resident Overwatchologist, has completed her last sidequest around these parts today and is heading off to new adventures elsewhere. We’d like to wish Cecilia the best as she embarks on the newest chapter in her career and tries to sell a whole new group of strangers on the merits of binge-watching Fooly Cooley...

In Defense Of Hop, The Unlucky Pokémon Rival In Defense Of Hop, The Unlucky Pokémon Rival

I will not hear any more slander about Hop, your rival in Pokémon Sword and Shield.Your rivals in the more recent Pokémon games have trended more towards “childhood pal” than the “bigtime assholes” you had to trounce in the older ones. I have strong memories of wanting to just beat the shit out of Blue from Red and Blue...

Rocket League Replaces Loot Boxes With Pricey Item Shop Rocket League Replaces Loot Boxes With Pricey Item Shop

Rocket League players will no longer have to worry about grinding for new crates and the keys that unlock them in order to get the custom skins they want. But the new alternative will hurt their wallets.Yesterday, Rocket League’s Blueprint update went live, kicking off season 12 and adding a new item shop where players can just buy whatever...

The State Of PC Gaming In 2019 The State Of PC Gaming In 2019

Year In ReviewYear In ReviewWe look back at the highs, lows, surprises, and standouts in and around video games this year.Prev NextView All Sometimes, as borders erode, new fissures form in their wake. In 2019, more games flowed between PC and other platforms than ever before, but a single major division cracked the PC gaming landscape in half.This year’s...

You Should Play Life Is Strange 2 You Should Play Life Is Strange 2

The final episode of Life Is Strange 2, “Wolves,” released on December 3 for PC, PlayStation 4, and Xbox One. When I try to understand why people did not react with an open outpouring of excitement and glee for Life Is Strange 2 after the success of its predecessor, I think of Marvel movies. Superhero films strike me as an...

Language