TikTok algorithm produces viral videos and harassment for creators

When Junna Faylee started making money on TikTok, the 21-year-old anime and gaming fan in London made it the centerpiece of her life. She devoted every night and weekend to making videos. She optimized her room in eye-catching pink. And she hired a management team to handle her video-branding deals and bookkeeping, even though she still lived at home.

Now, as the 9-million-follower “nintendo.grl,” she is one of the app’s biggest successes, and she feels like she’s achieved a creative dream. But competing for attention, she said, can often feel like working a shift that doesn’t end. And winning it can feel even worse, since her most viral videos also bring on the heaviest floods of hateful insults and sexist trolls. She has woken up in the middle of the night to check her phone and, after some videos, has refused to sleep, feeling too anxious about the response.

“There is this power TikTok has: It’s just so, so popular, and that can be a scary thing. … You have to be constantly fighting against other content creators to be seen,” she said.

“You don’t realize the impact of having so many eyes on you,” she added. “Those people who’ve chosen not to like you, they’re going to see you, right there on their screen, and nothing you do is going to make a difference. You’ve got to learn to deal with the hate.”

TikTok has become the world’s biggest gatekeeper for online fame, and its rise has helped supercharge an internet reality: One great moment can be the difference between a celebrity and a nobody. The app’s promise is especially attractive to the millions of young people aspiring to a life or career as an online influencer. What if anyone could be a star?

But this new era of instant, inexplicable attention has also come at a price. In interviews with more than three dozen TikTok creators, many noted that the app’s reach often brings with it relentless demands: from angry commenters, from audience expectations, even from the algorithm itself.

TikTok’s sticky, colorful, endlessly amusing platform promises a scale of virality unlike anything else on the internet, and its algorithm is built to blast creators’ videos to crowds of anonymous strangers in hopes of maximizing their spread.

But TikTok is designed for entertainment, and no one is guaranteed a receptive audience. The app’s culture of fast-twitch reactions and fleeting fame has left many creators feeling overwhelmed by dashed-off insults or mean-spirited critiques. And getting big too quickly, some said, often meant their videos would be seen by viewers who hated what they said, how they looked or who they were — and had many ways to tear them down.

“The phrase people use is ‘getting on the wrong side of TikTok,’” said Casey Fiesler, an associate professor at the University of Colorado at Boulder who researches online communities. “One person told me, ‘I wish I could just stop my videos at 30,000 views.’ ”

Some TikTok creators said they were surprised by how reactive and cruel their videos’ commenters could be: Some have received death threats for supporting abortion, making cheesy videos or selling expensive chocolate bars.

But it’s TikTok’s collaborative video tools that have really made its “hate comments come to life,” said Josh Helfgott, 33, who posts about LGBTQ issues to his more than 5 million followers. People often post quick reactions to others’ videos, using features like “duet” and “stitch,” that leverage the original videos’ virality to amplify their insults in what he called “a new form of bullying.” “The harsher you are to the original video,” he said, “the more views you’re going to get.”

Harassment is an internet-wide issue, and TikTok has said it’s working to strengthen its creator protections, including by allowing people to automatically block offensive comments. The company said last month it had removed more than 100 million videos between April and June for harassment, “hateful behavior” and other concerns — roughly 1 percent of the 11 billion videos that had been posted in those three months.

“We want people to feel safe, welcome and in control of their experience,” said Hilary McQuaide, a TikTok spokeswoman. “We pair robust safety policies against bullying, harassment and hateful behavior with tools to empower creators to decide who can watch and engage.”

But some TikTokers said they worry the app’s spotty policing and explosive popularity have left creators vulnerable to attack. Where Facebook’s algorithms once stoked in-group anger to boost engagement, TikTok has opened those groups to everyone, expanding the problem to global scale.

“Everything on TikTok is bigger. The views are bigger, the follower counts are higher, the love is louder, and the hate is way, way more enormous,” Helfgott said. “In the first second of the video people will be shouting insults into the camera. If your face is bright red or your eyes are welling up, people will stop scrolling and you’ll get more views. TikTok encourages emotion, and the most engaging emotion is anger.”

15 seconds of fame

The first brush with TikTok fame can be easy.

Creators don’t need an existing audience to win attention — the algorithm handles that — and the app’s viral popularity has quickly christened a new class of stars. Roughly 40,000 TikTok accounts have more than a million followers, compared with 23,000 on Instagram, even though the latter has been around twice as long, data from the analytics firm Social Blade shows.

Brandon Conway, 22, was lying in bed one night this summer at his family’s cattle farm outside Athens, Ga., when he posted his first TikTok, showing him singing Michael Jackson in a karaoke bar parking lot. Within a day, the video had soared past 9 million views, he’d gained 170,000 followers, and he’d started dreaming of life beyond his DJ night shifts at a local Italian grill. “I just want to be the next person” to make it big, he said.

TikTok viewers said they love the app because its algorithmic recommendations predict their interests and take away the anxiety of choice. And the personality-driven format encourages authenticity and intimacy; many videos are designed as if the creator is speaking directly through their screen.

Maribel Martinez, a 24-year-old mother in New Port Richey, Fla., went viral on TikTok after she dressed up as the superstrong Luisa from the Disney movie “Encanto,” gaining 2 million followers within three months. “You build this audience that tends to be more like a second family, and your followers really go in for you, they defend you,” she said. “But the hate we get, the rumors, the lies that get told about us — it can spread like wildfire.”

And keeping people’s attention can be a challenge, creators said. No one knows exactly what the algorithm rewards or punishes, leading many of them to regularly recalibrate what they talk about or how they behave in hopes of garnering its blessing. Several creators shared folk theories about the app’s most virally promising hashtags, topics or trends.

The mystery around how the algorithm chooses winners and losers fuels a constant sense of competitive anxiety. Some creators said their connections to their followers seemed more disposable and easily replaced than on other sites, leaving them feeling always one swipe away from irrelevance.

“The thing about a trend is: It only trends for a few months, if you’re lucky,” said one TikTok creator with more than 10 million followers, who spoke on the condition of anonymity so as not to affect their relationship with the company. “People get excited: ‘I got 200,000 followers!’ They don’t realize, once that trend’s done, it’s like: What are they going to do next?”

Just as TikTok’s algorithm rapidly promotes certain videos, it can also silently downrank or “shadowban” creators or videos for breaking its indiscernible rules, leaving many creators scrambling to proactively censor themselves. Hank Green, one of the app’s biggest stars, said in a widely watched YouTube video earlier this year that creators were “terrified” of the platform’s “haphazard” moderation systems, which banned people with little explanation.

“TikTokers are really scared, honestly, of criticizing the platform, and that’s just a recipe for unintentional exploitation,” he said.

When Belle Ives, a freelance photographer in Los Angeles who uses they and them pronouns, posted a video last year of two women kissing, TikTok removed it for breaking the company’s underage-safety rules against “predatory or grooming behavior.” When Ives joked that a Pride Month cake was a “gayke,” TikTok deleted the comment as “hate speech.”

Ives assumed these were simple mistakes. Then, in June, Ives posted a video showing police officers shoving protesters at an abortion rights rally, and TikTok not only removed the clip for “violent or graphic” content, it denied an appeal and threatened a permanent ban. Ives, who has 50,000 TikTok followers and sees the app as a career necessity, believed she had few options but to comply.

“TikTok has made it very clear they want their platform to be this joyous, silly, content app, but they outgrew that so long ago,” Ives said. “There’s a lot of reporting on TikTok: newsworthy content, activists speaking up about movements. And yet they’re worried they can’t even use the correct language … or they’ll get taken down.”

The TikTok ‘mob’

TikTok does not just create instant heroes. It also helps designate viral villains — and assembles audiences eager to take them down.

TikTok, creators said, has become a highly specialized and powerful tool for online witch hunts and pile-ons. Its viewers love drama. Its creators have shown that mocking people is a quick way to go viral. And its algorithm is optimized to build and promote can’t-look-away storylines.

The app’s tools make it easy to post quick-reaction videos, and the algorithm then shows those videos to the people it expects are likely to have an emotional response. And because TikTok dynamically creates new trends based on viewers’ behaviors, creators said hate campaigns can end up growing automatically. Unlike on Instagram or Twitter, interested viewers don’t have to actively share the posts for them to go viral; TikTok will just pick up on the fact that people are watching and give them an algorithmic boost.

Ginny Di, a 32-year-old content creator in Denver who asked that her real name not be used because of harassment concerns, said that even a decade of making videos on most major social platforms had not prepared her for the level of bile she received on TikTok.

The community is “particularly aggressive, full of really mean, bully-type behavior, where people are comfortable telling you to just go kill yourself,” she said. “It made it so that the better a video did, the worse I felt, the more upset I would get.”

And TikTok hate campaigns often bleed into offline life. Many creators said they’d dealt with a regular battery of threats, from people sharing their home addresses to making fake “swatting” calls to provoke the police. Griffin Brooks, a senior at Princeton University who uses they and them as pronouns, said people had called their family members and tried to get them suspended from school because of some TikTok videos supporting LGBTQ activism.

TikTok is all about the “mob mentality,” Brooks said. “People are all ganging up, and it’s really easy to jump in.”

Some of TikTok’s most prominent movements have fueled crowdsourced criminal investigations. When Gabby Petito went missing during a cross-country road trip last year with her fiance, amateur TikTok detectives scoured the internet for clues, compiled (sometimes baseless) theory videos and shared information with the FBI.

But some of these crusades have also spiraled out of control, forming networked hunts for anyone designated the enemy of the day. After a 19-second clip of the college student Robert McCoy’s less-than-enthusiastic greeting of his girlfriend last year went viral, McCoy — dubbed the “Couch Guy” — became the target of a vicious harassment campaign.

The response videos — “frame-by-frame body language analyses, armchair diagnoses of psychopathy, comparisons to convicted murderers” — led to people sharing his personal information and slipping notes under his door, McCoy wrote in Slate last year. “It felt like the entertainment value of the meme began to overshadow our humanity,” he said.

When Kit Graham, a 38-year-old food blogger in Chicago, was told last year that a TikTok account using her name and photo had started leaving racist comments on people’s profiles, she quickly asked TikTok to delete the impersonating account. But in the three weeks it took for the company to take action, TikTok exploded with reaction videos from creators attacking her, displaying her face alongside the racist comments in clips that amassed tens of thousands of views.

Graham and her husband were flooded with hate mail and ruthlessly harassed, with some people sharing their home address and random facts about their lives gleaned from the internet.

“People were digging deep and digging quickly, and they were all collaborating,” she said. “Once someone sees one video getting a lot of steam, people start doubling down and creating more videos. Comments flood in and people create their own reaction videos. They want to hop on a story.”

An ‘intimate’ lie

TikTok argues that it is working to make the app a healthy place to scroll. The company said it changed its algorithm last year to reduce the promotion of videos that would be “potentially problematic if viewed repeatedly,” such as those about “dieting” or “sadness.” It has launched programs for mental health, hired managers to develop rules against “harassing and bullying behavior” and said it would start giving videos an automatically generated “maturity score” to prevent younger viewers from seeing videos that dealt with “complex themes.”

TikTok’s creators have used the app to exchange accurate information and healthy debate, debunking disinformation, squashing sexism, sharing political commentary and explaining current events. But its immediacy and reach have also made the app unusually effective at spreading misinformation and viral lies, said Abbie Richards, a researcher at the Accelerationism Research Consortium, which studies extremism and the internet.

“Misinformation will go viral more easily … and the algorithm assists in the building of that hive mind,” Richards said. “There’s money to be made through building a following.”

Unlike on YouTube or Facebook, where bogus claims and conspiracy theories are displayed as long videos or text reports, TikTok can deliver them via personalized videos that don’t require people manually resharing them to build an audience, Richards said. The app’s vertical-video monologues, she said, often feel “like you’re FaceTiming with someone.”

“It’s a very intimate way to consume someone’s beliefs, through this face-to-face interaction on screen,” she said. “It’s easy for someone to get on TikTok and lie or intentionally mislead and for it to feel very believable. It feels like they’re speaking directly to you.”

Lies on TikTok can spread to millions before they’re taken down or corrected, Richards said, and people attempting to question the dominant narrative often end up belittled or ignored. TikTok’s mob mentality ends up reinforcing narratives as they gain more views, as some creators saw during the high-profile defamation trial between actors Johnny Depp and Amber Heard, when their attempts to defend Heard or fact-check claims in the case were met with relentless harassment and attack.

TikTok’s fast access to giant audiences offers obvious financial incentives. Spreaders of conspiracy-theory videos about the coronavirus pandemic and election fraud have turned to the app, in hopes they can build a major following without getting taken down. Fake images or false claims that have been banned on Facebook have surfaced in the background of their videos with help from TikTok’s “green screen” feature.

“People will get on TikTok and say anything that will get them 1 million views, even if it’s a boldfaced lie,” Brooks said. “The algorithm does not check if you’re lying. It checks to see if you curse or if you’re nude. … People lie and make a bunch of money and get away with it all the time.”

But it also raises fears that the platform could be manipulated for political gain. In the run-up to Kenya’s presidential election in August, researchers with the Mozilla Foundation found that the app had enabled “the rapid spread of disinformation and incendiary rhetoric,” including lies about candidates and calls for ethnic violence. “TikTok is failing its first real test,” the researcher Odanga Madung said. TikTok officials said the company is working to improve its election-integrity rules.

Ultimately, the pressure to compete against everyone else for attention could end up driving some TikTok creators away. Faylee, the 21-year-old in London with 9 million followers, said she has started thinking more lately about life after TikTok. After graduating this year from college, she’s now preparing to apply for jobs as a medical researcher and is excited to try something new.

She worries about shifting away from what some people see as a dream job, but she knows how much work it takes to make something that seems effortless. She also remembers the closure, in 2016, of the short-video app Vine — TikTok’s predecessor, with its own crop of instant stars — and how quickly it all went away.

“I grew up when Vine was huge. All my friends were on Vine. And one day it just disappeared,” she said. “That’s always in the back of my mind. One minute you can have it all, and the next you can have nothing.”