Pentagon explosion tweet was a hoax. Still, it went viral.

All in all, the hoax — the latest in a string of AI-generated images to fool some social media users — appears to have done little immediate damage. Twitter has since suspended the Bloomberg Feed account, which was not related to the real Bloomberg media organization, and within about 20 minutes, local authorities had debunked the report.

“Just looking at the image itself, that’s not the Pentagon,” said Nate Hiner, a captain with the fire department in Arlington, Va., where the Pentagon is located. “I have no idea what that building is. There’s no building that looks like that in Arlington.”

Yet the mechanisms involved, from the image’s amplification by large propaganda accounts to the almost instantaneous response from the stock market, suggest the potential for more such mischief if AI tools continue to make inroads in fields such as social media moderation, news writing and stock trading.

And Twitter is looking like an increasingly likely vector, as new owner Elon Musk has gutted its human workforce, laid off a team that used to fact-check viral trends, and changed account verification from a manual authentication process to one that’s largely automated and pay-for-play. The signature blue badges once indicated authority for public figures, large organizations, celebrities and others at risk of impersonation. Now, Twitter awards them to any one willing to pay $8 a month and confirm their phone number.

Twitter did not respond to a request for comment.

With experts predicting that AI will impact millions of human jobs, the concern becomes not just whether AI-generated misinformation might mislead people, but whether it might mislead its fellow automated systems.

“This isn’t an AI issue, per se,” said Renée DiResta, research manager at Stanford Internet Observatory and an expert on how misinformation circulates. “Anyone with Photoshop experience could have made that image — ironically, could probably have done it better. But it’s a look at how signals that help people decide whether information about breaking news is trustworthy on Twitter have been rendered useless, just as the capacity to create high-resolution unreality has been made available to everyone.”

Verified accounts spread the news

One of the first accounts to post about the fake event was a verified account called OSINTdefender, which tweeted the report of the explosion, along with the bogus image, to its 336,000 followers at 10:04 a.m. The building in the photograph looks little like the Pentagon, but it bears some the hallmarks of being AI-generated. The tweet has been deleted.

Reached via Twitter, the owner of the OSINTDefender account said they had first heard the report on the social platform Discord a few minutes earlier from a user who goes by the handle “Walter Bloomberg.” They said the image came from the Facebook page, since deleted, of a person who claimed to work in Arlington.

In the next few minutes, other large accounts posted similarly worded false reports on Twitter, including Walter Bloomberg, who goes by the same handle on both platforms. His 10:06 a.m. tweet garnered at least 730,000 views.

Many of the accounts that reshared it pose as aggregators of financial news, including a 386,000-follower account with the handle @financialjuice, named “Breaking Market News,” and another account called “Bloomberg Feed” that is unrelated to the real Bloomberg.

Some of the accounts had blue “verified” check marks, while legitimate organizations that shared the truth did not. The official account for the Pentagon Force Protection Agency, which polices the Pentagon, doesn’t pay for a blue check mark, and Twitter has not given it a gray check mark indicating it’s a verified institution. The agency retweeted a local law-enforcement message saying there was “NO explosion” at 10:27 a.m.; the tweet had only 78,000 views as of 4 p.m.

Local authorities scramble

Hiner, the Arlington Fire captain who handles the Northern Virginia department’s emergency communications, said it took about five minutes for him to realize the reports on Twitter were fake.

At 10:10 a.m., Hiner was in a meeting when he got the first call. He stepped out of the meeting to investigate.

The first sign something was off? He had not received any alerts from the department’s emergency software, First Due, which monitors dispatch and sends him a push notification when first responders are sent out for major incidents like fires.

Next, he checked his mobile data terminal — essentially a laptop that lists every active 911 incident in Arlington — and found no sign of anything going on near the Pentagon.

“There were no medical calls, no fire calls, no incidents whatsoever,” he said.

That’s when he finally pulled up social media himself, expecting to see some eyewitness accounts on Twitter. But again, there was nothing. All he saw was the doctored photo of the explosion.

Five ways to spot false AI images — and why you shouldn’t freak out about them.

At that point, he reached out to spokesmen at the Defense Department and at the Pentagon Force Protection Agency. By 10:27 a.m., he’d posted on Arlington Fire’s Twitter account that the reports were false.

“There is NO explosion or incident taking place at or near the Pentagon reservation,” the tweet said, “and there is no immediate danger or hazards to the public.”

Hiner said that he sometimes receives odd inquiries from Arlington residents after seeing a firetruck in their neighborhood or gets misguided calls based on scanner traffic. But he cannot recall another time, he said, “in which an emergency incident was being reported on social media that was just 100 percent inaccurate.”

New twist on an old problem

From Photoshopped images of a shark on a highway during Hurricane Sandy to false reports of celebrity deaths, viral lies are nothing new on Twitter. Generative AI tools, from chatbots such as ChatGPT that can pen fake news stories to AI art tools such as Midjourney and Stable Diffusion, are only the newest tools in the hoaxsters’ kit. They’ve been used in recent months to create other viral images, including one that appeared to show Donald Trump getting arrested and another depicting Pope Francis making a fashion statement.

For the most part, mainstream media outlets have successfully refuted the misinformation, and the world has marched on as before. Still, some hoaxes have wrought chaos, to varying degrees. In 2013, a fake tweet about an attack on the White House touched off a quick drop in financial markets.

Over time, social media users and the news media have learned to turn a skeptical eye on viral reports, especially from unverified sources. But Twitter’s new verification system means that the blue check mark, once a visual shortcut that conveyed a modicum of authority on an account, no longer serves that function.

Sam Gregory, the executive director of the human rights organization Witness, whose group has studied fake images and disinformation, said the Pentagon explosion image tweeted Monday carries multiple hallmarks of a fake, including visual glitches and an inaccurate view of the Pentagon. The challenge with such fakes, Gregory said, is the speed with which they can blast across the internet.

“These circulate rapidly, and the ability to do that fact-check or debunk or verification at an institutional level moves slower and doesn’t reach the same people,” he said.

Though the image may be obviously fake to some, the fact that it was attached to an authoritative-sounding claim made it that much more likely to gain attention, Gregory added.

“The way people are exposed to these shallow fakes, it doesn’t require something to look exactly like something else for it to get attention,” he said. “People will readily take and share things that don’t look exactly right but feel right.”

As for why the fakes were shared, it’s unclear. Some fakes have been shared to score political points, while others have been used to troll or build an audience that the account may hope to monetize.

“Sometimes they’re doing it maliciously, or sometimes they’re just doing it to get a lot of views,” he said. “You can get a lot of audience very quickly from this, and that is a powerful drug.”

Arlington County Board Chair Christian Dorsey (D) said local governments like Arlington’s face an increasingly steep challenge in responding to misinformation as AI makes it easier to rapidly generate plausible fakes. He said officials try to guide residents to follow local authorities on Twitter and turn to them for reliable information rather than “some random Twitter handle.” Arlington County, and its police and fire/EMS departments are all verified with a “silver check” on Twitter, indicating that they’re government-run accounts.

But he recognizes that may not be enough.

“Our number of followers pales in comparison to some of the most popular social media accounts out there. You always run the risk that they’re not going to penetrate as deeply,” Dorsey said. But “absent any magic bullet, where these platforms ensure only the best truthful information is relayed, I think it’s the best we can do.”

Jeremy Merrill and Faiz Siddiqui contributed to this report.