Twitter’s paid verification risks revealed by Pentagon attack hoax

Paid blue checks and generative AI make misinformation spread easily, surprising no one. A fake Pentagon explosion image went viral on Monday morning.

The image surfaced on a verified Twitter account called “Bloomberg Feed” within 30 minutes, which looked like a Bloomberg account due to its blue check. Suspending that account According to screenshots taken before the tweet was deleted, the Russian state-controlled news network RT also tweeted the photograph. DeItaone, OSINTdefender, and Whale Chart shared it. Indian television reported the bogus Pentagon blast. This phony photograph and news story’s origin are unclear.

Prime example of the dangers in the pay-to-verify system: This account, which tweeted a (very likely AI-generated) photo of a (fake) story about an explosion at the Pentagon, looks at first glance like a legit Bloomberg news feed. pic.twitter.com/SThErCln0p

— Andy Campbell (@AndyBCampbell) May 22, 2023

This isn’t the first time a phony image has fooled the internet, but an explosion at a U.S. federal facility has higher stakes. Some have linked the bogus image to a 25 basis point S&P 500 dip, but the dip was brief, and there’s no way to prove it was caused by the hoax. After Reddit did it, generative AI could be used to game the stock market in the future.

The rise of generative AI and Twitter’s verification system make misinformation extremely prolific. Elon Musk’s plan to devalue blue checks and allow anyone to buy them has been a problem from the start.

Even if blue checks no longer signal validity, it’s hard to break a 15-year perceptual habit: You’re likely to believe “Bloomberg Feed”‘s Pentagon attack post with a blue check. False news broadcasts like this will persist as bogus photographs become harder to recognize.