Grok Restricts Generated Images After Outcry Over Sexualized Deepfakes

The company behind Grok, the AI service that is part of the social media platform X, reportedly turned off its ability to generate images for some users on Friday. According to The Hollywood Reporter and several other reports, Grok returned the message “Image generation and editing are currently limited to paying subscribers” to some users. According to NBC News, Grok’s standalone app is still allowing users to generate images.

The move comes after a spate of users have asked Grok to take existing pictures on the platform and sexualize them, putting the people in the images in underwear or bikinis. Because of this, the U.K. has threatened to fine or ban X, according to THR. The country’s prime minister, Keir Starmer, has decried the platform for its ability to create pornographic images of women and children, calling it “disgraceful” and “disgusting.” Starmer has sought to outlaw “creating sexually explicit deepfake images,” according to the publication. Ofcom, which regulates communications in the country, has the right to fine companies up to 10 percent of their global revenue, per the Online Safety Act.

A rep for xAI, which operates Grok, responded to Rolling Stone, but did not offer meaningful comment.

“These new laws give Ofcom the power to start making a difference in creating a safer life online for children and adults in the U.K.,” Dame Melanie Dawes, Ofcom Chief Executive, said in 2023 when the law was announced. “We’ve already trained and hired expert teams with experience across the online sector, and today we’re setting out a clear timeline for holding tech firms to account.”

Trending Stories

Messy Breakups and That ICE Song: 5 Takeaways From Zach Bryan's New Album
Robby Krieger and John Densmore Play ‘Riders on the Storm’ With Artists From Around the Globe
Marilyn Manson Accuser Seeks Revival of Claims Based on New California Law
Zach Bryan’s New Album, ‘With Heaven on Top,’ Is Here

On New Year’s Eve, the content analysis firm Copyleaks reported that Grok was generating “roughly one nonconsensual sexualized image per minute,” where users could post them to X, ready to become viral. Elon Musk, who owns majority stake in xAI, which owns X and created Grok, placed blame for the controversial images on users, not the platform. “Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content,” he posted on Saturday. Musk has not yet commented on the reports that some users cannot generate images.

1 p.m. ET: This story has been updated to include that xAI responded to a request for comment.