In early 2026, the Grok AI chatbot faced intense scrutiny after users discovered it could generate non-consensual sexualized images — including deepfakes of real people, sometimes involving minors. The controversy escalated to the point where Apple reportedly issued an ultimatum to X (formerly Twitter): fix the problem or risk having the Grok app removed from the App Store.

According to a letter obtained by NBC News and detailed by 9to5Mac, Apple contacted X after receiving user complaints and reviewing media coverage. While X made some improvements, Apple determined that the Grok app was still not fully compliant with its strict guidelines on harmful content, particularly non-consensual explicit imagery and child sexual abuse material (CSAM).

This incident highlights the growing tension between AI companies pushing the boundaries of generative tools and app store gatekeepers enforcing strict content policies.

What Happened: The Deepfake Controversy Timeline

The issue exploded at the beginning of 2026 when a trend on X encouraged users to ask Grok to edit photos of real people — often celebrities or ordinary individuals — into revealing clothing such as bikinis or lingerie without their consent.

Some generated images reportedly depicted minors, which is a serious criminal offense in the United States and a direct violation of both Apple and Google’s App Store policies.

Key Developments:

• X initially restricted image generation to paid subscribers only.
• Elon Musk publicly stated that anyone using Grok to create illegal content would face the same consequences as uploading illegal material directly.
• Reports emerged that Musk resisted internal efforts by xAI staff to implement stronger guardrails, reportedly calling them “over-censorship.”
• Governments reacted strongly: Indonesia banned Grok, the UK threatened similar action, and California Attorney General Rob Bonta launched an investigation.
• X eventually banned all users from generating sexualized images via Grok and added a button allowing people to block Grok from editing public photos.

Despite these measures, NBC News reported that some users could still generate images of women in revealing clothing (such as towels or sports bras), prompting continued criticism.

Apple’s Response: The Ultimatum

In its letter to US senators, Apple explained that after reviewing complaints and news reports, it reached out to X and demanded improvements in content moderation for the Grok app.

While X “substantially resolved its violations,” Apple stated that the Grok app “remained out of compliance.” As a result, Apple rejected a Grok app submission and warned that further changes were required, or the app could be removed from the App Store.

A subsequent update was eventually approved, but the incident revealed how seriously Apple takes violations involving non-consensual explicit content and potential CSAM.

Why This Matters: App Store Policies and AI Responsibility

Apple has some of the strictest App Store review guidelines in the industry, particularly around:

• Child sexual abuse material (CSAM)
• Non-consensual intimate imagery (deepfakes)
• Harmful or deceptive content
• User safety and moderation

Google’s Play Store has similar policies, though enforcement can sometimes differ. Both companies face increasing pressure from lawmakers, advocacy groups, and the public to prevent AI tools from being used to create harmful content.

The Grok controversy is part of a broader debate about how much responsibility AI companies and platform owners should bear for content generated by their tools — especially when those tools can produce realistic images of real people without consent.

X and xAI’s Defense

X pushed back against the criticism, stating:

“We strictly prohibit users from generating non-consensual explicit deepfakes and from using our tools to undress real people. xAI has extensive safeguards in place to prevent such misuse, including continuous monitoring of public usage, analysis of evasion attempts in real time, frequent model updates, prompt filters, and additional safeguards.”

The company also emphasized that it took legal action against accounts involved in generating illegal content.

However, critics argue that initial resistance to adding guardrails and the ease with which some users could bypass restrictions showed that more robust protections were needed from the start.

The Bigger Picture: AI Deepfakes and Platform Accountability

This incident is not isolated. In 2025 and 2026, deepfake technology — particularly non-consensual intimate imagery — has become a major societal concern. High-profile cases involving celebrities, politicians, and ordinary people have prompted calls for stronger regulation.

Lawmakers in the US and Europe are actively debating bills that would criminalize non-consensual deepfakes and hold platforms more accountable for hosting or enabling such content.

For AI companies like xAI, the challenge is balancing innovation and “maximum truth-seeking” with the need to prevent real-world harm. Elon Musk has consistently positioned Grok as less censored than competitors like ChatGPT, but this philosophy has clashed with app store rules and legal requirements.

What Changed After Apple’s Warning?

According to reports:

• X implemented additional prompt filters and monitoring systems.
• A button was added allowing users to prevent Grok from editing their public photos.
• Image generation capabilities were further restricted for certain types of requests.
• An updated version of the Grok app was eventually approved by Apple.

Whether these changes are sufficient long-term remains to be seen, especially as generative AI technology continues to advance rapidly.

Implications for Users and Developers

For users:

• Be cautious when sharing personal photos online, as they can potentially be used in deepfake generation.
• Understand that even “fun” AI image tools can be misused.

For developers and AI companies:

• Robust safety measures and content moderation are no longer optional — they are essential for staying on major app stores.
• Transparency about safeguards and quick response to misuse are critical.

Final Thoughts

Apple’s reported threat to remove the Grok app underscores the serious responsibilities that come with deploying powerful generative AI tools. While X and xAI have taken steps to address the issue, the incident serves as a reminder that innovation in AI must be paired with strong ethical guardrails and user protections.

As generative AI becomes more accessible, the line between creative freedom and potential harm will continue to be tested. Platforms, regulators, and companies will need to work together to ensure these powerful tools are used responsibly.

The Grok deepfake controversy is far from the last time we’ll see this kind of clash between free expression, innovation, and safety in the AI era.

Frequently Asked Questions (FAQ)

1. What is the Grok AI controversy about?
The controversy involves the Grok AI chatbot generating non-consensual sexualized images, including deepfakes of real people, which raised serious ethical and legal concerns.

2. Why did Apple warn about removing the Grok app?
Apple warned X that the Grok app violated App Store policies, particularly those related to harmful content like non-consensual explicit imagery and potential CSAM.

3. What actions did X take to fix the issue?
X restricted image generation features, added stronger content filters, banned sexualized image creation, and introduced tools allowing users to block Grok from editing their photos.

4. Did Grok AI generate illegal content?
Reports indicated that some outputs included inappropriate or illegal deepfakes, including images involving minors, which are strictly prohibited under global laws and platform policies.

5. What are Apple’s rules regarding AI-generated content?
Apple enforces strict guidelines banning non-consensual explicit content, child sexual abuse material (CSAM), and harmful or misleading AI-generated media on its App Store.

6. Is Grok AI still available on the App Store?
Yes, but only after updates were made. Apple approved a revised version of the app after X implemented additional safeguards.

7. Why are deepfakes considered dangerous?
Deepfakes can be used to create misleading or harmful content, damage reputations, spread misinformation, and violate privacy—especially when involving non-consensual or explicit imagery.

8. How does this affect the future of AI regulation?
The Grok incident highlights the growing need for stricter AI regulations, stronger content moderation, and accountability for companies developing generative AI tools.

9. What should users be aware of when using AI image tools?
Users should avoid generating harmful or illegal content and be cautious about sharing personal images online, as they could potentially be misused in deepfake generation.

10. How does Grok compare to other AI chatbots in safety?
Compared to tools like ChatGPT or Claude, Grok initially had fewer restrictions, which contributed to the controversy and increased scrutiny.