Microblogging platform X has acknowledged lapses in content moderation linked to its AI chatbot Grok and removed around 3,500 pieces of content and more than 600 user accounts after the government flagged the circulation of obscene material, according to official sources.
The company has assured authorities that it will comply with Indian laws and strengthen safeguards to prevent a recurrence. The steps follow directions issued earlier this month by the ministry of electronics and information technology (MeitY), which warned of action under the law if unlawful content generated through Grok was not taken down promptly.
Government sources said X accepted that violations had occurred on its platform and acted to block content and delete accounts found to be hosting or amplifying obscene imagery. The company has also conveyed that it will not allow such material going forward and will enforce its user policies more strictly.
Failure of Safeguards
The issue has put the spotlight on Grok, developed by xAI and integrated into X, amid global scrutiny of generative AI systems over content moderation, data safety and the misuse of tools to create non-consensual sexually explicit material. X has faced criticism in multiple jurisdictions over user prompts that enable digital undressing and synthetic image manipulation.
Earlier this month, MeitY directed the US-based platform to immediately remove vulgar, obscene and unlawful content generated through Grok and sought a detailed action-taken report within 72 hours. The ministry asked X to spell out technical and organisational measures adopted, the role of its chief compliance officer, action taken against offending users and accounts, and mechanisms to ensure compliance with mandatory reporting requirements under Indian law.
In its directive, the government said Grok was being misused to create or operate accounts that host or generate obscene images or videos of women in a derogatory manner. It noted that the misuse extended beyond fake accounts to targeting women who post their own images, through prompts, image manipulation and synthetic outputs, and described this as a failure of platform-level safeguards.
Legal Accountability
The government underlined that compliance with the Information Technology Act and related rules is mandatory, and that the safe harbour protection under Section 79 is conditional on strict due diligence. It warned that failure to meet these obligations could lead to loss of immunity and action under the IT Act and the Bharatiya Nyaya Sanhita.
X’s initial response to the notice outlined its general content takedown policies, including action against illegal and non-consensual sexualised images, but was seen by the ministry as lacking specific details on enforcement related to Grok. Officials said the reply missed key information on takedowns and preventive steps, prompting the government to seek further clarification.
Following this, X has conveyed acceptance of shortcomings and assured full compliance with Indian regulations, sources said, adding that enforcement measures against violators would be strengthened.