Microblogging platform X has implemented new technological measures to prevent its AI chatbot Grok from generating images of real people in revealing clothing in jurisdictions where such content is illegal. The move comes after the platform faced heavy backlash over the circulation of obscene deepfakes generated by Grok.
X said the restriction applies to all users, including paid subscribers, according to a post shared on its official Safety handle.
Image generation on Grok limited to paid subscribers
X further announced that image creation and image-editing features via the Grok account on X are now available only to paid subscribers. The platform said this additional restriction adds a layer of protection by helping ensure that individuals who attempt to abuse Grok to violate laws or platform policies can be held accountable.
“We now geoblock the ability of all users to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it’s illegal,” X said.
The company also confirmed that it has implemented technological safeguards to prevent the editing of images of real people in revealing clothing, such as bikinis. X reiterated that this restriction applies to all users, including paid subscribers.
Safety protocols and content moderation measures
X asserted that these changes do not alter its existing safety protocols, under which all AI prompts and AI-generated content posted on the platform must strictly adhere to X’s rules.
“However content is created or whether users are free or paid subscribers, our Safety team are working around the clock to add additional safeguards, take swift and decisive action to remove violating and illegal content, permanently suspend accounts where appropriate, and collaborate with local governments and law enforcement as necessary,” the company said.
X reiterated its zero-tolerance policy towards child sexual exploitation, non-consensual nudity, and unwanted sexual content.
“We take action to remove high-priority violative content, including Child Sexual Abuse Material (CSAM) and non-consensual nudity, taking appropriate action against accounts that violate our X rules,” the post said.
The platform added that it also reports accounts seeking child sexual exploitation material to law enforcement authorities when required.
IT Ministry intervention and compliance with Indian laws
X recently admitted its mistake and removed around 3,500 pieces of content and deleted over 600 accounts after India’s IT Ministry raised concerns over obscene content linked to Elon Musk-backed AI chatbot Grok. X had assured authorities that it would comply with Indian laws.
There has been mounting pressure on Grok from governments worldwide, including India, as regulators intensify scrutiny of the generative AI engine over content moderation, data safety, and non-consensual sexually explicit images that have flooded X in recent days.
On January 2, the IT Ministry directed X to immediately remove all vulgar, obscene, and unlawful content generated by Grok or face action under the law.
Government demands detailed action taken report
In its directive, the ministry asked X to submit a detailed Action Taken Report (ATR) outlining:
- Specific technical and organisational measures adopted or proposed for Grok
- The role and oversight of the Chief Compliance Officer
- Actions taken against offending content, users, and accounts
- Mechanisms to ensure compliance with mandatory reporting requirements under Indian laws
The ministry noted that Grok AI, developed by X and integrated into the platform, was being misused to create fake accounts hosting and sharing obscene images or videos of women in a derogatory manner.
“Importantly, this is not limited to creation of fake accounts but also targets women who host or publish their images or videos, through prompts, image manipulation and synthetic outputs,” the ministry said.
It asserted that such conduct reflects a serious failure of platform-level safeguards and enforcement mechanisms, amounting to gross misuse of AI technologies in violation of the law.
Warning on loss of safe harbour protection
The government made it clear that compliance with the IT Act and its rules is mandatory, and that exemptions under Section 79 of the IT Act, which provides safe harbour to online intermediaries, are conditional on strict due diligence.
X was warned that failure to observe these obligations would result in the loss of immunity from liability, along with possible action under other laws, including the IT Act and the Bharatiya Nyaya Sanhita.
The platform was also instructed to enforce user terms of service and AI usage restrictions, including deterrent actions such as suspension and termination of violating accounts. X was asked to remove or disable access to all unlawful content “without delay,” in line with timelines prescribed under the IT Rules, 2021, while preserving evidence.
Initial response deemed inadequate
Although X’s initial response to the notice was detailed, the IT Ministry viewed it as insufficient, stating that it lacked key information such as takedown specifics and concrete actions taken to prevent future misuse of Grok.
The response fell short of expectations, prompting the government to toughen its stance and demand additional details. X later accepted its mistake and informed the government that it would comply with Indian laws and regulations, sources said.
Global pressure mounts on X over Grok deepfake issue
Apart from India, the UK and the European Union have also mounted pressure on X over the Grok deepfake image issue, as global regulators increase scrutiny of generative AI platforms.
ALSO READ: Is Apple losing its status as the world’s tech visionary?
