BENGALURU: Amid growing concerns over the misuse of artificial intelligence to generate explicit and morphed images using real-life photographs, the Bengaluru City Police have cautioned citizens against the rising threat of AI-enabled cybercrimes.
The warning comes against the backdrop of the Ministry of Electronics and Information Technology (MeitY) reportedly writing to AI platform Grok, directing it to conduct a review and seek clarification regarding the misuse of its tools to generate obscene and explicit content.
In an advisory issued on Saturday, the Bengaluru City Police said AI-generated explicit material is increasingly being weaponised to harass, blackmail and extort individuals, particularly women.
Officials from the city’s cybercrime wing said a majority of such cases originate from images sourced from social media platforms. “Publicly available photographs are often misused to create morphed or explicit content using AI tools. These images are then circulated or used to threaten victims,” an official said.
The police urged users to keep their social media accounts private wherever possible and exercise caution while accepting friend requests or engaging with unknown profiles. Citizens have also been advised to verify online identities and refrain from sharing personal photographs or sensitive information with strangers.
“Even casual interactions on social media can expose people to serious cyber risks,” the official added.
City Police Commissioner Seemant Kumar Singh told TNIE that while complaints specifically related to AI-generated explicit content are still limited, the broader category of AI-enabled and internet-based crimes is steadily increasing.
“There are currently no dedicated legal provisions that deal exclusively with this type of offence, which is why regulation in this space is becoming increasingly important,” he said. Singh said Bengaluru police stations are equipped with advanced technological tools to trace IP addresses, identify offenders and initiate content takedowns. However, he stressed that timely reporting plays a crucial role in effective action. “In cyber crimes, delays can make evidence harder to trace. Immediate reporting helps us act swiftly to block content and identify its source,” he said.