Discover how Indian security agencies are leveraging open-source intelligence and artificial intelligence to gather crucial information from public sources and social media, all while ensuring privacy is protected.
The home ministry has told a parliamentary panel that security agencies use open-source intelligence from public sources, including social media, to gather information, emphasising that no privacy breach occurs as personal data isn't collected.
The ministry's submission was made to the Standing Committee on Communications and Information Technology (2024-25), chaired by Lok Sabha member Nishikant Dubey, which tabled its report on Monday.
The committee had sought to know from the ministry how it dealt with the issue of privacy while scraping through the internet and through social media.
"Publicly available information on the internet and social media platforms is used for intelligence gathering. No private or personal information is gathered from social media. Hence, privacy is never violated," the ministry told the panel.
Scraping generally refers to the use of computer programmes, tools or software (called web scrapers) to automatically browse public web pages or social media posts; extract specific information (e.g., names, phone numbers, keywords, hashtags, trends, images); store or analyse that information for law enforcement or intelligence purposes, it said.
The ministry said the type of data that can be scraped by the authorised security agencies only from publicly available sources using open-source intelligence technique may include social media posts like public tweets, Facebook posts, YouTube videos, deepfakes or morphed media, fake news or misinformation, including viral instigating content spreading communal hatred.
The scraping may be used to collect hashtags and trends, for example, from YouTube channels and Telegram groups, for radical content and posts that promote extremist ideologies or share propaganda videos or bomb-making tutorials.
It said the technique may be used to monitor scam websites or links to track, for example, online gambling, fake job scams, and fake investment scams.
It may be used on matrimonial and dating Platforms for honeytraps or fraud, and from dark web marketplaces for the extraction of cryptocurrency wallet addresses, it said, giving examples of the process.
"Public profiles on matrimonial/dating sites may be scraped in cybercrime investigations where people are blackmailed or trapped into sharing sensitive data," it said while giving examples.
The ministry told the panel that AI is being used for intelligence gathering and counter-intelligence, with key application including face recognition, social media parsing and network analysis, natural language processing for analysis of structured and unstructured data, pattern analysis and hidden connections, etc.
"Additionally, AI is used for entity resolution, enabling accurate identification and correlation of individuals across multiple data sources," it said.
Giving details about the use of artificial intelligence, the home ministry said it is helping security agencies in enhancing their capabilities for intelligence gathering and counter-terrorism efforts by rapidly analysing vast datasets, detecting anomalies, predicting patterns, cross-linkages, etc., thereby improving decision-making, speed and accuracy.
The ministry said the CRPF is using AI in identifying narratives/sentiment analysis on open source (social media platforms).
"An AI-driven intelligence fusion centre is in the final stage of deployment. It will ingest a huge amount of structured and unstructured data and provide analysis, creation of a decision support system which can help in smart explorative interpretations, and solutions for operational requirements of CRPF," it said.
Underlining the potential of AI in intelligence gathering, the ministry said AI can automate the processing of vast and diverse data sources, including communication records, open-source intelligence, financial transactions, and surveillance feeds.
"This allows for quicker identification of threats, detection of anomalies, and linkage analysis across different data points. Natural language processing tools can be used for multilingual monitoring, including regional dialects, aiding in decoding sensitive content from open and dark web sources," it said.