Could TikTok hoaxes tip upcoming South Asian elections?

Divyendra Singh Jadoun was busy making artificial intelligence-based visual effects and voice clones for film and television in India, when he began getting calls from politicians: could he create AI videos, or deepfakes, for their election campaign?

With a hotly-contested local election in his home state of Rajasthan last November, and a national election due by May this year, the opportunity for his company, The Indian Deepfaker, is tremendous. But Mr. Jadoun was reluctant.

“The technology to create deepfakes is so good now, it can be done almost instantaneously, with very little effort – and people cannot tell if it’s real or fake,” he said. “There are no guidelines on deepfakes, and that’s worrying, as it has the potential to influence how a person votes.”

Instagram reels of Indian Prime Minister Narendra Modi singing in regional languages have gone viral recently, as have TikTok videos of Indonesian presidential candidates Prabowo Subianto and Anies Baswedan speaking in fluent Arabic.

But they were all created with AI, and posted with no label.

With elections due in India, Indonesia, and Pakistan in the coming weeks, misinformation is rife on social media platforms, and deepfakes – video or audio made using AI and broadcast as authentic – are particularly concerning, tech experts and authorities say.

In India, where more than 900 million people are eligible to vote, Mr. Modi has said deepfake videos are a “big concern,” and authorities have warned social media platforms they could lose the safe-harbor status that protects them from liability for third-party content posted on their sites if they don’t act.

In Indonesia – where more than 200 million voters will go to the polls on Feb. 14 – deepfakes of all three presidential candidates and their running mates are circulating online and have the potential to influence election outcomes, said Nuurrianti Jalli, who studies misinformation on social media.

“From microtargeting of voters with disinformation to spreading false narratives at a scale and speed unachievable by human actors alone, these AI tools can significantly influence voter perceptions and behavior,” she said.

“In environments where misinformation is already prevalent, AI-generated content can further skew public perception and influence voting behavior,” added Ms. Jalli, an assistant professor at Oklahoma State University’s media school.

Political propaganda

Deepfake images and videos churned out by generative AI tools such as Midjourney, Stable Diffusion, and OpenAI’s Dall-E popped up ahead of elections from New Zealand to Turkey and Argentina last year, with growing concerns about their impact on the U.S. presidential election in November.

AI makes the creation and spread of disinformation faster, cheaper, and more effective, the U.S. nonprofit Freedom House said in a recent report.

In Bangladesh, where Prime Minister Sheikh Hasina won her fourth straight term on Jan. 7, deepfake videos of female opposition politicians Rumin Farhana in a bikini and Nipun Roy in a swimming pool emerged during the campaign.

While they were debunked quickly, they continued circulating, and even poor-quality deepfake content is misleading people, said Sayeed Al-Zaman, an assistant professor of journalism at Bangladesh’s Jahangirnagar University, who studies social media.

“Given the low levels of information and digital literacy in Bangladesh, deepfakes can be potent carriers of political propaganda if crafted and deployed effectively,” he said. “But the government does not appear concerned.”

The ministry of information did not respond to a request for comment.

In Pakistan, where an election is scheduled for Feb. 8, Imran Khan, who is in prison on an official secrets acts case after being ousted as prime minister last year, used a AI-generated image and voice clone to address an online election rally in December, which drew more than 1.4 million views on YouTube and was attended live by tens of thousands.

While Pakistan has drafted an AI law, digital rights activists have criticized the lack of guardrails against disinformation, and to protect vulnerable communities including women.

“The threat that disinformation poses to elections and the overall democratic process in Pakistan cannot be stressed upon enough,” said Nighat Dad, co-founder of the non-profit Digital Rights Foundation.

“In the past, disinformation on online platforms has managed to sway voting behavior, party support, and even influenced legislation change. Synthetic media will make this easier to do,” she added.

Dangerous sign

At least 500,000 video and voice deepfakes were shared on social media sites globally in 2023, estimated DeepMedia, a company developing tools to detect synthetic media.

Platforms have struggled to keep up.

Meta, which owns Facebook, Instagram, and WhatsApp, said it aims to remove synthetic media when the “manipulation is not apparent and could mislead, particularly in the case of video content.”

Google, which owns YouTube, said in November that the video-sharing platform requires “creators to disclose altered or synthetic content that is realistic, including using AI tools, and we’ll inform viewers about such content through labels.”

But countries including India, Indonesia, and Bangladesh have recently passed laws to more closely police online content and penalize social media sites for content deemed misinformation, so platforms are “holding their punches,” said Raman Jit Singh Chima, Asia policy director at advocacy group Access Now.

In these countries, “this election cycle is actually worse than the last cycle – platforms are not set up to handle problems, and they are not being responsive and proactive enough. And that’s a very dangerous sign,” he said.

“There is a danger that the world’s attention is only on the U.S. election, but the standards being applied there, the effort being made there should be duplicated everywhere,” he added.

In India, where Modi is widely forecast to win a third term, Mr. Jadoun – who had declined to make deepfake campaign videos for the state elections – is gearing up to make them for the general election.

These will be personalized video messages from politicians for party workers, not voters, that can be sent on WhatsApp.

“They can really have an impact, because there are hundreds of thousands of party workers and they will, in turn, forward them to their friends and family,” he said.

“But we will add a watermark to show that it is made with AI, so there is no misunderstanding. That’s important.”

This story was reported by the Thomson Reuters Foundation.