Chinese tech firms’ ‘digital humans’ who look and sound like you: What are their uses and pitfalls?

SHENZHEN: Singaporean educator and entrepreneur Andrew Chin can’t wait to meet his new “digital human” – a virtual avatar of himself designed and created by Chinese internet giant Tencent.
His digital doppelganger will look and sound very much like him and will be programmed to speak an impressive four languages: English, Chinese, Thai and Bahasa Indonesia.
He plans to use the virtual avatar for public speeches and to have it also star in recorded video tutorials in place of himself, for his teaching and business cohorts at Singapore Management University as well as the Singapore University of Technology and Design.
“I’m not a professional. I can’t record or edit (myself) properly,” Mr Chin told CNA earlier this month at the Global Digital Ecosystem Summit held in Shenzhen. But using AI technology, recorded text and speech will then be transformed into engaging videos.
To most people, the concept of a digital human might sound strange. But in China, where artificial intelligence (AI) research and application has been advancing rapidly, the use of digital people has been widespread and widely accepted in sectors like gaming, animation and the entertainment industry. 
Support and endorsements by central and regional governments have been vital for the industry, experts say. More than 30 Chinese cities and provinces including Beijing, Shanghai and Guangdong, have since introduced policies supporting the digital human industry. 
This official endorsement by authorities also goes a long way and adds an air of credibility, they add. Chinese state media outlets like the CGTN news network have introduced digital reporters and news anchors to front live stream shows and weather forecasts as well as to report on major events.  
In China, there are around 1.14 million registered firms and companies creating digital people, according to latest estimates by Chinese business and data research platform Tianyancha and demand continues to soar, the firm said. 
Kuaishou, one of China’s leading short-video platforms which often features user-generated content, reported over 300,000 livestreams that were conducted by virtual hosts in 2023. 
Chinese e-commerce giant JD.com, which saw revenues of more than US$152 billion in 2023, unveiled an AI clone of its founder Liu Qiangdong during a livestream session in April that accurately replicated Mr Liu’s appearance, voice and accent, as well as his known habit of waving his hands while speaking. 
Mr Chin’s digital human will be the first two-dimensional digital human created by Tencent outside of mainland China. He runs an entrepreneurship education platform where around 50 Singapore businesses have expressed interest to follow his lead.
“I see (the technology) very much as a productivity tool,” he said, adding that numerous associates he knew who are running theme parks and eldercare care centres had expressed their interest because they want “more immersive experiences”.  
Checks by CNA on various Chinese platforms showed surprisingly affordable costs of creating digital human avatars, although the quality of technology has varied, ranging from hyper realistic to cartoon animations.
On Taobao, services were seen going for as low as 29 yuan (US$4) per month to slightly higher prices of 50 yuan (US$7) for unique characters.
Some providers even offered free trials. Customers must provide photos for sellers to animate, paying special attention to mouth features to match words.  
At the forefront of the industry is Chinese internet giant Tencent, which launched its own digital human production platform in 2023 – leveraging advanced AI technology like voice interaction, image generation and facial synchronisation, enabling users to create 2D and 3D avatars for live streams and more. 
At its annual tech conference held this month in Shenzhen, Mr Coby Chiu, the group’s vice-president and also Chief Operating Officer of Tencent’s Cloud and Smart Industries Group, told CNA the global market has outpaced performance in mainland China over the past two years. 
Packages for users in China range from between 6,000 yuan to 790,000 yuan, allowing users to create their own 2D and 3D avatars.  
When asked about prices for international users, Tencent representatives declined to reveal more details. 
Speaking to CNA, Mr Chin shared some details about costs he has incurred for his digital human. For his own digital human, he opted for a mid-tier technology package, costing between S$1,000 and S$2,000 which he said allowed for generating up to 10 hours’ worth of content. 
But it is worth it, he says, considering long term benefits. 
Earlier this month, I attended Tencent’s Global Digital Ecosystem Summit in Shenzhen, where I got to experience their Digital Human showcase at the Shenzhen World Exhibition & Convention Center.
Along with other international media — including journalists from Malaysia and Indonesia — we were introduced to a digital version of Mr Dowson Tong, CEO of Tencent Cloud. Big crowds gathered to watch a virtual version of Mr Tong appear on a large screen and deliver a presentation in three languages: Chinese, English, and then Bahasa Indonesia.
Then it was my turn (I was quite eager to try out the technology).
With my phone, I scanned a QR code which directed me to Tencent’s website where I submitted a recent photo, a 30-second voice recording, and 100 words of text paragraph for my digital human to say. 
I also had the freedom to choose from nine different languages for my digital human: Chinese, English, Korean, Japanese, Arabic, Bahasa Indonesia, Thai, French, or German. There was even an option to switch the voice to a different gender, which I found fascinating.
Unfortunately there were technical difficulties. Despite help from Tencent staff, I wasn’t able to generate my digital human on site so the process had to be completed remotely once I returned to Singapore.
I had help from the team at Tencent Cloud and Smart Industries Group. I provided a one-minute voice recording, along with a 30-second video. I was told that my face and mouth had to be clearly visible throughout. And for fun, I decided to recite tongue twisters —making sure my face moved in sync with my words.
Before the final step, I was required to provide consent so I recited: “I, Melody Chan, am aware that recordings of my voice will be used by Tencent Cloud to create and use a synthetic version of my voice.”
When the final product arrived, it was startlingly lifelike, almost a mirror of the video I had submitted. The voice, while slightly robotic, was impressive nonetheless. Hearing myself speak fluently in languages I don’t know, like Thai, Arabic, and French, was surreal.
However, there were a few small glitches. The sentence breaks felt a bit unnatural, and if you paid close attention, you could spot my hand gestures repeating themselves.
In the end, creating my own hyper-realistic digital doppelganger was undeniably fun, but also a little dystopian. It felt like something straight out of Black Mirror.
Seeing ‘Digital Melody’ come to life made me wonder… how long before AI comes for our jobs?
But as virtual beings grow in popularity and become more advanced and lifelike, experts have raised issues about ethics, the rise of deep fakes and the spread of misinformation. 
“There are many issues with (digitally) replicating humans,” said Jeannie Paterson, a professor from the Melbourne Social Equity Institute. “Will the use of AI infringe a person’s moral integrity? For people interacting with the digital human, there is the risk of being misled or manipulated or even (suffering) emotional harm.” 
Potential privacy risks, long associated with the sharing of biometric data, is also another area of concern, Ms Paterson added.
“Typically, users need to be told what will be done with their data and consent required for use of sensitive personal information but often, users are unaware of these protections and don’t read privacy policies,” she said. 
“Users should be cautious in sharing biometric information. This is why it’s important to have strong regulators.” 
SenseTime, one of China’s AI pioneers, “resurrected” its late founder Tang Xiao’ou who had passed away last December to deliver a speech to employees in an online meeting on Mar 1 – triggering discussions about death and the ethics of technology.
“Look forward, never look back,” the digital Mr Tang said. “Let’s drive forward on the path of artificial intelligence.”
Mr Terence Siau, an expert at the Center for Strategic Cyberspace and International Studies has cautioned against the misuse of digital human technology, warning that its potential to “spread harmful or deceitful messages could pose critical threats if fallen into the wrong hands”. 
He cited an example from Singapore to combat AI manipulation by proposing a law to ban the use of deepfakes during elections. “I believe regulators have to come up with very strict frameworks… to ensure (such technology) will not be abused by bad actors,” Mr Siau told CNA.
In China, authorities and social media platforms have started to respond to potential risks that come with AI usage. 
On Sep 9, China’s National Technical Committee 260 on Cybersecurity released the first version of its AI Safety Governance Framework, prioritising AI-related issues like ethical risks, sensitive data collection and outlining data security guidelines for creators. 
Popular Chinese apps like WeChat and Douyin have also stepped up on AI content guidelines and require creators to “prominently label AI-generated content” so users can clearly distinguish between real and virtual.
Experts do not believe that AI will completely replace humans but this overreliance on technology could become “problematic”. 
“Interacting with real humans certainly should not mean being dependent on AI,” said Ms Paterson, adding that technology “is ultimately a commercial product which has the option of being turned off”.
“Machines are tools,” she said. “Humans thrive best when interacting with other humans. That alone shows these relationships aren’t truly real.”  
For now, Mr Chin is looking forward to more usage of his own digital avatar. He continues to see the technology as a valuable teaching aid and once it’s ready, he plans to introduce it to other educators and hopes they will follow suit with their own digital humans. 
“Lecturers will be happy because they can engage participants on any issues and dive into case discussions,” he said.
While he is aware of the controversies and potential risks such advanced technology could pose, he remains optimistic about the role of digital humans in education. 
“It’s complementary, not a total replacement,” he said, noting that students would come better prepared after watching pre-recorded lectures.