YouTube introduces AI-powered Dubbing Tool

In a move to simplify the process of video dubbing and expand content reach, YouTube announced a new AI-powered dubbing tool at VidCon 2023. The tool, developed in collaboration with Aloud an AI-powered dubbing service from Google’s in-house incubator Area 120 transcribes, translates, and produces dubbed versions of videos.

Creators will be able to review and edit the transcription, and once satisfied, Aloud will handle the complex tasks of translation, timing, and speech synthesis, translating the video into the language of choice.

According to Amjad Hanif, YouTube’s VP of Creator Products, the tool will facilitate content creators in reaching a wider international audience at no additional cost. Since the start of 2023, over 10,000 videos have been dubbed in more than 70 languages, demonstrating the tool’s potential to enhance multilingual content accessibility.

The AI-powered dubbing tool is currently being tested with hundreds of creators and will be expanded to all creators in the future. Currently, Aloud supports English, Spanish, and Portuguese, with plans to add more languages, including Hindi and Bahasa Indonesian.

Future enhancements are already on the horizon. YouTube plans to introduce features such as voice preservation, better emotion transfer, and lip reanimation to make translated audio tracks sound more like the original creator’s voice, with improved expression and lip sync. These features are anticipated to roll out in 2024.

While the official launch date of the Aloud update remains unknown, the service’s promising capabilities are on full display. To get a sense of how Aloud sounds, users can check out the Spanish dub track in a video from the Amoeba Sisters channel on YouTube.

The platform ensures that only the original creators can dub their content, adding a layer of content security. Creators keen on early access can join the program by filling out an official Google Docs form. By integrating AI into language translation and dubbing, YouTube is paving the way for a more inclusive and global content creation landscape.