Voice Cloning Is Reshaping Film Audio: What ‘The Brutalist’ Taught Us About AI Voice Refinement

Filmmaking has always been a place where technology and creativity meet. However, the recent revelation that AI played a huge role in the post-production of The Brutalist, a 2024 drama directed by Brady Corbet, has sparked quite a backlash about the role of artificial intelligence in preserving the authenticity of performances.

The film, which was a critical contender in this year's awards season, used AI tools to adjust Hungarian accents and even generate architectural designs. This news gives us an excellent opportunity for us to dive into how AI is changing the film industry, specifically in the realm of voice refinement, and how tools like LALAL.AI’s Voice Cloner are contributing to creators' approach to post-production.

AI in Post-Production: The Case of The Brutalist

The Brutalist is an ambitious, Golden Globe and Oscar-winning film about László Tóth, a Hungarian architect who survives the Holocaust and later emigrates to the United States. Despite being produced on a modest budget of just $10 million, the film has been praised for its grand scope and impressive performances from Adrien Brody and Felicity Jones. However, the movie has also grabbed headlines (such as below) due to its use of artificial intelligence to refine the actors' Hungarian accents and assist in creating architectural mockups.

According to director Brady Corbet, the actors spent months working with dialect coach Tanera Marshall to perfect their Hungarian accents. Yet, despite this intensive training, some inaccuracies remained in the final audio track. But the creative team found a way, which was an AI tool that assisted in refining the actors' pronunciations. The process was done after the traditional Automated Dialogue Replacement (ADR) process, which had already addressed some of the accent issues, but the results still needed fine-tuning.

While the use of AI raised questions in some circles about the integrity of the performances, Corbet said that no changes were made to the actors' English-language lines, and the intent was not to replace or alter their work, but rather to ensure accuracy. He clarified to Deadline, "The aim was to preserve the authenticity of Adrien and Felicity’s performances in another language, not to replace or alter them and done with the utmost respect for the craft." In essence, the AI was there to enhance the speech, and, ironically, make it sound more natural, not alter it which is something that’s not always easy to achieve, especially when tackling a difficult language like Hungarian.

Despite such mixed reactions to the use of AI in The Brutalist, the film remains a strong contender in this year's awards season. The movie has received numerous awards, including a Golden Globe for Best Motion Picture – Drama and several Oscar wins, namely Best Actor in a Leading Role, Best Cinematography, and Best Original Score. 

Where Voice Cloning Comes In

For filmmakers and sound engineers looking to refine their audio in a manner similar to The Brutalist, tools like LALAL.AI’s Voice Cloner offers an ideal solution. This cutting-edge tool allows for high-quality voice cloning, enabling users to create a voice model based a speaker's characteristics and adjusts specific elements like pronunciation.

Hypothetically, here's how the workflow of the speech refinement in The Brutalist might've looked like:

1. Adrien Brody and Felicity Jones worked with a dialect coach, Tanera Marshall, for months to perfect their Hungarian accents but some sounds still ended up unnatural.
2. The team first tried to use ADR to correct the difficult sounds, but this didn't give satisfactory results either. They also attempted to replace the dialogue entirely with other actors, which also failed.
3. The production team turned to an AI voice cloning company which finally allowed for the creation of a voice model based on a speaker's characteristics. We think they may have cloned Brody’s and Jones’ voices to generate AI-powered audio that retained their vocal identity but applied the refined Hungarian pronunciation from the native speaker’s performance.
4. Editor Dávid Jancsó, a native Hungarian speaker, re-recorded parts of the corrected dialogue to ensure that the changes were natural, delivering the same lines as Adrien Brody and Felicity Jones as a reference for both linguistic accuracy and the natural rhythm of Hungarian speech.
5. Jancsó blended his own voice with the actors' performances to enhance authenticity. He noted that "most of their Hungarian dialogue has a part of me talking in there," so the team did take care to maintain the actors' original delivery while making minimal adjustments for accuracy.
6. Once the AI-generated dialogue was created, the sound team would have carefully synchronized it with the actors' lip movements. Any inconsistencies in timing or articulation could have been corrected in post-production using traditional ADR techniques or AI-powered lip-syncing tools.

If your native language isn't English, you probably know how funny it can sound when Hollywood actors try to replicate your native language. But it looks like the days of those linguistic giggles are coming to an end.

Why AI Voice Refinement Is the Future of Filmmaking

In The Brutalist, AI helped streamline what would have otherwise been a tedious manual process of refining dialogue. This isn’t the first time a film has incorporated AI technology to streamline post-production though, and it certainly won’t be the last. According to Deadline, adjusting audio in post is a routine process across the film industry.

Let's take Rami Malek’s Oscar-winning turn as Freddie Mercury in Bohemian Rhapsody as an example. As Rolling Stone reported, much of the film’s singing wasn’t purely Malek’s voice; it was a fusion of Queen’s original vocal tracks and recordings by Canadian singer Marc Martel, whose voice closely resembles Mercury’s. The final result was a carefully crafted blend of multiple sources.

A similar approach was taken in Netflix’s Maria, where Angelina Jolie’s singing was mixed with recordings of opera legend Maria Callas. More recently, Emilia Perez, another Netflix awards contender, reportedly used AI to expand Karla Sofía Gascón’s vocal range by combining her voice with that of composer Camille, who scored the film.

For filmmakers, content creators, and anyone working with audio, tools like LALAL.AI’s Voice Cloner are not just a glimpse into the future, they are becoming a part of the must-have stack.