Audio Cloning

Mon, 23 Jun 2025 00:00:00 +0000

This is an extension of the Video Translator tool that generates cloned audio for short subtitles, improving the quality of the dubbed output. It uses OpenVoice and MeloTTS to clone the original speaker’s voice.

Challenges

Library compatibility. The libraries weren’t functional on Python 3.12, which I was using — they worked on 3.10. So the TTS stage couldn’t live in the same script as the rest of the pipeline.
Short subtitles. The libraries needed a longer sample to clone a voice well, and very short lines (hello, hi, bye) errored out.

What I changed in the fork

openvoice/se_extractor.py had a check that discarded audio segments shorter than 1.5 seconds. Words like hello fall under that. I relaxed the check to allow segments down to ~0.5 seconds, and added a skip for lines too short to process.
Added melo-tts to requirements.txt (the docs mention it but don’t pin it).
Added a driver script that points at a video-translator output folder and runs cloning across every generated audio file.
Patched the library to run on CPU for Apple M1.

The fork is at github.com/pratts/OpenVoice ; test1.py runs the cloning over the translator’s output.

MeloTTS on Prateek Sharma

Audio Cloning

Challenges

What I changed in the fork