I’ve been waiting for an open source TTS model that was actually good enough to capture some of the subtleties of language and synthesize them in a natural-sounding way that makes sense. I think I finally found one that fits the requirements.
Model: https://huggingface.co/fishaudio/fish-speech-1.5
It uses an encoder rather than relying on phonemes, and generations sometimes vary because of that, but the amount of errors I’ve gotten are minimal, and the variations in the generation are all surprisingly natural in slightly different ways, which is very exciting.
Give it a spin if you are also looking for a TTS model that sounds good. It uses voice cloning, so find a good 10-20 second reference clip to have the generations use the same voice.
For a minute I thought there were actually recordings of fish noises from underwater and that someone has put them into TTS.
But their logo is a whale!