Awesome quality for a zero-shoot! I'll give it a try. Also thanks for the update on the upcoming voice clone video! I'm glad it's coming, hopefully during this week!

Cool stuff. Appreciate the update on the next voice clone repository video

It is worth to note that both Voicecraft and Audiocraft cannot be used commercially (weights have non-commercial licenses). Most people probably will not care as it is hard to prove, but I guess some people would like to know that. AFAIK you can use Tortoise (and probably RVC) commercially. Thanks for the video though, the results from just 3s of audio are just stunning. I get it that you can copy the voice patterns pretty well if you get like a few minutes of speech, but what you can get here with just 3 seconds is crazy.

@FenrirRobu

3 ай бұрын

Exactly,

SWEET finally, we are so BACK!

VoiceCraft seems to be the best one I've heard so far -- at least on KZread.

@Ovyron

3 ай бұрын

Lyrebird was at this level long time ago, but it was commercial, yeah, finally open source caught up!

Thanks Man for all you do

Hello, why do I need one voice sample in the tortoise auto aggressive fine-tuned model in the voices folder? Why can’t I select just none and will the voice sample affect the quality of the trained voice I mean when you put voice samples in the voices folder after training and selecting the model and wears the prepare a new language tab and how does it work?

Any chance you can do your magic and create a one click install that will work on windows if you have an Nvidia GPU? Pretty please?

@JaysterJayster

Ай бұрын

Just found a KZreadr who did it on his patreon :)

oh, awesome. progress. we need more of it!

thanks for all the insights !!

very cool I have to say I learned a lot from you especially when it comes to RVC so thank you very much but even with all of that I could never get a voice model that is identical to the original voice any help with that?

How does the cloning do on voices with special effects tied to them? I.e. SHODAN from System Shock? Will a large enough finetuning dataset accomplish good results, or will the inherent underlying training cause issue from not training on voices with special effects/pattern variances?

I can't get docker working, it's breaking on the first cell. Would LOVE a walkthrough.

Thank you for good content. I'm just curious about the commercial use . Since Coqui Public Model License is only for non-commercial use . Is it good to use output of this model in youtube ? I'm also planning to use tts in my content.

@Jarods_Journey

3 ай бұрын

Probably not, I'm not a lawyer lol.

please do a video on windows setup. I really want something to clone voices from small datasets. I been replacing game audio of Castlevania SOTN. and the PSX voice for Dracula doesn't clone because of only one minute of audio. I really would like to use all the PSX voices but right now been using PSP Dracula. thank you for any help. and thank you for the video.

Love your stuff. I am using this repo and was able to reproduce the output wav files. I need to be able to load the model and then make many inference calls without having to reload the model every time. Do you have insight on how to do this or can you share your visual studio code project on github? So far, I am loading the model using fastapi and unicorn to host it. I can do inference via a post call the first time, but when trying to run again, I get WARNING:phonemizer:words count mismatch I think I need to unload or re-execute something to reset it, without removing the entire model.

Can this do things like laughing, sneezing, coughing or things like that?

great stuff!

@Jarods_Journey

3 ай бұрын

Thanks :)!

Hey Jarrod. Do you think this would be a viable replacement for tortoise for me to use to make audio books? I really need something soon.

@FenrirRobu

3 ай бұрын

The license is "Attribution-NonCommercial-ShareAlike 4.0 International", whether or not that affects you is a complicated question. Tortoise is free to use for almost *anything*.

@Jarods_Journey

3 ай бұрын

If the audiobooks are for personal usage, I don't see why not. But unfortunately, I won't be able to implement this into any of my projects anytime soon

Before I try to istall this: How does it perform with other languages?

Can you share your build? This is the quickest way to use VoiceCraft

have you tried like indian accent or singlish accent ? will it follow?

What you think about using cloud gpu?

I think the voice engine by open ai is just hype now because its from open ai

What about the best open source speech to text? Is there nothing better than Whisper in accuracy?

@Jarods_Journey

3 ай бұрын

Whisper is the best afaik

What's the lowest power Nvidia card needed to make this reasonably work

@zachary3603

Ай бұрын

if you don't know the answer to this, installing it is going to be a NIGHTMARE

could you make a one click installer for us? that would be greatly appreciated!

It works only in english?

Do you have suggestions for real time TTS models that are open source?

@pawelszpyt1640

3 ай бұрын

Tortoise with the right settings I guess. Select DeepSpeed and low samples (2) / iterations (

@AEONIC_MUSIC

3 ай бұрын

Alltalk is the best I have seen better then tortoise and it has a streaming mode it does one word at a time and plays it do you could probably do real time. It's also faster

@pawelszpyt1640

3 ай бұрын

​@@AEONIC_MUSIC It uses XTTSv2, which is very powerful model, but released under terrible Coqui license. Not open source at all. I considered paying Coqui for it, but before I pulled the trigger, they shut down their business and now you have no way to legally use it at all.

@Jarods_Journey

3 ай бұрын

Tortoise TTS would be it or xtts if you don't care about licensing. Styletts2 is a good contender as well

Thank you for this useful and wonderful channel. I have been searching for a long time for a program that converts text to speech for Arabic. I have tried many programs and found that they do not reach the desired level regarding the Arabic language. The best one was eleven labs, but it's not free. Also, xtts coqui is good for the Arabic language, but it needs improvement. Currently I am looking for a program that converts text into speech. It can be trained to pronounce the Arabic language.. I do not know how to train xtts coqui in the Arabic language.

How does this compare to my boy tort TTs RVC ?

@AEONIC_MUSIC

3 ай бұрын

Alltalk is way better and faster at everything then tortoise but im curious to see what this one is like compared to allltalk

@Jarods_Journey

3 ай бұрын

Haven't done any thourough comparisons, but, it's pretty darn good lol

Does it work well for other languages?

@Jarods_Journey

3 ай бұрын

I think it only does English right now

Does this come with the webui?

@greypsyche5255

3 ай бұрын

No, it comes with a jupyter note and I have no idea how to use this thing.

@Jarods_Journey

3 ай бұрын

No, I built the webui for it to make it easier to use. Someone said they were going to PR their own version of one on the repo, so we'll have to wait and see

Cool. Does it run on CPU or you must have a graphics card.

@Jarods_Journey

3 ай бұрын

Need a GPU, will run on CPU, but way too slow.

Pleas can use with arabic voice

Hi, it can talk another languages ?? By the way, tanks for all the work, nice channel to creators.

@Jarods_Journey

3 ай бұрын

Just English I think for now

it's getting scarier and scarier

Can I run this without GPU any method 😢

@Jarods_Journey

3 ай бұрын

I don't recommend it. It seems to run on CPU, but the outputs take much too long to generate.

I made an API server for VoiceCraft as well as added it to my audiobook/dubbing generation app, Pandrator. Both run on Windows and Pandrator has a one-click installer. I'm not sure what I think about it yet, to be honest. I achieve very good results with XTTS, but I cannot experiment with VoiceCraft too much, because generation is very slow on my measly 4GB 3050 (laptop), slower than processing XTTS results with RVC, even. I have only tried the smaller model (though, according to the author, the difference in quality is negligible). Sometimes it drastically changes the pitch, it sounds as though a sentence or a part of one was generated using a different voice altogether. It can be mitigated by playing with the parameters a little, probably. The quality of voice cloning alone is much better than XTTS (I'm not sure about XTTS+RVC), but consistency seems to be worse.

@Ravisidharthan

3 ай бұрын

Hi, man thanks for the work.. Does it support apple silicon? Can you pack one click installer for mac mps? is there any demo online that Mac users can test, Huggingface? And does it support silero Indic languages? ❤ Great work

@vineyardworker

3 ай бұрын

Thanks for Pandrator, much appreciated.

Jarod, How was the inference speed and GPU utilization on your 4090(I assume you're using one)?

@Jarods_Journey

3 ай бұрын

Eh, about 5-15 seconds once alignment was finished using MFA. It is pretty fast generally, but I'm sure it's not optimized either

@joshuashepherd7189

3 ай бұрын

@@Jarods_Journey That's not too bad at all! I'm trying to move away from freaking ElevenLabs for my project. So I'm about to get real heavy into RVC and Tortoise. Gonna hafta binge your videos XD

@HarryClipzFilmz

2 ай бұрын

@@joshuashepherd7189 What is the difference from ElevenLabs and this? I am new to voice cloning and just want to understand the programs to use to do it

Does it support CHinese and cantonese , vietnamese ?

@Jarods_Journey

3 ай бұрын

Just English

How to run in Google colab

@Jarods_Journey

3 ай бұрын

Since it was built from Linux, you can probably just follow the readme

This license allows only non-commercial use of a machine learning model and its outputs. Why even bother...

@FenrirRobu

3 ай бұрын

Did they specify it for outputs? Usually it's been vague but risky, did they finally admit that it's just not usable?

@user-xj5gz7ln3q

3 ай бұрын

@@FenrirRobu Yup.. for output.

@yuyutsurao

3 ай бұрын

You can use they are not going to re check you

@Jarods_Journey

3 ай бұрын

It's cool tech, the abilities for prosody and pitch maintenance from 3 seconds of audio is pretty wild. To put it out in the open means that another project will build upon it, possible with more permissive licenses

@FenrirRobu

3 ай бұрын

@@Jarods_Journey let's see if lucidrains takes up the task.

Hello, why do I need one voice sample in the tortoise auto aggressive fine-tuned model in the voices folder? Why can’t I select just none and will the voice sample affect the quality of the trained voice I mean when you put voice samples in the voices folder after training and selecting the model and wears the prepare a new language tab and how does it work?