https://www.dafei.me/series/%E8%AF%AD%E9%9F%B3%E5%85%8B%E9%9A%86/ Recent content in 语音克隆 on 大飞的博客 Hugo zh-cn Tue, 26 May 2026 00:00:00 +0000 https://www.dafei.me/posts/voice-02-overview/ Tue, 26 May 2026 00:00:00 +0000 https://www.dafei.me/posts/voice-02-overview/ <p>目前开源语音克隆项目百花齐放，本文整理了 GitHub 上最热门的项目，对比各自的架构、语言支持、适用场景，帮你快速选型。</p> <hr> <h2 id="stars-排行榜">Stars 排行榜</h2> <table> <thead> <tr> <th>排名</th> <th>项目</th> <th>Stars</th> <th>核心特点</th> </tr> </thead> <tbody> <tr> <td>1</td> <td><a href="https://github.com/CorentinJ/Real-Time-Voice-Cloning">Real-Time-Voice-Cloning</a></td> <td>~60k</td> <td>经典三阶段流水线，最早流行的克隆项目</td> </tr> <tr> <td>2</td> <td><a href="https://github.com/RVC-Boss/GPT-SoVITS">GPT-SoVITS</a></td> <td>~58k</td> <td>中文最强，1分钟数据即可微调</td> </tr> <tr> <td>3</td> <td><a href="https://github.com/coqui-ai/TTS">Coqui TTS (XTTS)</a></td> <td>~45k</td> <td>工业级框架，17种语言，3秒零样本</td> </tr> <tr> <td>4</td> <td><a href="https://github.com/2noise/ChatTTS">ChatTTS</a></td> <td>~39k</td> <td>中文对话语音质量领先，支持笑声停顿等</td> </tr> <tr> <td>5</td> <td><a href="https://github.com/suno-ai/bark">Bark</a></td> <td>~39k</td> <td>创意音频，可生成笑声/哭声/背景音效</td> </tr> <tr> <td>6</td> <td><a href="https://github.com/babysor/MockingBird">MockingBird</a></td> <td>~37k</td> <td>中文版 Real-Time-Voice-Cloning</td> </tr> <tr> <td>7</td> <td><a href="https://github.com/myshell-ai/OpenVoice">OpenVoice</a></td> <td>~37k</td> <td>音色迁移架构，推理极快</td> </tr> <tr> <td>8</td> <td><a href="https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI">RVC</a></td> <td>~36k</td> <td>歌声转换首选，支持实时变声</td> </tr> <tr> <td>9</td> <td><a href="https://github.com/fishaudio/fish-speech">Fish Speech</a></td> <td>~31k</td> <td>端到端，多语言，工业级质量</td> </tr> <tr> <td>10</td> <td><a href="https://github.com/svc-develop-team/so-vits-svc">so-vits-svc</a></td> <td>~28k</td> <td>AI翻唱鼻祖，歌声克隆质量极高</td> </tr> <tr> <td>11</td> <td><a href="https://github.com/resemble-ai/chatterbox">Chatterbox</a></td> <td>~25k</td> <td>Resemble AI开源，情感可控</td> </tr> <tr> <td>12</td> <td><a href="https://github.com/FunAudioLLM/CosyVoice">CosyVoice</a></td> <td>~21k</td> <td>阿里出品，LLM+流匹配，5种语言</td> </tr> <tr> <td>13</td> <td><a href="https://github.com/index-tts/index-tts">Index-TTS</a></td> <td>~21k</td> <td>bilibili出品，工业级稳定</td> </tr> <tr> <td>14</td> <td><a href="https://github.com/SWivid/F5-TTS">F5-TTS</a></td> <td>~15k</td> <td>流匹配，非自回归，推理快</td> </tr> <tr> <td>15</td> <td><a href="https://github.com/k2-fsa/OmniVoice">OmniVoice</a></td> <td>~6.6k</td> <td>支持600+语言，扩散语言模型</td> </tr> </tbody> </table> <hr> <h2 id="完整对比表">完整对比表</h2> <table> <thead> <tr> <th>项目</th> <th>架构</th> <th>支持语言</th> <th>零样本克隆</th> <th>推理速度</th> <th>需要训练</th> <th>最适合场景</th> </tr> </thead> <tbody> <tr> <td><a href="https://github.com/CorentinJ/Real-Time-Voice-Cloning">Real-Time-Voice-Cloning</a></td> <td>三阶段流水线</td> <td>英文</td> <td>是（5s）</td> <td>接近实时</td> <td>否</td> <td>学习/原型验证</td> </tr> <tr> <td><a href="https://github.com/RVC-Boss/GPT-SoVITS">GPT-SoVITS</a></td> <td>GPT + VITS</td> <td>中/英/日</td> <td>是（可微调）</td> <td>中等</td> <td>可选</td> <td><strong>中文配音首选</strong></td> </tr> <tr> <td><a href="https://github.com/coqui-ai/TTS">Coqui TTS</a></td> <td>多架构框架</td> <td>17种</td> <td>是（3s）</td> <td>中等</td> <td>否</td> <td>多语言生产部署</td> </tr> <tr> <td><a href="https://github.com/suno-ai/bark">Bark</a></td> <td>自回归LM</td> <td>13种</td> <td>弱</td> <td>慢</td> <td>否</td> <td>创意/情感音频</td> </tr> <tr> <td><a href="https://github.com/2noise/ChatTTS">ChatTTS</a></td> <td>生成式LM</td> <td>中/英</td> <td>是</td> <td>中等</td> <td>否</td> <td><strong>中文对话TTS</strong></td> </tr> <tr> <td><a href="https://github.com/babysor/MockingBird">MockingBird</a></td> <td>三阶段流水线</td> <td>中/英</td> <td>是（5s）</td> <td>接近实时</td> <td>否</td> <td>中文克隆</td> </tr> <tr> <td><a href="https://github.com/myshell-ai/OpenVoice">OpenVoice</a></td> <td>音色迁移</td> <td>6种</td> <td>是</td> <td><strong>极快</strong></td> <td>否</td> <td>快速克隆部署</td> </tr> <tr> <td><a href="https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI">RVC</a></td> <td>VITS + 特征检索</td> <td>语言无关</td> <td>需少量训练</td> <td><strong>实时</strong></td> <td>是（&lt;10min）</td> <td><strong>歌声/实时变声</strong></td> </tr> <tr> <td><a href="https://github.com/fishaudio/fish-speech">Fish Speech</a></td> <td>端到端AR + VQGAN</td> <td>8+种</td> <td>是（10s）</td> <td>快</td> <td>否</td> <td>高质量多语言</td> </tr> <tr> <td><a href="https://github.com/svc-develop-team/so-vits-svc">so-vits-svc</a></td> <td>SoftVC + VITS</td> <td>语言无关</td> <td>需训练</td> <td>中等</td> <td>是</td> <td>AI翻唱</td> </tr> <tr> <td><a href="https://github.com/resemble-ai/chatterbox">Chatterbox</a></td> <td>s3gen + 扩散</td> <td>英文</td> <td>是</td> <td>快</td> <td>否</td> <td>英语情感配音</td> </tr> <tr> <td><a href="https://github.com/FunAudioLLM/CosyVoice">CosyVoice</a></td> <td>LLM + 流匹配</td> <td>中/英/日/粤/韩</td> <td>是（3-10s）</td> <td>快</td> <td>否</td> <td>商业TTS服务</td> </tr> <tr> <td><a href="https://github.com/index-tts/index-tts">Index-TTS</a></td> <td>GPT-style</td> <td>中/英</td> <td>是</td> <td>快</td> <td>否</td> <td>工业生产</td> </tr> <tr> <td><a href="https://github.com/neonbjb/tortoise-tts">Tortoise TTS</a></td> <td>自回归 + 扩散</td> <td>英文</td> <td>是</td> <td>慢</td> <td>否</td> <td>高质量英语有声书</td> </tr> <tr> <td><a href="https://github.com/SWivid/F5-TTS">F5-TTS</a></td> <td>流匹配 DiT</td> <td>中/英</td> <td>是</td> <td>快</td> <td>否</td> <td>快速零样本克隆</td> </tr> <tr> <td><a href="https://github.com/yl4579/StyleTTS2">StyleTTS2</a></td> <td>扩散 + SLM</td> <td>英文</td> <td>是</td> <td>中等</td> <td>否</td> <td>英语质量基准</td> </tr> <tr> <td><a href="https://github.com/Plachtaa/VALL-E-X">VALL-E-X</a></td> <td>神经编解码LM</td> <td>英/中/日</td> <td>是</td> <td>中等</td> <td>否</td> <td>跨语言克隆</td> </tr> <tr> <td><a href="https://github.com/hexgrad/kokoro">Kokoro</a></td> <td>StyleTTS2变体</td> <td>英+中日韩法</td> <td>否（固定音色）</td> <td><strong>极快</strong></td> <td>否</td> <td>本地轻量部署</td> </tr> <tr> <td><a href="https://github.com/k2-fsa/OmniVoice">OmniVoice</a></td> <td>扩散语言模型</td> <td><strong>600+种</strong></td> <td>是</td> <td>快</td> <td>否</td> <td><strong>超多语言</strong></td> </tr> <tr> <td><a href="https://github.com/netease-youdao/EmotiVoice">EmotiVoice</a></td> <td>BERT + TTS</td> <td>中/英</td> <td>有限</td> <td>中等</td> <td>否</td> <td>情感化中文TTS</td> </tr> <tr> <td><a href="https://github.com/jasonppy/VoiceCraft">VoiceCraft</a></td> <td>自回归 + 码书重排</td> <td>英文</td> <td>是</td> <td>中等</td> <td>否</td> <td><strong>语音片段编辑</strong></td> </tr> <tr> <td><a href="https://github.com/QwenLM/Qwen3-TTS">Qwen3-TTS</a></td> <td>LLM-native</td> <td>多语言</td> <td>是</td> <td>中等</td> <td>否</td> <td>与LLM集成</td> </tr> <tr> <td><a href="https://github.com/metavoiceio/metavoice-src">MetaVoice</a></td> <td>自回归 + 扩散(1B)</td> <td>英文</td> <td>是（30s）</td> <td>中等</td> <td>否</td> <td>高质量英语克隆</td> </tr> <tr> <td><a href="https://github.com/huggingface/parler-tts">Parler-TTS</a></td> <td>描述控制TTS</td> <td>英文</td> <td>文本描述控制</td> <td>中等</td> <td>否</td> <td>可控TTS研究</td> </tr> <tr> <td><a href="https://github.com/k2-fsa/ZipVoice">ZipVoice</a></td> <td>流匹配 + 蒸馏</td> <td>中/英</td> <td>是</td> <td><strong>极快</strong></td> <td>否</td> <td>低延迟实时TTS</td> </tr> <tr> <td><a href="https://github.com/espnet/espnet">ESPnet</a></td> <td>多架构工具包</td> <td>多语言</td> <td>有</td> <td>依模型</td> <td>通常需要</td> <td>学术研究</td> </tr> </tbody> </table> <hr> <h2 id="按使用场景推荐">按使用场景推荐</h2> <h3 id="中文语音克隆">中文语音克隆</h3> <p><strong>GPT-SoVITS &gt; ChatTTS &gt; CosyVoice &gt; Index-TTS</strong></p> https://www.dafei.me/posts/voice-01-omnivoice/ Mon, 25 May 2026 00:00:00 +0000 https://www.dafei.me/posts/voice-01-omnivoice/ <p>想让 AI 用你的声音说话？只需要一段 3 到 10 秒的录音，OmniVoice 就能克隆你的声音，说出任意文字。</p> <hr> <h2 id="omnivoice-是什么">OmniVoice 是什么</h2> <p>OmniVoice 是小米 / k2-fsa 团队开发的开源零样本 TTS 模型，支持 600+ 种语言，基于<strong>扩散语言模型</strong>架构，推理速度极快（RTF 最低 0.025，比实时快 40 倍）。</p> <p>支持三种模式：</p> <ul> <li><strong>语音克隆</strong>：提供参考音频，克隆声音说出任意文字</li> <li><strong>语音设计</strong>：用文字描述声音（性别、年龄、音调、口音等）</li> <li><strong>自动语音</strong>：模型自动选择声音</li> </ul> <hr> <h2 id="环境准备">环境准备</h2> <h3 id="安装-conda-环境">安装 conda 环境</h3> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>conda create -n omnivoice python<span style="color:#f92672">=</span>3.11 -y </span></span><span style="display:flex;"><span>conda activate omnivoice </span></span></code></pre></div><h3 id="安装-pytorch">安装 PyTorch</h3> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#75715e"># Apple Silicon</span> </span></span><span style="display:flex;"><span>pip install torch<span style="color:#f92672">==</span>2.8.0 torchaudio<span style="color:#f92672">==</span>2.8.0 </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># NVIDIA GPU（以 CUDA 12.8 为例）</span> </span></span><span style="display:flex;"><span>pip install torch<span style="color:#f92672">==</span>2.8.0+cu128 torchaudio<span style="color:#f92672">==</span>2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128 </span></span></code></pre></div><h3 id="安装-omnivoice">安装 OmniVoice</h3> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>git clone https://github.com/k2-fsa/OmniVoice.git </span></span><span style="display:flex;"><span>cd OmniVoice </span></span><span style="display:flex;"><span>pip install -e . </span></span></code></pre></div><hr> <h2 id="下载模型">下载模型</h2> <p>第一次运行会自动下载。如果在国内连不上 HuggingFace，设置镜像：</p>