spkname128 ckpt22010 Prosody 朗读

spkname128 checkpoint22010 未合并

用上述模型直接朗读 /proj/mmllm/prosody/texts 的 4 个文本(1 段独白 + 3 段客服对话)。无指令、无 caption、无 system prompt:每个 turn 以 <|im_start|>角色\n文本[SEP_1] 输入,模型自由续写 CosyVoice3 音频 token 直到采样出 [SEP_2];同一文本的 turn 按预训练格式串联在上下文中。对话的角色名直接用原文件里的 user / agent。

checkpoint/proj/spans/lyf/checkpoint/spkname128-20261026/checkpoint-22010 加载方式Qwen3-8B-Base + 原始 LoRA(未与 Instruct 合并、未缩放) 采样评测规则:文本 0.1 / 音频 0.7(质量守恒)、top_k 150、top_p 0.9;每 turn 最多 1000 音频 token;seed = turn 序号 解码音色CosyVoice3 flow+HiFT,参考音频只决定音色,语言模型看不到。对话里 agent(Baymax Health 客服)句子用 EARS p001 真人男声(愉悦版录音)解码;user 句子和 Boston 独白用新闻播报女声 转写核对WhisperX large-v2 转写每个 turn,WER 对比输入文本(小写、去标点、连字符拆开)

汇总

文本变体turn正常结束音频 s音频 tokenWER 均值WER 中位WER≤0.2
Boston整段一个 turn,角色 host11/118.14520.1360.1361/1
Boston每行一个 turn(上下文串联),角色 host33/316.74180.060.03/3
Dialog 1:急诊室保险覆盖查询每行一个 turn(上下文串联),角色名按原文件:user→user、agent→agent1717/1748.312080.0650.016/17
Dialog 2:会员卡补发每行一个 turn(上下文串联),角色名按原文件:user→user、agent→agent2019/2099.424840.0580.019/20
Dialog 3:配偶团体计划生效确认每行一个 turn(上下文串联),角色名按原文件:user→user、agent→agent1212/1237.19270.0080.012/12

WER 均值按单句上限 1.0 截断;拼接音频里 turn 之间插入了 0.3 s 静音,仅用于播放。"OK." 被 WhisperX 写成 "Okay.",WER 记为 1.0,属于拼写归一化问题而非朗读错误。

Boston(独白,3 句)

Yeah! We'd love to talk about Boston! It is one of the oldest cities in the United States with a rich history dating back to sixteen thirty when it was first settled by English Puritans. You'll be able to walk along the Freedom trail, which takes you to sixteen historic sites that played a big role in American history.

同一文本用 2 种设置各合成一次(同一模型、同一采样参数),下面每个卡片是一种设置:

设置 1:整段一个 turn,角色 host

1 个 turn · 拼接 18.38 s · 生成耗时 19.25 s
整段连播(1 句按顺序拼接,句间 0.3 s 静音):
逐句(每句独立音频):

设置 2:每行一个 turn(上下文串联),角色 host

3 个 turn · 拼接 17.62 s · 生成耗时 19.74 s
整段连播(3 句按顺序拼接,句间 0.3 s 静音):
逐句(每句独立音频):

Dialog 1:急诊室保险覆盖查询(17 句)

user: A family member is critically ill and in the emergency room, I need to verify the coverage. agent: I can't imagine how hard this must be for you. agent: I will get the information right away. agent: In order to verify who you are, please provide your date of birth. user: Sure. My date of birth is January fourth, nineteen ninety-one. agent: Thank you. I can confirm your plan has ER coverage with a co-pay of one hundred dollars. agent: There is no deductible or out of pocket for emergency room visits. agent: Would you like me to send you a summary of your coverage through text? user: Yes. agent: Thank you. I will text you the summary of your coverage to the number you are calling from ending with six six five four. user: OK. agent: Please let me know if I can provide any other assistance. user: No, I'm good. agent: I am sorry about your situation. agent: I hope your family member gets better soon. agent: Please reach out with any further concerns. agent: Thank you for calling Baymax Health.

每行一个 turn(上下文串联),角色名按原文件:user→user、agent→agent

17 个 turn · 拼接 53.42 s · 生成耗时 64.29 s
整段连播(17 句按顺序拼接,句间 0.3 s 静音):
逐句(每句独立音频):

Dialog 2:会员卡补发(20 句)

user: I have requested a member ID card multiple times, but have not yet received it. agent: I am sorry you feel frustrated. agent: I will help you get a member ID card right away. agent: In order to verify who you are, I need you to provide your date of birth. user: Sure. My date of birth is January fourth, nineteen ninety-one. agent: Thank you. I will text you a copy of your member ID card to the number you are calling from, ending with six six five four. agent: This ensures you will immediately get your ID. user: OK. agent: I have just sent you the ID card through text. agent: Please confirm if you have received it. user: I got it. agent: Would you also like me to send a physical copy of your ID card to your home address with zip code one one zero zero one? user: Yes. agent: I have sent the ID card. agent: It will arrive at your address within seven business days. agent: Is there anything else I can help you with? user: No. agent: I am sorry it took us so long to respond to your request. agent: Please reach out with any further concerns. agent: Thank you for calling Baymax Health.

每行一个 turn(上下文串联),角色名按原文件:user→user、agent→agent

20 个 turn · 拼接 105.36 s · 生成耗时 118.13 s
整段连播(20 句按顺序拼接,句间 0.3 s 静音):
逐句(每句独立音频):

Dialog 3:配偶团体计划生效确认(12 句)

user: I recently got married and got the benefits through my spouse group plan. I want to check if my plan is active. agent: Congratulations! agent: I would be glad to help you! agent: In order to verify who you are, please provide your date of birth. user: Sure. My date of birth is January fourth, nineteen ninety one. agent: Thank you! I can confirm your coverage is effective as of June first, twenty twenty-five, and you are enrolled in the PPO plus plan. agent: Would you like me to provide you a summary of your in-network benefits? user: No. agent: OK, is there anything else I can help you with? user: No, I'm good. agent: Thank you for calling Baymax Health! agent: Have a great day!

每行一个 turn(上下文串联),角色名按原文件:user→user、agent→agent

12 个 turn · 拼接 40.68 s · 生成耗时 47.59 s
整段连播(12 句按顺序拼接,句间 0.3 s 静音):
逐句(每句独立音频):

每句一个独立音频(mp3 48 kbps,由该句的 WAV 转码),变体顶部的播放器是整段拼接;播放任一音频时其他音频自动暂停。原始 24 kHz WAV、result.json 和 prompt/response JSONL 在实验目录 outputs/ 下。