8000 Add flash attn2 support attn_mask, fix some minor bug in eval by ZhikangNiu · Pull Request #1066 · SWivid/F5-TTS · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add flash attn2 support attn_mask, fix some minor bug in eval #1066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
Jun 11, 2025
Merged

Add flash attn2 support attn_mask, fix some minor bug in eval #1066

merged 11 commits into from
Jun 11, 2025

Conversation

ZhikangNiu
Copy link
Collaborator

No description provided.

SWivid
SWivid reviewed May 31, 2025
View reviewed changes
SWivid
SWivid reviewed May 31, 2025
View reviewed changes
@ZhikangNiu
Copy link
Collaborator Author

1 epoch LibriTTS F5TTS v1 Small

flash_attn(with mask):
Max memory allocated: 30925.52 MB
Max memory reserved: 64118.00 MB
1epoch 25min
loss=0.999

standard F5TTS v1 Small no mask
Max memory allocated: 29942.87 MB
Max memory reserved: 61876.00 MB
1epoch 22min
loss=0.939

@ZhikangNiu ZhikangNiu changed the title [WIP]add flash attn2 support attn_mask Add flash attn2 support attn_mask, fix some minor bug in eval Jun 11, 2025
@SWivid SWivid merged commit 0914170 into SWivid:main Jun 11, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SWivid SWivid SWivid left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ZhikangNiu @SWivid
0