<
8000
svg focusable="false" aria-label="Issue" class="octicon octicon-issue-opened Octicon-sc-9kayk9-0 hTWZgt" role="img" viewBox="0 0 16 16" width="16" height="16" fill="currentColor" display="inline-block" overflow="visible" style="vertical-align:text-bottom">
Description
🚀 The feature, motivation and pitch
As suggested in the error message, I am reporting this error so its implementation can be prioritized.
Trying to use forward AD with _scaled_dot_product_flash_attention that does not support it because it has not been implemented yet.
Trying to use forward AD with _scaled_dot_product_flash_attention_for_cpu that does not support it because it has not been implemented yet.
Alternatives
No response
Additional context
torch.__version__: '2.3.1+cu121'
cc @ezyang @albanD @gqchen @pearu @nikitaved @soulitzer @Varal7