Description
🐛 Describe the bug
As of the 20250430 nightly cpu build, some previously working torch operations are now returning a "Can't be indexed using 32-bit iterator" error.
For example:
torch.group_norm and torch.bmm
I tried to create a simple test case, but it require a 1.45 GB tensor file and while the test case gives a "Can't be indexed using 32-bit iterator" error on the 20250430 build, it gives a "Expected all tensors to be on the same device, but found at least two devices, mps:0 and cpu!" on the 20250429 build. If I load the tensor on to the CPU both builds work as expected, which makes me believe that the tensor is not being loaded on to the CPU for some reason. Whatever change causes the error message to change appears to be causing the problem.
I'm not sure how long the tensor needs to be, but it ends with "device='mps:0', dtype=torch.float16"
import torch
import torch.nn as nn
groups = 32
eps = 1e-6
in_channels = 256
hidden_states = torch.load("tensor.pt")
input = torch.randn(20, 6, 10, 10)
m = nn.GroupNorm(groups, in_channels, eps=eps, affine=True)
m(hidden_states)
I'm getting the torch.group_norm running https://github.com/brandon929/FramePack with any Torch release after 20250429. I get the following running with the default settings:
File "/Users/user/python/FramePack/.venv-312/lib/python3.12/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 174, in forward
hidden_states = self.norm1(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user/python/FramePack/.venv-312/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1767, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user/python/FramePack/.venv-312/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1778, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user/python/FramePack/.venv-312/lib/python3.12/site-packages/torch/nn/modules/normalization.py", line 313, in forward
return F.group_norm(input, self.num_groups, self.weight, self.bias, self.eps)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user/python/FramePack/.venv-312/lib/python3.12/site-packages/torch/nn/functional.py", line 2974, in group_norm
return torch.group_norm(
^^^^^^^^^^^^^^^^^
File "/Users/user/python/FramePack/.venv-312/lib/python3.12/site-packages/torch/_refs/__init__.py", line 3235, in native_group_norm
out = (input_reshaped - mean) * rstd
~~~~~~~~~~~~~~~^~~~~~
RuntimeError: Can't be indexed using 32-bit iterator
See comfyanonymous/ComfyUI#8066 (comment) for the torch.bmm error.
Here's the commits from that time period - https://github.com/pytorch/pytorch/commits/main/?since=2025-04-29&until=2025-04-30&before=c2e91157575a6064e19f2701e85af4035d33a23a+35
These look somewhat suspect based on the title: #152371, #152436
Versions
Collecting environment information...
PyTorch version: 2.8.0.dev20250430
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 15.5 (arm64)
GCC version: Could not collect
Clang version: 17.0.0 (clang-1700.0.13.5)
CMake version: version 4.0.2
Libc version: N/A
Python version: 3.10.17 (main, Apr 8 2025, 12:10:59) [Clang 17.0.0 (clang-1700.0.13.3)] (64-bit runtime)
Python platform: macOS-15.5-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Apple M4 Max
Versions of relevant libraries:
[pip3] numpy==1.26.2
[pip3] torch==2.8.0.dev20250430
[pip3] torchaudio==2.6.0.dev20250430
[pip3] torchsde==0.2.6
[pip3] torchvision==0.22.0.dev20250430
[conda] No relevant packages
cc @ezyang @gchanan @zou3519 @kadeng @msaroufim @kulinseth @albanD @malfet @DenisVieriu97 @jhavukainen