8000 fix(datasets): Fix contiguous buffer in load MNIST dataset by andreyvelich · Pull Request #86 · ml-explore/mlx-data · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix(datasets): Fix contiguous buffer in load MNIST dataset #86 8000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

fix(datasets): Fix contiguous buffer in load MNIST dataset #86

wants to merge 1 commit into from

Conversation

andreyvelich
Copy link
Contributor
@andreyvelich andreyvelich commented Mar 24, 2025 •
edited
Loading

This should fix this error when using the load_mnist() dataset:

    return dx.buffer_from_vector(pickle.load(f))
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
RuntimeError: [to_array] Contiguous buffer expected -- maybe cast to np.array

Additionally, I correctly re-shape an MNIST image, like here.

I have a working example using this API for distributed MNIST training here: kubeflow/trainer#2565
If you're happy with the changes, I can update the MNIST example to support distributed training and re-use MLX Data API:

from mlx.data.datasets import load_mnist

/assign @angeloskath @awni

@andreyvelich
fix(datasets): Fix contiguous buffer in MNIST dataset
4610f40 
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
@angeloskath angeloskath mentioned this pull request Mar 24, 2025
Merged
@angeloskath
Copy link
Member

Thanks for finding and reporting that! I pushed a better fix in #87 so I will close this. Feel free to reopen or open another if you encounter more issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@andreyvelich @angeloskath
0