8000 [wds] Align aspect ratio bucker on single image + bugfix previous Pr and deadock by blefaudeux · Pull Request #121 · Photoroom/datago · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[wds] Align aspect ratio bucker on single image + bugfix previous Pr and deadock #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 31, 2025

Conversation

blefaudeux
Copy link
Contributor

No description provided.

bugfixing the previous PR, ideally we should unit test more
@blefaudeux blefaudeux changed the base branch from main to ben/webdataset May 31, 2025 22:30
@blefaudeux blefaudeux merged commit 5d91029 into ben/webdataset May 31, 2025
7 checks passed
blefaudeux added a commit that referenced this pull request Jun 2, 2025
* Better error messages on http path

- async tarball pull, but behavior is clunky
- general arch could be simpler and using tokio more
- handling jpg/png/jpeg/cls/txt/json types
- some shuffling handling

missing unit tests, and better behavior, doing pauses at the moment

better documentation

big rewrite, nicer and smaller code I believe (#117)

Co-authored-by: Benjamin Lefaudeux <ben@photoroom.com>

Async tarball pull and dispatch

Random_sampling in the config, at least for now. Thanks for the review Roman !

* Code review (#120)

Some missing items (would be good to propagate the archive name for instance), but most fixes should be there

* second round, hoopefully good to go. Perf could probably be improved, competing sample pull

* handling multi image samples (#121)

bugfixing the previous PR, ideally we should unit test more

* final update round

* second review, not perfect but feels like we can land this and carry on

---------

Co-authored-by: Benjamin Lefaudeux <ben@photoroom.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant
0