8000 [GPU] Enable vector distribute on reduction operations by default by pashu123 · Pull Request #20751 · iree-org/iree · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[GPU] Enable vector distribute on reduction operations by default #20751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
May 13, 2025
Merged

[GPU] Enable vector distribute on reduction operations by default #20751

merged 10 commits into from
May 13, 2025

Conversation

pashu123
Copy link
Contributor
@pashu123 pashu123 commented May 7, 2025 •
edited
Loading

-- Set only reduction tile sizes on the parallel operation (the
workgroup tile sizes are dominated by the reduction operation).
-- The bitwidth now selects max(operands_bitwidth) for an operation,
this is just to match warp distribution numerics.

@benvanik
Copy link
Collaborator
benvanik commented May 8, 2025

Every kernel, huh?
(Please provide useful descriptions on PRs ;)

@pashu123
Copy link
Contributor Author
pashu123 commented May 8, 2025

Every kernel, huh? (Please provide useful descriptions on PRs ;)

Apologies, I was just testing on the CI.

@benvanik
Copy link
Collaborator
benvanik commented May 8, 2025

(that's cool - in the future, please note that everyone watching the repository sees these and they are enshrined in the repo forever - write something :)

@pashu123 pashu123 changed the title Every kernel [GPU] Enable vector distribute pipeline by default. May 8, 2025
@pashu123 pashu123 force-pushed the every_kernel branch 4 times, most recently from 47491d1 to a7939a8 Compare May 9, 2025 19:55
@pashu123 pashu123 changed the title [GPU] Enable vector distribute pipeline by default. [GPU] Enable vector distribute on reduction operations by default May 9, 2025
@pashu123 pashu123 marked this pull request as ready for review May 12, 2025 21:46
Groverkss
Groverkss approved these changes May 13, 2025
View reviewed changes
Copy link
Contributor
@Groverkss Groverkss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pashu123 added 10 commits May 13, 2025 20:59
@pashu123 pashu123 merged commit c402b9c into iree-org:main May 13, 2025
41 checks passed
@benvanik
Copy link
Collaborator

going to try a rerun, but fyi this may have caused a regression:
https://github.com/iree-org/iree/actions/runs/15007320793/job/42169379980

ERROR iree-test-suites/sharktank_models/benchmarks/model_benchmark_run.py::sdxl :: vae_rocm - check 69.94073186302559 <= 68.2: sdxl vae benchmark time should not regress more than a factor of 1.1

@pashu123
Copy link
Contributor Author

going to try a rerun, but fyi this may have caused a regression: https://github.com/iree-org/iree/actions/runs/15007320793/job/42169379980

ERROR iree-test-suites/sharktank_models/benchmarks/model_benchmark_run.py::sdxl :: vae_rocm - check 69.94073186302559 <= 68.2: sdxl vae benchmark time should not regress more than a factor of 1.1

Sure, if it persists, I'll revert.

@benvanik
Copy link
Collaborator

Looks like it's right on the edge - subsequent merges on main are failing: https://github.com/iree-org/iree/actions/runs/15007458464/job/42169770312

you may need to up the tolerance (or figure out why it regressed)

@pashu123
Copy link
Contributor Author

Looks like it's right on the edge - subsequent merges on main are failing: https://github.com/iree-org/iree/actions/runs/15007458464/job/42169770312

you may need to up the tolerance (or figure out why it regressed)

The latest one is passing though: https://github.com/iree-org/iree/actions/runs/15007816465/job/42171017802

@pashu123
Copy link
Contributor Author

Looks like it's right on the edge - subsequent merges on main are failing: https://github.com/iree-org/iree/actions/runs/15007458464/job/42169770312
you may need to up the tolerance (or figure out why it regressed)

The latest one is passing though: https://github.com/iree-org/iree/actions/runs/15007816465/job/42171017802

This PR also goes through: https://github.com/iree-org/iree/actions/runs/15007320793/job/42172666060 . The CI would be flaky.

@pashu123
Copy link
Contributor Author

Oh! I missed it. Yes, it's right on edge.

@MaheshRavishankar
Copy link
Contributor

Actually looks like only VAE regressed. Would be good to triage that, but almost everything else got better. Maybe we should change the golden time for others as well

@benvanik
Copy link
Collaborator

yeah, if timings went down the lower so we don't backslide!

@pashu123 pashu123 mentioned this pull request May 14, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Groverkss Groverkss Groverkss approved these changes

@MaheshRavishankar MaheshRavishankar Awaiting requested review from MaheshRavishankar MaheshRavishankar is a code owner

@qedawkins qedawkins Awaiting requested review from qedawkins qedawkins is a code owner

@kuhar kuhar Awaiting requested review from kuhar kuhar is a code owner

@hanhanW hanhanW Awaiting requested review from hanhanW hanhanW is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pashu123 @benvanik @MaheshRavishankar @Groverkss
0