10000 Question regarding computing ValueRanges for symbolic nodes · Issue #128640 · pytorch/pytorch · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Question regarding computing ValueRanges for symbolic nodes #128640

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
peri044 opened this issue Jun 13, 2024 &mid 8000 dot; 9 comments
Closed

Question regarding computing ValueRanges for symbolic nodes #128640

peri044 opened this issue Jun 13, 2024 · 9 comments
Labels
module: dynamic shapes oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@peri044
Copy link
Contributor
peri044 commented Jun 13, 2024 •
edited by pytorch-bot bot
Loading

🐛 Describe the bug

Hello,

For some models, we have observed SymInt nodes in the graph which are of the form s0*s1. s0 and s1 are other SymInt Nodes in the graph.
Here's the shape_env info about s0*s1 node,

(Pdb) node
s0*s1
(Pdb) shape_env.var_to_val
{s0: 2, s1: 2}
(Pdb) shape_env.var_to_range
{s0: VR[2, 10], s1: VR[2, 10]}

I would like to get the ValueRanges for s0*s1 directly. This is currently working for us

(Pdb) node = dim.node # dim is `SymInt`
(Pdb) expr = node.expr
(Pdb) var_range  = shape_env.bound_sympy(expr)
(Pdb) var_range
VR[4, 100]
(Pdb) var_val = expr.xreplace(shape_env.var_to_val)
(Pdb) var_val
4

We use var_to_val as the optimal (or general) value that the variable has and var_range to be the (min, max) values it can take.

So is shape_env.bound_sympy and expr.xreplace(shape_env.var_to_val) correct way to compute ValueRanges for such SymInt nodes such as s0*s1 ? Are there any other recommendations ? Thanks

cc: @ezyang @avikchaudhuri @angelayi

Versions

[pip3] numpy==1.26.4
[pip3] pytorch-triton==3.0.0+45fff310c8
[pip3] torch==2.4.0.dev20240610+cu121
[pip3] torch-tensorrt==2.4.0.dev0+a8a079715
[pip3] torchvision==0.19.0.dev20240610+cu121
[pip3] triton==2.3.1
transformers==4.41.2

cc @ezyang @anijain2305 @chauhang
@peri044 peri044 mentioned this issue Jun 13, 2024
Closed
@eellison
Copy link
Contributor

Currently we do not support symbolic value ranges, although that would be great to add.

cc @lezcano

@eellison eellison added module: dynamic shapes triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Jun 14, 2024
@lezcano
Copy link
Collaborator
lezcano commented Jun 14, 2024

As it stands, it'd be quite a big effort (or straight up impossible) to extend value ranges to return symbols. There are too many if-elses reflecting how the functions act on different values (whether they are increasing, decreasing etc)

@eellison
Copy link
Contributor

@lezcano, maybe it'd be easier within constraints. e.g. here, the inputs are both size like, >= 2. and within limited set of operators. but yea, it's tricky

@peri044
Copy link
Contributor Author
peri044 commented Jun 14, 2024 •
edited
Loading

Thanks for your comments @eellison @lezcano. Curently shape_env.bound_sympy(expr) gives the correct ValueRanges (in this case [4, 100]) and expr.xreplace(shape_env.var_to_val) (in the current scenario = 4) gives the correct value in the context for s0*s1 node. Can we use this ? Are there any unintended consequences of using this ?

@lezcano
Copy link
Collaborator
lezcano commented Jun 14, 2024

What do you mean by "can we use this"?

@eellison nah, as soon as you have a minus you have the same issues. In general it's just a very difficult problem without using external algebraic solvers.

@eellison
Copy link
Contributor

@peri044 if the expression is not monotonically increasing than substituting in the symints will not necessarily give you the upper bound. as @lezcano is pointing out with minus..

@peri044
Copy link
Contributor Author
peri044 commented Jun 14, 2024

What do you mean by "can we use this"?

@eellison nah, as soon as you have a minus you have the same issues. In general it's just a very difficult problem without using external algebraic solvers.

Sorry if I was unclear. shape_env.bound_sympy(expr) seems to be computing ValueRanges correctly. Here expr is s0*s1 and both s0 and s1 have ValueRanges to be {s0: VR[2, 10], s1: VR[2, 10]}. Hence s0*s1 value range should be [4, 100] which is the output of shape_env.bound_sympy(expr). As per the documentation, this seems to be the function to compute any sym expressions such as s0*s1.
I wanted to know if this is correct approach to solve it (goal is to get a ValueRanges for s0*s1 and in general any sym expressions).

@lezcano
Copy link
Collaborator
lezcano commented Jun 14, 2024

for that, yes, that shoñuld give you an upper bound on the value ranges of any expression. Elias thought you were asking to get bounds on an expression in terms of its symbolic values, which we don't support.

In general, to ask questions, we ask people to use the forums https://discuss.pytorch.org/

Closing as not an issue.

@lezcano lezcano closed this as completed Jun 14, 2024
@peri044
Copy link
Contributor Author
peri044 commented Jun 14, 2024

@peri044 if the expression is not monotonically increasing than substituting in the symints will not necessarily give you the upper bound. as @lezcano is pointing out with minus..

Sorry I missed this comment (about the same time when I posted mine). Thank you both for your help.

@peri044 peri044 mentioned this issue Jun 14, 2024
Merged
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
module: dynamic shapes oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@lezcano @peri044 @eellison @soulitzer
0