Make `rustworkx` build and run with pyiodide #1447

IvanIsCoding · 2025-05-10T15:12:51Z

Closes #703

This adds support for Pyodide. We list Pyodie in a brand new tier that is Tier Experimental. Currently, we build with pyoide-build==0.30.2 using the following flags:

export "RUSTFLAGS="-C target-feature=+atomics,+bulk-memory,+mutable-globals -C link-arg=-sSIDE_MODULE=2 -C link-arg=-sWASM_BIGINT -Z emscripten-wasm-eh"
pyodide build

This requires a Rust with nightly-2025-02-01 toolchain and emsdk 4.0.8. I tested with Pyodide 0.28.1a which maps to Python 3.13. Because we depend on so much unstable stuff, this is not ready yet for our CI. I need to figure out a way of adding it.

Related: pyodide/pyodide-recipes#90

coveralls · 2025-05-10T15:26:13Z

Pull Request Test Coverage Report for Build 15334803033

Details

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 95.236%

Totals
Change from base Build 15333628866:	0.0%
Covered Lines:	18733
Relevant Lines:	19670

💛 - Coveralls

IvanIsCoding · 2025-05-10T15:38:39Z

So I tried following: https://pyodide.org/en/stable/usage/building-and-testing-packages.html

I had to install emsdk 4.0.8., pyodide-build==0.30.1, rustc 1.88.0-nightly (00095b3da 2025-04-03)

I got this error: https://pastebin.com/TaT892sU. So yeah, I'd like to keep this build outside of our repository but at least add the code such that when emscripten-forge builds we run.

IvanIsCoding · 2025-05-10T15:39:44Z

@sorin-bolos how did you setup the SDK to build the version from your PR? I might need some help here

IvanIsCoding · 2025-05-10T21:37:59Z

Maybe I should try using the version pinned here: https://github.com/pyodide/pyodide-build/blob/7126c5f1bd56ed7f40ce29cbcbe588267a42434c/pyodide_build/config.py#L240C24-L240C42

IvanIsCoding · 2025-05-10T21:50:32Z

Somehow I got Successfully built /home/ivan/Projects/rustworkx-dev/dist/rustworkx-0.17.0-cp39-abi3-emscripten_4_0_6_wasm32.whl, but only when running with RUSTFLAGS="-v" pyodide build. I will test the wheel.

IvanIsCoding · 2025-05-10T21:57:59Z

Well, I can pip install using the pyodide environment but a simple import rustworkx crashes:

(.venv-pyodide) ➜  tests git:(more-uv-features) python
which: no grealpath in (/home/ivan/Projects/.venv-pyodide/bin:/home/ivan/.local/bin:/usr/share/Modules/bin:/home/ivan/.cargo/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:/home/ivan/.dotnet/tools:/home/ivan/.dotnet/tools)
Python 3.13.2 (main, Apr  3 2025, 07:34:46) [Clang 21.0.0git (https:/github.com/llvm/llvm-project 4775e6d9099467df9363e1a3cd on emscripten
Type "help", "copyright", "credits" or "license" for more information.
warning: can't use pyrepl: No module named 'msvcrt'
>>> import rustworkx
Pyodide has suffered a fatal error. Please report this to the Pyodide maintainers.
The cause of the fatal error was:
Error: Dynamic linking error: cannot resolve symbol invoke_ii
    at stubs.<computed> (/home/ivan/.cache/.pyodide-xbuildenv-0.30.2/0.28.0a1/xbuildenv/pyodide-root/dist/pyodide.asm.js:9:19341)
    at wasm://wasm/0122615e:wasm-function[3874]:0x3ca8c6
    at wasm://wasm/020b656e:wasm-function[4078]:0x2a30bc
    at wasm://wasm/020b656e:wasm-function[4054]:0x2a0f12
    at wasm://wasm/020b656e:wasm-function[4070]:0x2a28c5
    at wasm://wasm/020b656e:wasm-function[2181]:0x1ad753
    at wasm://wasm/020b656e:wasm-function[1119]:0x16345c
    at wasm://wasm/020b656e:wasm-function[1121]:0x163668
    at wasm://wasm/020b656e:wasm-function[1122]:0x1636e6
    at wasm://wasm/020b656e:wasm-function[3519]:0x25a95b {
  pyodide_fatal_error: true
}
Stack (most recent call first):
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1320 in create_module
  File "<frozen importlib._bootstrap>", line 813 in module_from_spec
  File "<frozen importlib._bootstrap>", line 921 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "/home/ivan/Projects/.venv-pyodide/lib/python3.13/site-packages/rustworkx/__init__.py", line 14 in <module>
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1026 in exec_module
  File "<frozen importlib._bootstrap>", line 935 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "<stdin>-0", line 1 in <module>
  File "/lib/python313.zip/_pyrepl/main.py", line 30 in interactive_console
  File "/lib/python313.zip/_pyrepl/__main__.py", line 6 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
Error: Dynamic linking error: cannot resolve symbol invoke_ii
    at stubs.<computed> (/home/ivan/.cache/.pyodide-xbuildenv-0.30.2/0.28.0a1/xbuildenv/pyodide-root/dist/pyodide.asm.js:9:19341)
    at wasm://wasm/0122615e:wasm-function[3874]:0x3ca8c6
    at wasm://wasm/020b656e:wasm-function[4078]:0x2a30bc
    at wasm://wasm/020b656e:wasm-function[4054]:0x2a0f12
    at wasm://wasm/020b656e:wasm-function[4070]:0x2a28c5
    at wasm://wasm/020b656e:wasm-function[2181]:0x1ad753
    at wasm://wasm/020b656e:wasm-function[1119]:0x16345c
    at wasm://wasm/020b656e:wasm-function[1121]:0x163668
    at wasm://wasm/020b656e:wasm-function[1122]:0x1636e6
    at wasm://wasm/020b656e:wasm-function[3519]:0x25a95b {
  pyodide_fatal_error: true
}

IvanIsCoding · 2025-05-10T22:30:33Z

I think setting verbose on the previous comment erased the flags from pyodide build so we are back at step one.

IvanIsCoding · 2025-05-11T03:21:52Z

Ok, we definetely build. Any code that calls Rayon fails, but:

https://gist.github.com/IvanIsCoding/8bb97d428169e073af4afa598e1d2932 has the code of a index.html page that calls Floyd-Warshall
Download index.html from the gist and https://github.com/IvanIsCoding/rustworkx/releases/download/v0.17.0-alpha-pyodide/rustworkx-0.17.0-cp39-abi3-pyodide_2025_0_wasm32.whl, put the files in the same folder
Run python -m http.server --directory . in the same folder to serve a very simple HTTP server
Open http://localhost:8080

It works! The issues inside the pyodide env seem to only affect Node.js, inside a browser it works like a charm:

IvanIsCoding · 2025-05-11T04:32:40Z

After telling the nightly compiler to rebuild std, I built https://github.com/IvanIsCoding/rustworkx/releases/download/v0.17.0-alpha2-pyodide/rustworkx-0.17.0-cp39-abi3-pyodide_2025_0_wasm32.whl which seems to be working.

I will try to see if I figure out how to run our unit tests to confirm things work. But this is shockingly good.

IvanIsCoding · 2025-05-11T15:56:59Z

.cargo/config.toml

@@ -9,3 +9,16 @@ rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
 ]
+
+[target.wasm32-unknown-emscripten]
+rustflags = [


Note: it seems like pyodide-build overrides these, so I will open an issue to talk to them and check for alternatives. In the meantime, this is helpful for reference and when we compile outside pyodide-build

It also is useful for rustworkx-core and using rustworkx without python in wasm.

IvanIsCoding · 2025-05-11T16:21:35Z

I have no clue how to include this on CI or even if we should, but there more I test the more things seem to work. I will open an issue on the pyodide-build repository and make a recipe on the pyodide-recipe repository.

mtreinish

Overall this looks fine to me, Just one nit inline. I'd like to try to find time soon to pull this locally and test out wasm and pyodide before approving though just to validate all the new options.

mtreinish · 2025-05-29T11:23:16Z

.cargo/config.toml

@@ -9,3 +9,16 @@ rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
 ]
+
+[target.wasm32-unknown-emscripten]
+rustflags = [


It also is useful for rustworkx-core and using rustworkx without python in wasm.

releasenotes/notes/pyodide-support-2172f2313b06f10d.yaml

IvanIsCoding · 2025-05-29T12:45:25Z

Overall this looks fine to me, Just one nit inline. I'd like to try to find time soon to pull this locally and test out wasm and pyodide before approving though just to validate all the new options.

Some minor notes about Rust WASM. There are three Rust targets! We could support more, but here are the caveats:

wasm32-unknown-unknown: easiest target to build for rustworkx-core. impossible to build rustworkx because of Python. cargo doesn't have a default runner for this so it's hard to test.
wasm32-unknown-emscripten: used by Pyodide. emscripten goes from LLVM -> WASM from what I understood, so there needs to be some coordination between emscripten and rustc versions. I higly recommend using Pixi from Add tests for Pyodide build with Pixi and Node.js #1450 to build it
wasm32-wasip1: in theory Python supports this, but I have yet to see someone building and publishing for WASI

After #1450 lands, I think we could test wasm32-unknown-unknown for rustworkx-core with https://rustwasm.github.io/wasm-pack/

Pyodide: force only one thread

7bdeaff

IvanIsCoding added 2 commits May 10, 2025 23:24

Add .cargo/config.toml

7ce8078

Unstable feature that makes rayon work

b440e6b

IvanIsCoding added 2 commits May 11, 2025 11:45

Newline

e70b263

Merge remote-tracking branch 'upstream/main' into pyodide-rayon

bf15e3b

IvanIsCoding commented May 11, 2025

View reviewed changes

IvanIsCoding added 2 commits May 11, 2025 12:12

Document pyodide

76f5dff

Add release note for pyodide

4b17893

IvanIsCoding marked this pull request as ready for review May 11, 2025 16:19

8000

IvanIsCoding changed the title ~~[WIP] Make rustworkx build and run with pyiodide~~ Make rustworkx build and run with pyiodide May 11, 2025

IvanIsCoding added the on hold label May 11, 2025

This was referenced May 11, 2025

Add rustworkx alpha recipe pyodide/pyodide-recipes#90

Merged

Wheel built with pyodide-build works on the browser but fails in Python from pyodide venv pyodide/pyodide-build#201

Closed

IvanIsCoding added 3 commits May 11, 2025 16:01

Fix separator

0e3b0ce

Consolidate the bare minimum of flags

3527a6f

Add emscripten-wasm-eh flag

72ceb1a

IvanIsCoding removed the on hold label May 13, 2025

IvanIsCoding requested a review from mtreinish May 13, 2025 04:03

This was referenced May 13, 2025

Add Pixi workflow to verify Pyodide build works #1449

Open

Add tests for Pyodide build with Pixi and Node.js #1450

Open

Merge branch 'main' into pyodide-rayon

826fbe7

IvanIsCoding added this to the 0.17.0 milestone May 17, 2025

IvanIsCoding mentioned this pull request May 17, 2025

Add rustworkx alpha emscripten-forge/recipes#2327

Merged

mtreinish reviewed May 29, 2025

View reviewed changes

IvanIsCoding and others added 2 commits May 29, 2025 08:21

Merge branch 'main' into pyodide-rayon

b6fcd9a

Solve nit

aa34512

Merge branch 'main' into pyodide-rayon

1a04b00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Make `rustworkx` build and run with pyiodide #1447

Make `rustworkx` build and run with pyiodide #1447

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Make rustworkx build and run with pyiodide #1447

Are you sure you want to change the base?

Make rustworkx build and run with pyiodide #1447

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

Pull Request Test Coverage Report for Build 15334803033

Details

💛 - Coveralls

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Make `rustworkx` build and run with pyiodide #1447

Make `rustworkx` build and run with pyiodide #1447