8000 Add tests for Pyodide build with Pixi and Node.js by IvanIsCoding · Pull Request #1450 · Qiskit/rustworkx · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add tests for Pyodide build with Pixi and Node.js #1450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
7bdeaff
Pyodide: force only one thread
IvanIsCoding May 10, 2025
7ce8078
Add .c 8000 argo/config.toml
IvanIsCoding May 11, 2025
b440e6b
Unstable feature that makes rayon work
IvanIsCoding May 11, 2025
e70b263
Newline
IvanIsCoding May 11, 2025
bf15e3b
Merge remote-tracking branch 'upstream/main' into pyodide-rayon
IvanIsCoding May 11, 2025
76f5dff
Document pyodide
IvanIsCoding May 11, 2025
4b17893
Add release note for pyodide
IvanIsCoding May 11, 2025
0e3b0ce
Fix separator
IvanIsCoding May 11, 2025
3527a6f
Consolidate the bare minimum of flags
IvanIsCoding May 12, 2025
72ceb1a
Add emscripten-wasm-eh flag
IvanIsCoding May 12, 2025
65d4eb8
Use pixi to have a reproducible pyodide build
IvanIsCoding May 13, 2025
8c4ed7b
Add GitHub Actions to execute
IvanIsCoding May 13, 2025
d99c029
Fix main.yml syntax
IvanIsCoding May 13, 2025
615d5bd
Specify pyproject.toml
IvanIsCoding May 13, 2025
32e44cf
Actually download repo
IvanIsCoding May 14, 2025
ad1f19d
Add pyodide build docs
IvanIsCoding May 14, 2025
901c674
Minor stuff
IvanIsCoding May 14, 2025
92b8910
Minor update to CONTRIBUTING.md
IvanIsCoding May 14, 2025
25a5763
Add very first smoke test
IvanIsCoding May 14, 2025
a44bcfc
Trying to make local test happen
IvanIsCoding May 14, 2025
2bce487
More smoke test
IvanIsCoding May 14, 2025
d303f91
Use unittest module!?
IvanIsCoding May 14, 2025
c03fc6a
Add Python smoke test file separately
IvanIsCoding May 15, 2025
1ca1317
Handle dirname correctly
IvanIsCoding May 15, 2025
d308579
Smoke test works now
IvanIsCoding May 15, 2025
765ba1e
Actually catch failures
IvanIsCoding May 15, 2025
b63e334
Local imports only
IvanIsCoding May 15, 2025
f393309
Add pixi tasks
IvanIsCoding May 15, 2025
e2bad50
It works!
IvanIsCoding May 15, 2025
b8c2f58
Throw error if there are multiple wheels
IvanIsCoding May 15, 2025
588f0d0
Update CONTRIBUTING.md to explain how to run tests
IvanIsCoding May 15, 2025
233ffe6
Minor improvements
IvanIsCoding May 15, 2025
7d42387
Even more minor fixes
IvanIsCoding May 15, 2025
d81928e
Minor grammar changes
IvanIsCoding May 15, 2025
fba5351
Complete test suite runs
IvanIsCoding May 16, 2025
a66716e
Run complete test suite
IvanIsCoding May 16, 2025
1599f18
Update CONTRIBUTING.md to reflect full test suite
IvanIsCoding May 16, 2025
ec111ad
Black
IvanIsCoding May 16, 2025
289329c
Minor correction
IvanIsCoding May 16, 2025
826fbe7
Merge branch 'main' into pyodide-rayon
IvanIsCoding May 17, 2025
61383bf
Merge branch 'pyodide-rayon' into pixi-pyodide
IvanIsCoding May 17, 2025
3fe5aa1
Merge branch 'pixi-pyodide' into test-pyodide
IvanIsCoding May 17, 2025
65d8a5e
smoke test -> unit test
IvanIsCoding May 17, 2025
b6fcd9a
Merge branch 'main' into pyodide-rayon
IvanIsCoding May 29, 2025
aa34512
Solve nit
IvanIsCoding May 29, 2025
854e672
Merge branch 'pyodide-rayon' into pixi-pyodide
IvanIsCoding May 29, 2025
b0ba40e
Merge branch 'pixi-pyodide' into test-pyodide
IvanIsCoding May 29, 2025
1a04b00
Merge branch 'main' into pyodide-rayon
IvanIsCoding May 29, 2025
b0042ff
Merge branch 'pyodide-rayon' into pixi-pyodide
IvanIsCoding May 30, 2025
d7bb215
Bump pyodide to 0.27.6 and pyodide-build to 0.30.4
IvanIsCoding May 30, 2025
d6b1faa
Update to 0.27.6 and simplify setup
IvanIsCoding May 30, 2025
8b2a5c3
Merge branch 'pixi-pyodide' into test-pyodide
IvanIsCoding May 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,14 @@ rustflags = [
"-C", "link-arg=-undefined",
"-C", "link-arg=dynamic_lookup",
]

[target.wasm32-unknown-emscripten]
rustflags = [
"-C", "target-feature=+atomics,+bulk-memory,+mutable-globals",
"-C", "link-arg=-sSIDE_MODULE=2",
"-C", "link-arg=-sWASM_BIGINT",
"-Z", "emscripten-wasm-eh",
]

[unstable]
build-std = ["std", "panic_abort"]
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
docs/* linguist-documentation
releasenotes/* linguist-documentation
# SCM syntax highlighting & preventing 3-way merges
pixi.lock merge=binary linguist-language=YAML linguist-generated=true
13 changes: 13 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -198,3 +198,16 @@ jobs:
with:
name: html_docs
path: docs/build/html
pyodide-build:
if: github.repository_owner == 'Qiskit'
needs: [tests]
name: Pyodide Build + Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: prefix-dev/setup-pixi@v0.8.8
with:
manifest-path: pyproject.toml
pixi-version: v0.46.0
environments: default
- run: pixi run build_pyodide_and_test
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,15 @@ rustworkx-core/Cargo.lock
**/.DS_Store
venv/
.python-version

# pixi environments
.pixi
*.egg-info

# pyodide xbuild
.xbuildenv

# Node.js
node_modules/
jspm_packages/
.npm
74 changes: 74 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -560,6 +560,80 @@ has the necessary approvals and is tagged as `automerge` unless it has a merge
conflict or has a failed CI run. Doing so will just waste CI resources and
delay everything from merging, including your PR.

### Pyodide Support

`rustworkx` has experimental support for Pyodide, the Python distribution for
the browser.

Because building for Pyodide is a more involved process, we use [Pixi](https://pixi.sh/latest/)
to manage the dependencies of the build. Currently, the scripts work only for Linux x86-64.
It is also possible to run the build on Windows with Windows Subsystem for Linux (WSL). At
the moment, aarch64 platforms like macOS cannot run the script.

Please refer to the [Pixi](https://pixi.sh/latest/) page for the latest instructions on how
to install Pixi. Once installed, there's a single command that needs to be run.

#### Building for Pyodide

At the root of the directory, simply run:

```bash
pixi run build_pyodide
```

This will create a separate environment with all of the required toolchains. At the end,
a Pyodide wheel will be available in the `dist` folder if the build is successful.

#### Testing Pyodide Wheels

After running `pixi run build_pyodide`, we can run the test suite with:

```bash
pixi run pyodide_test
```

It is also possible to run the tests with a specific wheel via an **absolute path**:
```
pixi run pyodide_test $ABSOLUTE_PATH_TO_WHEEL
```

> [!WARNING]
> If there are multiple Pyodide wheels in the dist/ folder, the test will fail
> to prevent anyone from testing an outdated wheel. This scenario can happen when
> we update the rustworkx version, update the minimum Python ABI, or update Pyodide. To fix,
> clean the directory with `rm -r dist` and re-run `pixi run build_pyodide`

> [!TIP]
> You can build and test with a single command with `pixi run build_pyodide_and_test`.

#### Updating `pyodide-build` and dependencies

All the dependencies for the Pyodide build are listed under `[tool.pixi.dependencies]`. To find a set
of versions that works, visit the [pyodide-cross-build-environments.json](https://github.com/pyodide/pyodide/blob/main/pyodide-cross-build-environments.json) file in the `pyodide` repository.

We'll need to align the [Emscripten](https://anaconda.org/conda-forge/emscripten) version from `conda-forge` with one
of the public releases. Then, we pick a `pyodide-build` version higher than the required build version and the equivalent Python
version also specified in the cross build environments. Lastly, update `[tool.pixi.tasks.install_xbuildenv]` to install
the selected version of Pyodide.

We need to pin the Rust compiler. To find an appropriate Rust compiler version, run:

```bash
pixi shell
pyodide config list
```

This will output a list including `rust_toolchain`. Currently, `pyodide-build` requires Rust Nightly. Because conda-forge
only provides stable releases, we'll need to map a nightly version to a stable version. Some repositories like [oxalica/rust-overlay]([oxalica/rust-overlay](https://github.com/oxalica/rust-overlay/tree/master/manifests/nightly/)) contain a list of the nightly releases. For example, `nightly-2025-02-01`
maps roughly to `1.86`. If that version was not yet stable, we could try picking `1.85` as well.

After updating the versions in `[tool.pixi.dependencies]`, run `pixi lock` which will update `pixi.lock`. Onwards, all builds
will use the same environment. As long as `pixi run build_pyodide` passes locally or on CI it should keep compiling and building.

Lastly, remember to update the Pyodide version in `tests/pyodide_tests`. Change the version in `tests/pyodide_tests/package.json`
and run `pixi run install_pyodide_test_env` to generate the new lockfile. If you forget to update the Pyodide version, the tests
will fail.

### Stable Branch Policy and Backporting

The stable branch is intended to be a safe source of fixes for high-impact bugs,
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -75,5 +75,5 @@ default-features = false
features = ["multi_thread"]

[profile.release]
lto = 'fat'
lto = "fat"
codegen-units = 1
16 changes: 16 additions & 0 deletions docs/source/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,10 @@ source.
- i686 or x86_64
- :ref:`tier-2` (Python < 3.10), :ref:`tier-3` (Python >= 3.10)
-
* - Pyodide
- WASM (Emscripten)
- :ref:`tier-experimental`
-


.. _manylinux 2014: https://peps.python.org/pep-0599/>
Expand Down Expand Up @@ -166,6 +170,18 @@ functioning Python environment and may require a C/C++ compiler or additional
programs to build dependencies from source as part of the installation process.
Support for these platforms are best effort only.

.. _tier-experimental:

Tier Experimental
-----------------

Tier Experimental platforms are not tested upstream as part of the development process.
Pre-compiled binaries are built by the external community in separate repositories. Not all of rustworkx might compile for
platforms of this tier and features can be removed. Often, platforms in this tier use unstable features
from the Rust compiler and might break at any time. Support for these platforms are best effort only.

Currently, the only platform in this tier is Pyodide, which is a port of Python that can run in the browser and on Node.js.

Using rustworkx
===============

Expand Down
46 changes: 46 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "rustworkx-pyodide",
"dependencies": {
"pyodide": "^0.27.6"
}
}
Loading
0