Fix self bootstrap without Rust installed #2653
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
What's the difference between the two versions, why does this version work but the old one doesn't? |
|
when running at that point it does not have Line 52 in ea5adfc |
|
To reproduce #2642 docker run --rm -it -v $(pwd):/src python:3.13 bash
root@6b009d208709:/# python3 -m venv venv
root@6b009d208709:/# venv/bin/pip install /src -v
Using pip 25.1.1 from /venv/lib/python3.13/site-packages/pip (python 3.13)
Processing /src
Running command pip subprocess to install build dependencies
Using pip 25.1.1 from /venv/lib/python3.13/site-packages/pip (python 3.13)
Ignoring tomli: markers 'python_version < "3.11"' don't match your environment
Collecting setuptools
Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/a3/dc/17031897dae0efacfea57dfd3a82fdd2a2aeb58e0ff71b77b87e44edc772/setuptools-80.9.0-py3-none-any.whl.metadata
Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
Collecting setuptools-rust>=1.11.0
Obtaining dependency information for setuptools-rust>=1.11.0 from https://files.pythonhosted.org/packages/b3/01/37e1376f80578882e4f2d451f57d1fb42a599832057a123f57d9f26395c8/setuptools_rust-1.11.1-py3-none-any.whl.metadata
Downloading setuptools_rust-1.11.1-py3-none-any.whl.metadata (9.6 kB)
Collecting semantic_version<3,>=2.8.2 (from setuptools-rust>=1.11.0)
Obtaining dependency information for semantic_version<3,>=2.8.2 from https://files.pythonhosted.org/packages/6a/23/8146aad7d88f4fcb3a6218f41a60f6c2d4e3a72de72da1825dc7c8f7877c/semantic_version-2.10.0-py2.py3-none-any.whl.metadata
Downloading semantic_version-2.10.0-py2.py3-none-any.whl.metadata (9.7 kB)
Downloading setuptools-80.9.0-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 45.5 MB/s eta 0:00:00
Downloading setuptools_rust-1.11.1-py3-none-any.whl (28 kB)
Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Installing collected packages: setuptools, semantic_version, setuptools-rust
Successfully installed semantic_version-2.10.0 setuptools-80.9.0 setuptools-rust-1.11.1
Installing build dependencies ... done
Running command Getting requirements to build wheel
Traceback (most recent call last):
File "/venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
main()
~~~~^^
File "/venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
return hook(config_settings)
File "/src/maturin/bootstrap.py", line 29, in get_requires_for_build_wheel
reqs = _orig_get_requires_for_build_wheel()
File "/tmp/pip-build-env-bqzb5qn5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 331, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-bqzb5qn5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 301, in _get_build_requires
self.run_setup()
~~~~~~~~~~~~~~^^
File "/tmp/pip-build-env-bqzb5qn5/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 317, in run_setup
exec(code, locals())
~~~~^^^^^^^^^^^^^^^^
File "<string>", line 52, in <module>
ModuleNotFoundError: No module named 'puccinialin'
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /venv/bin/python3 /venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /t
8000
mp/tmp0z4d0u7m
cwd: /src
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.on this branch docker run --rm -it -v $(pwd):/src python:3.13 bash
root@464e2f0f921a:/# python3 -m venv venv
root@464e2f0f921a:/# venv/bin/pip install /src -v
Using pip 25.1.1 from /venv/lib/python3.13/site-packages/pip (python 3.13)
Processing /src
Running command pip subprocess to install build dependencies
Using pip 25.1.1 from /venv/lib/python3.13/site-packages/pip (python 3.13)
Ignoring tomli: markers 'python_version < "3.11"' don't match your environment
Collecting setuptools
Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/a3/dc/17031897dae0efacfea57dfd3a82fdd2a2aeb58e0ff71b77b87e44edc772/setuptools-80.9.0-py3-none-any.whl.metadata
Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
Collecting setuptools-rust>=1.11.0
Obtaining dependency information for setuptools-rust>=1.11.0 from https://files.pythonhosted.org/packages/b3/01/37e1376f80578882e4f2d451f57d1fb42a599832057a123f57d9f26395c8/setuptools_rust-1.11.1-py3-none-any.whl.metadata
Downloading setuptools_rust-1.11.1-py3-none-any.whl.metadata (9.6 kB) Collecting semantic_version<3,>=2.8.2 (from setuptools-rust>=1.11.0) Obtaining dependency information for semantic_version<3,>=2.8.2 from https://files.pythonhosted.org/packages/6a/23/8146aad7d88f4fcb3a6218f41a60f6c2d4e3a72de72da1825dc7c8f7877c/semantic_version-2.10.0-py2.py3-none-any.whl.metadata
Downloading semantic_version-2.10.0-py2.py3-none-any.whl.metadata (9.7 kB)
Downloading setuptools-80.9.0-py3-none-any.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 48.0 MB/s eta 0:00:00 Downloading setuptools_rust-1.11.1-py3-none-any.whl (28 kB) Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Installing collected packages: setuptools, semantic_version, setuptools-rust
Successfully installed semantic_version-2.10.0 setuptools-80.9.0 setuptools-rust-1.11.1
Installing build dependencies ... done
Running command Getting requirements to build wheel
Getting requirements to build wheel ... done
Running command pip subprocess to install backend dependencies
Using pip 25.1.1 from /venv/lib/python3.13/site-packages/pip (python 3.13)
Collecting puccinialin<0.2,>=0.1
Obtaining dependency information for puccinialin<0.2,>=0.1 from https://files.pythonhosted.org/packages/b0/25/91d671da2a6d98e95094f7456754c55d972febcc35de3b6d4e9a9017c445/puccinialin-0.1.5-py3-none-any.whl.metadata
Downloading puccinialin-0.1.5-py3-none-any.whl.metadata (4.3 kB)
Collecting httpx<0.29,>=0.28.1 (from puccinialin<0.2,>=0.1)
Obtaining dependency information for httpx<0.29,>=0.28.1 from https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl.metadata
Downloading httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB)
Collecting platformdirs<5,>=4.3.6 (from puccinialin<0.2,>=0.1)
Obtaining dependency information for platformdirs<5,>=4.3.6 from https://files.pythonhosted.org/packages/fe/39/979e8e21520d4e47a0bbe349e2713c0aac6f3d853d0e5b34d76206c439aa/platformdirs-4.3.8-py3-none-any.whl.metadata
Downloading platformdirs-4.3.8-py3-none-any.whl.metadata (12 kB)
Collecting tqdm<5,>=4.67.1 (from puccinialin<0.2,>=0.1)
Obtaining dependency information for tqdm<5,>=4.67.1 from https://files.pythonhosted.org/packages/d0/30/dc54f88dd4a2b5dc8a0279bdd7270e735851848b762aeb1c1184ed1f6b14/tqdm-4.67.1-py3-none-any.whl.metadata
Downloading tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
Collecting anyio (from httpx<0.29,>=0.28.1->puccinialin<0.2,>=0.1)
Obtaining dependency information for anyio from https://files.pythonhosted.org/packages/a1/ee/48ca1a7c89ffec8b6a0c5d02b89c305671d5ffd8d3c94acf8b8c408575bb/anyio-4.9.0-py3-none-any.whl.metadata
Downloading anyio-4.9.0-py3-none-any.whl.metadata (4.7 kB)
Collecting certifi (from httpx<0.29,>=0.28.1->puccinialin<0.2,>=0.1)
Obtaining dependency information for certifi from https://files.pythonhosted.org/packages/84/ae/320161bd181fc06471eed047ecce67b693fd7515b16d495d8932db763426/certifi-2025.6.15-py3-none-any.whl.metadata
Downloading certifi-2025.6.15-py3-none-any.whl.metadata (2.4 kB)
Collecting httpcore==1.* (from httpx<0.29,>=0.28.1->puccinialin<0.2,>=0.1)
Obtaining dependency information for httpcore==1.* from https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl.metadata
Downloading httpcore-1.0.9-py3-none-any.whl.metadata (21 kB)
Collecting idna (from httpx<0.29,>=0.28.1->puccinialin<0.2,>=0.1)
Obtaining dependency information for idna from https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl.metadata
Downloading idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting h11>=0.16 (from httpcore==1.*->httpx<0.29,>=0.28.1->puccinialin<0.2,>=0.1)
Obtaining dependency information for h11>=0.16 from https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl.metadata
Downloading h11-0.16.0-py3-none-any.whl.metadata (8.3 kB)
Collecting sniffio>=1.1 (from anyio->httpx<0.29,>=0.28.1->puccinialin<0.2,>=0.1)
Obtaining dependency information for sniffio>=1.1 from https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl.metadata
Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Downloading puccinialin-0.1.5-py3-none-any.whl (12 kB)
Downloading httpx-0.28.1-py3-none-any.whl (73 kB)
Downloading httpcore-1.0.9-py3-none-any.whl (78 kB)
Downloading platformdirs-4.3.8-py3-none-any.whl (18 kB)
Downloading tqdm-4.67.1-py3-none-any.whl (78 kB)
Downloading h11-0.16.0-py3-none-any.whl (37 kB)
Downloading anyio-4.9.0-py3-none-any.whl (100 kB)
Downloading idna-3.10-py3-none-any.whl (70 kB)
Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)
Downloading certifi-2025.6.15-py3-none-any.whl (157 kB)
Installing collected packages: tqdm, sniffio, platformdirs, idna, h11, certifi, httpcore, anyio, httpx, puccinialin
Creating /tmp/pip-build-env-j4y8e0w2/normal/bin
changing mode of /tmp/pip-build-env-j4y8e0w2/normal/bin/tqdm to 755
changing mode of /tmp/pip-build-env-j4y8e0w2/normal/bin/httpx to 755
changing mode of /tmp/pip-build-env-j4y8e0w2/normal/bin/puccinialize to 755
Successfully installed anyio-4.9.0 certifi-2025.6.15 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 idna-3.10 platformdirs-4.3.8 puccinialin-0.1.5 sniffio-1.3.1 tqdm-4.67.1
Installing backend dependencies ... done
Running command Preparing metadata (pyproject.toml)
Python reports SOABI: cpython-313-aarch64-linux-gnu
Computed rustc target triple: aarch64-unknown-linux-gnu
Installation directory: /root/.cache/puccinialin
Downloading rustup-init from https://static.rust-lang.org/rustup/dist/aarch64-unknown-linux-gnu/rustup-init
Rust not found, installing into a temporary directory
Downloading rustup-init: 0%| | 0.00/19.3M [00:00<?, ?B/s]
Downloading rustup-init: 3%|▎ | 590k/19.3M [00:00<00:03, 5.34MB/s]
Downloading rustup-init: 19%|█▉ | 3.67M/19.3M [00:00<00:00, 19.6MB/s]
Downloading rustup-init: 59%|█████▉ | 11.5M/19.3M [00:00<00:00, 45.6MB/s]
Downloading rustup-init: 90%|████████▉ | 17.4M/19.3M [00:00<00:00, 46.9MB/s]
Downloading rustup-init: 100%|██████████| 19.3M/19.3M [00:00<00:00, 40.7MB/s]
Installing rust to /root/.cache/puccinialin/rustup
info: profile set to 'minimal'
info: default host triple is aarch64-unknown-linux-gnu
info: syncing channel updates for 'stable-aarch64-unknown-linux-gnu'
info: latest update on 2025-05-15, rust version 1.87.0 (17067e9ac 2025-05-09)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
^C Preparing metadata (pyproject.toml) ... canceled
ERROR: Operation cancelled by user |
2 tasks
|
Oh I see! Can we instead skip calling into setuptools in these hooks? afaik setuptools would never return anything there anyway, and the code works for me with pip. I prefer that over using exceptions for control flow, and I prefer having less reliance on setuptools (it has a lot of unexpected behaviors). """Support installing rust before compiling (bootstrapping) maturin.
Installing a package that uses maturin as build backend on a platform without maturin
binaries, we install rust in a cache directory if the user doesn't have a rust
installation already. Since this bootstrapping requires more dependencies but is only
required if rust is missing, we check if cargo is present before requesting those
dependencies.
https://setuptools.pypa.io/en/stable/build_meta.html#dynamic-build-dependencies-and-other-build-meta-tweaks
"""
from __future__ import annotations
import os
import shutil
from typing import Any
# noinspection PyUnresolvedReferences
from setuptools.build_meta import * # noqa:F403
from setuptools.build_meta import (
get_requires_for_build_sdist as _orig_get_requires_for_build_sdist,
)
from setuptools.build_meta import (
get_requires_for_build_wheel as _orig_get_requires_for_build_wheel,
)
def get_requires_for_build_wheel(config_settings: dict[str, Any] | None = None) -> list[str]:
if not os.environ.get("MATURIN_NO_INSTALL_RUST") and not shutil.which("cargo"):
return ["puccinialin>=0.1,<0.2"]
return []
def get_requires_for_build_sdist(config_settings: dict[str, Any] | None = None) -> list[str]:
if not os.environ.get("MATURIN_NO_INSTALL_RUST") and not shutil.which("cargo"):
return ["puccinialin>=0.1,<0.2"]
return [] |
|
That breaks part of #2612, we are already using |
df2e5ca to
dc50ed1
Compare
|
I've changed it, let's deal with |
|
Thanks! If we want to ensure that we don't wheel in addition to setuptools (as mentioned in #2612), we can put a lower bound on setuptools. |
konstin
approved these changes
Jun 18, 2025
|
Any plans to release a quick fix? The tools that depend on it to be built it are broken. |
|
@messense released this as v1.9.0 |
This was referenced Jun 24, 2025
bmwiedemann
pushed a commit
to bmwiedemann/openSUSE
that referenced
this pull request
Jun 24, 2025
https://build.opensuse.org/request/show/1288127 by user mia + anag_factory - Update to 1.9.0 * Update pyproject-toml to 0.13.5 gh#PyO3/maturin#2645 * Fix clippy lints gh#PyO3/maturin#2648 * ZipWriter requires a compression level of None for the stored method gh#PyO3/maturin#2644 * Implement PEP 639 Support gh#PyO3/maturin#2647 * Don't go through Display for platform tag to policy gh#PyO3/maturin#2652 * Add --compatibility pypi to avoid building for unsupported architectures gh#PyO3/maturin#2650 * Fix self bootstrap without Rust installed gh#PyO3/maturin#2653
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #2642