Error when installing third-party library - mu 1.1.0 alpha - commit f67852f #822
Comments
|
2019-04-30 20:02:25,310 - mu.logic:1162(sync_package_state) INFO: Synchronize package states... |
|
The solution is: /usr/bin/python3 -m pip install pytechbrain --system --target /home/adasiek/.local/share/mu/site-packages I found pypa/pip#3826 when searching and it is working. |
|
Reopening as this could still be an issue for users in a debian based platform. |
|
@abixadamj if you just run |
|
adasiek@abix-edukacja:$ /usr/bin/python3 -m pip install pytechbrain --target /home/adasiek/.local/share/mu/site-packages |
|
Can you try installing each of |
|
|
|
|
Whatever else... |
|
Thank you @abixadamj, I was hoping one of the packages was doing something weird, but it doesn't look that way, problem persist with any pip install with the |
|
Can you check the environmental variables on your system? Specifically anything that starts with For installing packages Mu will fork the same version of Python that has launched Mu, and if it is using the system Python it will use the system pip as well. If the problem is from the debian patch to pip mentioned in pypa/pip#3826, then perhaps we can try using a different pip version. |
|
Based on this it sounds like this is maybe an issue when running Mu on the system Python and not in a virtual environment: https://sources.debian.org/patches/python-pip/18.1-5/set_user_default.patch/ |
|
@carlosperate this appears entirely to do with Debian patching Python in a way that makes using pip inconsistent with any other OS or pip's own documentation. This was, I believe, discussed a few weeks ago on another ticket and/or on the chat forum. My hunch is this is a problem of Debian's making. There is no reliable way to detect if Mu is running on a Debian based distribution, so building in a "special case" for Debian appears to be a can of worms (I'd rather not open). I'm at a loss about how to fix this -- there's not a lot Mu can do, and upstream Debian don't appear to want to change their patching of pip. The best we can suggest is that the user run Mu in a virtualenv (as I do on Ubuntu). 😢 🐮 💔 |
|
Yes, we were also discussing this in #836 (comment), not sure if there was also something in the chat as well, but I moved the conversation here as it makes more sense to have it where people will find it if they search for the error message in the repo.
It looks like it, based on pypa/pip#3826 (comment) and https://sources.debian.org/patches/python-pip/18.1-5/set_user_default.patch/.
Even if there was an easy way to detect if it was debian, it would prefer to find a way around applying platform-specific special cases. However, if the patch was implemented in a way that we could circumvent, like a My concern here is that this is a patch to upstream Debian, and probably present in the Raspbian version as well. This means that when the new Mu .deb installers fo Raspbian and Debian Buster are released, they will use the default python3-pip version with the patch and users will encounter the same problem when installing via
That would have been my recommendation as well if it didn't affect "deb-installed" versions of Mu. There are other options we could consider, like https://github.com/spotify/dh-virtualenv, or to find a way to package or use a different version of pip. Or maybe there is a better way, which is why I think this is worth exploring. |
|
@carlosperate I think this photo, taken at the recent PyCon, pretty much sums up Python packaging... ;-) |
|
OK, today I have 5 minuts free to come back to this issue.
So, I want to propose solution : Create environment variable, maybe MU_PIP_DEBIAN, and check in code, if there will be such env (export MU_PIP_DEBIAN=YES), and then (only then) add --system to pip code in MU-editor This will not breake other systems, and If there will be situation like in me, there will be a way to solve it. What do you think @ntoll ? If you will approve it, I will make PR... |
|
@abixadamj 👍 on finding something that will work. I'd rather things worked the other way around from a logical point of view and were not Debian specific (I'll explain below). In terms of functionality:
I'm partially playing devil's advocate here, so it's an opportunity to encourage some discussion before someone (@abixadamj?) creates a PR with the agreed upon solution. So, thoughts on this would be most welcome..! Let's see what sort of consensus we can arrive at so we can both fix @abixadamj's problem and perhaps add a bit of flexibility to Mu (after all, some folk have mentioned they'd like Mu to use system Python's packages / path too). Thoughts..? :-) |
Unfortunately the Perhaps there is a way to query pip to see if the flag is available and then added if it is? And we could do that independently of the operating system. |
|
Chiming in with 2c of a thought that keeps coming up on my mind whenever a "can't install 3rd party package with Mu" new issue is raised and solutions are sought (and I believe there have been a few, recently). Here are a few facts I suppose are correct, concerning Python mode:
Here's a thought / vision I believe leads to a more robust overall solution that, AFAICT, will not need platform specific "thingies", whichever they are:
Benefits:
Trade-off:
Challenges:
IMPORTANT NOTE: I don't mean to hijack this issue. My intent is the best. But I do believe that once we start going the "if this then do that, else if env var is set but the sun is setting then dance while holding three sharp knives on fire" we'll never see an end to it. I honestly believe that what I propose above is for the better of all users and is the right answer to most (all?) "can't install 3rd party package" issues. I see no technical reason why it shouldn't work, even when Debian hacks |
|
PS: As an extra, after re-reading @ntoll's comment above, if people want to use the system Python's packages, Mu could be told to run / debug using not its venv Python executable, but any Python executable that advanced beginners would set: no other changes would be needed at all. :) |
|
Morning @tmontes. That's a really interesting proposal, which I like a lot. Some thoughts below:
This feels a promising direction to take and I agree that the benefits are strong. I'm trying to remember why we didn't go with the venv route before. Perhaps (most likely) it's because we (I) didn't think of it. ;-) |
|
Glad that you feel the approach is promising. There is one thing that I suppose will be more complex but, from skimming the docs/code, seems doable: the Jupyter REPL. We should not require Jupyter and friends to be installed on the venv: thus, Juptyer/QtConsole will need to run from within Mu's own Python environment, connected to a Kernel running the venv's Python... This should be fun and good learning opportunity. :) |
|
My friends, But - if there will be no different way, I will must live with it. That is all. |
|
@abixadamj, thanks for sharing your concerns regarding possible constraints deriving from the infrastructure you work with: that's helpful. Importantly, your experience of 3 minutes (!!!) to launch Scratch feels like a very negative factor in a classroom environment. For clarification sake, how long does a Mu launch currently take after a clean boot? What about after being cached (launch/quit/launch)? And how long does it take to start running a program? Have you tried importing large packages like Regarding virtual environments:
Of course each person's use case is potentially very different, but I wonder if you are being over-conservative due to a bad experience you might have had before. Even though 0.5 GB of venv is not unheard of -- in particular when data analysis tools are brought in -- I'd say that such size is probably an over estimation of what the real size on disk will probably be (you will know better, of course!) :) With this in mind, my note in this issue is the culmination of observing many people, like you, having issues installing 3rd party packages -- a hugely wanted capability by many -- and things not working seamlessly. Using a venv will address most, if not all, of the issues that have been recently reported, while at the same time having the potential to simplify Mu's code by itself -- on the contrary, the more Please do not take this as personal confrontation: it is not. :-) It's more about an idea which I believe to be sounder, simpler, and that has the potential to address many user's issues, including yours. With such a solution, the potential for performance impact in your infrastructure could come out as something like:
Benefits:
:-) |
|
Just to chime in: I had the same sort of idea when we were brainstorming previously for the 3rd-party packages solution. I don't remember whether I voiced it publicly, but I think @ntoll came to his --user/--target solution before I developed the notion of a private venv further. When @tmontes raised it the other in passing I was glad to hear an independent version of the same notion and broadly support it. While the devil is obviously in the details. I think that the private venv does provide the neatest solution to the issue of safely and consistently pip-installing/uninstalling 3rd-party packages because it basically does the most tested thing in this sphere: combining pip & a venv. In addition, the notion of a pip-freeze to essentially copy the venv when upgrading obviously buys into the same idea: using the most commonly-invoked mechanism for generating one venv as a copy of another. Nor do I think the use of a venv, once it's established, should cause too much code churn elsewhere. eg the core process runner is at mu\mu\interface\panes.py:start_process and, potentially, maybe, just needs to switch from sys.executable to path-to-venv-executable. I acknowledge the problems which @abixadamj and others might face who are using any kind of unusual storage regime. Assuming that the cost of implementing a PoC isn't too high, we could try to put something in place and ask @abixadamj to try it out to see what the effect is on the runtime etc. In short: I'm in favour, altho 6D40 ugh I'm not proposing to offer a PR right here right now, but I'd be happy to assist in review or implementation in good time. |
|
My friends, |
|
@abixadamj now that everything is installed inside a new venv created with the Would you be able to test again with the latest release? https://github.com/mu-editor/mu/releases/ |
|
@carlosperate of course, after easter... |
|
@carlosperate |
|
Awesome, thanks for all the testing @abixadamj! |
When I want to install my own library: https://pypi.org/project/PyTechBrain/
I got an error:
Collecting pytechbrain
Downloading https://files.pythonhosted.org/packages/97/a1/4fb5521501dd2d02f5c06f9c0d40e81d53d49a1f8480dad7c7beabbf1bcd/PyTechBrain-0.7.0-py3-none-any.whl
Collecting pymata-aio (from pytechbrain)
Downloading https://files.pythonhosted.org/packages/c2/38/34346ae97407612ca49091cda062f9fa271c60e80f2cf601c37ed925330d/pymata_aio-2.28-py2.py3-none-any.whl (56kB)
Collecting pyserial (from pymata-aio->pytechbrain)
Downloading https://files.pythonhosted.org/packages/0d/e4/2a744dd9e3be04a0c0907414e2a01a7c88bb3915cbe3c8cc06e209f59c30/pyserial-3.4-py2.py3-none-any.whl (193kB)
Collecting websockets (from pymata-aio->pytechbrain)
Downloading https://files.pythonhosted.org/packages/43/71/8bfa882b9c502c36e5c9ef6732969533670d2b039cbf95a82ced8f762b80/websockets-7.0-cp36-cp36m-manylinux1_x86_64.whl (63kB)
Installing collected packages: pyserial, websockets, pymata-aio, pytechbrain
Exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 360, in run
prefix=options.prefix_path,
File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/usr/lib/python3/dist-packages/pip/wheel.py", line 247, in move_wheel_files
prefix=prefix,
File "/usr/lib/python3/dist-packages/pip/locations.py", line 153, in distutils_scheme
i.finalize_options()
File "/usr/lib/python3.6/distutils/command/install.py", line 274, in finalize_options
raise DistutilsOptionError("can't combine user with prefix, "
distutils.errors.DistutilsOptionError: can't combine user with prefix, exec_prefix/home, or install_(plat)base
FINISHED
The text was updated successfully, but these errors were encountered: