Proof of concept: RattlerFS #1182
Draft
+1,850
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Very exciting! |
|
@chrisburr 👋 Following up from our discussions in March, just curious if you have any plans for the future of this? |
|
In September I'll have an intern who will be working on this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This add a proof-of-concet FUSE based filesystem that transparently proxies the data in the package cache on demand.
Inspired by #1162, read there for some of the motivation. Unlike the overlayfs approach, I think this could work on linux, macos and even windows.
Also this is my first time writing rust so it's a bit of a mess but it shows the idea:
I would imagine this would normally be exposed by some kind of config setting for tools like pixi such that
pixi run xxxsets up the fuse mount, runs the command and then exists. This works but it does wipe out the kernel filesystem cache every time, so there would be an argument for having apixi mountcommand or similar.The main missing functionality is:
conda-meta/directoryPost link scripts (and pyc files) could be implemented either with overlayfs or by making RattlerFS writable (presumably as an overlay to another directory).
.pycfiles could alternatively be done by adding them to the central package cache and compiling them on demand.And if anyone would like to take over this I'd be happy to let them 😄