8000 POC: optionally use rustic_core by aawsome · Pull Request #92 · drdo/redu · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

POC: optionally use rustic_core #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

POC: optionally use rustic_core #92

wants to merge 2 commits into from

Conversation

aawsome
Copy link
@aawsome aawsome commented Apr 17, 2025

Here is a working POC which optionally uses rustic_core, see #87.

I think there are several things to enhance/fix, but I'll leave that to you @drdo ;-)

Here are the points, I stumbled over in the PoC:

  • error handling has been done just quick&dirty using anyhow or sometimes unhandled
  • Configuration of repository (storage+password): Here rustic has a complete different philosophy as restic and prefers config files to do the configuration. For the POC only local storage and a entered password works, see the unimplemented!() in the code and the docu to BackendOptions and RepositoryOptions in rustic_core. Actually if you were only using rustic_core, I would suggest to you to use those as leading config structs. Currently for the only working local storage, there is also way too much dependency overhead which is mostly because of the many storage backends rustic_core supports via opendal.
  • Due to the design (which I didn't want to break), for each snapshots and ls call, rustic (re-)opens the repository and (re-)reads the index. If you would change the design, things could work with just a single opening and index-load which would give much better performance when reading the tree from multiple snapshots.
  • I realized that you use all snapshot data to identify snapshots - snap.id would suffice. I also just mapped the needed data, feel free to complement the mapping from rustic_core::SnapshotFile to redu::Snapshot

@aawsome
Copy link
Author
aawsome commented Apr 18, 2025

Now the repository is only opened once and index is only read once (in Rustic::new()) which gives an immense performance boost especially when scanning multiple snapshots tree. It is still not optimal - better would be to not read the index for config() and snapshots() (not needed there) and only read it once before all ls() calls (and don't read it at all if no ls() is called) - but that would require a structural change outside of the restic module - feel free to do that yourself!

@drdo
Copy link
Owner
drdo commented Apr 29, 2025

Is there something in rustic to parse repo strings in a restic-compatible way? For example an sftp repo.

@aawsome
Copy link
Author
aawsome commented Apr 29, 2025 •
edited
Loading

Is there something in rustic to parse repo strings in a restic-compatible way? For example an sftp repo.

BackendOptions::to_backends() already does string parsing; however only local, rclone, rest and opendal are supported. To use a backend supported by opendal, you need to give opendal:<SERVICE> as repo string (e.g. opendal:sftp) and supply options (like server, username etc) in BackendOptions::options

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@aawsome @drdo
0