8000 GitHub - Erisa/tigrisfs at refs/heads/patch-1
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
/ tigrisfs Public
forked from tigrisdata/tigrisfs

High performance FUSE filesystem for AI workloads with S3 compatible backends

License

Notifications You must be signed in to change notification settings

Erisa/tigrisfs

 
 

Repository files navigation

unittests xfstests cluster-test

TigrisFS is a high-performance FUSE-based file system for S3-compatible object storage written in Go.

Overview

TigrisFS allows you to mount an S3 or compatible object store bucket as a local file system.

TigrisFS is based on GeeseFS, which is a fork of Goofys. GeeseFS focused on solving performance problems which FUSE file systems based on S3 typically have, especially with small files and metadata operations. It solves these problems by using aggressive parallelism and asynchrony.

The goal of TigrisFS is to further improve on performance and reliability especially in distributed cluster setup.

In the first release we focused on improving security and reliability of the code base:

  • Improved security by:
    • removing bundled, outdated AWS SDK with critical vulnerabilities.
    • upgrading dependencies to fix security vulnerabilities.
  • Improved reliability by:
    • fixing all race conditions found by race detector, which is now enabled by default in tests.
    • fixing all linter issues and enabling linting by default in CI.
    • running more extensive tests and enabling them by default in CI.

Tigris specific features

When mounted with the Tigris backend TigrisFS supports:

  • POSIX permissions, special files, symbolic links.
  • Auto-preload content of small files on directory list in single request.
  • Allows to auto prefetch directory data to the region on list.

Installation

Prebuilt DEB and RPM packages

  • Download the latest release: DEB, RPM.

  • Install the package:

    • Debian-based systems:
      dpkg -i tigrisfs_1.2.0_linux_amd64.deb
    • RPM-based systems:
      rpm -i tigrisfs_1.2.0_linux_amd64.rpm
  • Configure credentials TigrisFS can use credentials from different sources:

    • Standard AWS credentials files ~/.aws/credentials and ~/.aws/config. Use aws configure to set them up and export AWS_PROFILE environment variable to use a specific profile.
    • Environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY.
    • TigrisFS credentials in /etc/default/tigrisfs or mount specific credentials in /etc/default/tigrisfs-<bucket>. See docs for more details.
  • Mount the bucket

    • as current user
      systemctl --user start tigrisfs@<bucket>
      The bucket is mounted at $HOME/mnt/tigris/<bucket>.
    • as root
      systemctl start tigrisfs@<bucket>
      The bucket is mounted at /mnt/tigris/<bucket>.

Binary install

  • Download and unpack the latest release:
    • MacOS ARM64
        curl -L https://github.com/tigrisdata/tigrisfs/releases/download/v1.2.0/tigrisfs_1.2.0_darwin_arm64.tar.gz | sudo tar -xz -C /usr/local/bin
      
  • Configuration is the same as for the DEB and RPM packages above.
  • Mount the bucket:
    • as current user
      /usr/local/bin/tigrisfs <bucket> $HOME/mnt/tigrisfs/<bucket>
    • as root
      sudo /usr/local/bin/tigrisfs <bucket> /mnt/tigrisfs/<bucket>

License

Licensed under the Apache License, Version 2.0

See LICENSE and AUTHORS

About

High performance FUSE filesystem for AI workloads with S3 compatible backends

Resources

License

Stars

Watchers

Forks

Sponsor this project

 

Packages

No packages published

Languages

  • Go 92.1%
  • Shell 6.4%
  • Other 1.5%
0