This repository contains scripts to automatically back up all of your S3-compatible buckets to your local machine on a scheduled basis. It works with:
- Amazon S3
- Cloudflare R2
- Automatically backs up all buckets from your S3-compatible service
- Configurable retention policy (default: keep the 3 most recent backups per bucket)
- Scheduled backups (default: every other week)
- Secure credential management using .env file (not committed to git)
- Works with any S3-compatible storage service
- macOS (Mac mini or other Mac)
- Homebrew (will be installed if not present)
- AWS CLI (will be installed via Homebrew if needed)
- Access credentials for your S3-compatible service
-
Clone this repository to your Mac:
git clone https://github.com/yourusername/s3-backup.git cd s3-backup -
Run the setup script:
chmod +x setup.sh ./setup.sh
-
Edit the
.envfile with your S3 credentials:nano .env
Update the following variables:
S3_ENDPOINT_URL: Your S3 service endpoint URLS3_ACCESS_KEY: Your access keyS3_SECRET_KEY: Your secret keyS3_REGION: Your S3 region (if applicable)LOCAL_BACKUP_DIR: Where to store your backups (default: ~/backups/s3)MAX_BACKUPS: Number of backups to retain per bucket (default: 3)
-
Make the backup script executable:
chmod +x scripts/s3_backup.sh
Here are endpoint URLs for the supported S3-compatible services:
S3_ENDPOINT_URL=https://s3.amazonaws.com
S3_REGION=us-east-1 # Replace with your region
S3_ENDPOINT_URL=https://accountid.r2.cloudflarestorage.com
S3_REGION=auto
To run a backup manually:
./scripts/s3_backup.shThe setup script automatically configures a cron job to run the backup every other Sunday at 2:00 AM.
To view the current cron job:
crontab -lTo modify the schedule:
crontab -eThe setup script will automatically install the AWS CLI if it's not already present on your system. It uses Homebrew to handle the installation.
If you prefer to install it manually, you can run:
# Install Homebrew if not already installed
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
# Install AWS CLI
brew install awscli
# Verify installation
aws --version- IMPORTANT: Never commit your
.envfile to version control - The
.gitignorefile is configured to exclude the.envfile containing your credentials - Only the
.env.templateshould be committed to git
Backup logs are stored in the same directory as the scripts, with filenames formatted as backup_YYYYMMDD.log.
If you encounter issues:
- Check the log files for error messages
- Verify your S3 credentials in the
.envfile - Ensure the AWS CLI is installed correctly:
aws --version
- Test your connection to your S3 service:
aws s3 ls --endpoint-url your-endpoint-url-here
- If you're getting permission errors, verify your access key and secret key have the necessary permissions
MIT