s3-cleaner

README

s3-cleaner

It is a tool for finding desired files in a S3 bucket and cleans them with a rule set.

You can provide access credentials of your AWS account with below environment variables or CLI flags. Keep in mind that command line flags will override environment variables if you set both of them:

"--accessKey" CLI flag or "AWS_ACCESS_KEY" environment variable
"--secretKey" CLI flag or "AWS_SECRET_KEY" environment variable
"--region" CLI flag or "AWS_REGION" environment variable
"--bucketName" CLI flag or "AWS_BUCKET_NAME" environment variable

Configuration

Usage:
  s3-cleaner start [flags]

Flags:
      --autoApprove             Skip interactive approval (default false)
      --dryRun                  specifies that if you just want to see what to delete or completely delete them all (default false)
      --fileExtensions string   selects the files with defined extensions to clean from target bucket, "" means all files (default "")
  -h, --help                    help for start
      --keepLastNFiles int      defines how many of the files to skip deletion in specified criteria, 0 means clean them all (default 1)
      --maxFileSizeInMb int     maximum size in mb to clean from target bucket, 0 means no upper limit (default 15)
      --minFileSizeInMb int     minimum size in mb to clean from target bucket, 0 means no lower limit (default 10)
      --sortBy string           defines the ascending order in the specified criteria, valid options are "lastModificationDate" and "size" (default "lastModificationDate")

Global Flags:
      --accessKey string        access key credential to access S3 bucket (default "")
      --bucketName string       name of the target bucket on S3 (default "")
      --fileNamePrefix string   folder name of target bucket objects, means it can be used for folder-based object grouping buckets (default "")
      --region string           region of the target bucket on S3 (default "")
      --secretKey string        secret key credential to access S3 bucket (default "")
  -v, --verbose                 verbose output of the logging library (default false)

Installation

Binary

Binary can be downloaded from Releases page.

After then, you can simply run binary by providing required command line arguments:

$ ./s3-cleaner start --accessKey=xxxxx --secretKey=xxxxx --region=xxxxx --bucketName=xxxxx

or by providing environment variables for access credentials:

$ AWS_ACCESS_KEY=xxxxx AWS_SECRET_KEY=xxxxx AWS_REGION=xxxxx AWS_BUCKET_NAME=xxxxx ./s3-cleaner start

Homebrew

This project can be installed with Homebrew:

$ brew tap bilalcaliskan/tap
$ brew install bilalcaliskan/tap/s3-cleaner

Then similar to binary method, you can run it by calling below command:

$ s3-cleaner start --accessKey=xxxxx --secretKey=xxxxx --region=xxxxx --bucketName=xxxxx

or by providing environment variables for access credentials:

$ AWS_ACCESS_KEY=xxxxx AWS_SECRET_KEY=xxxxx AWS_REGION=xxxxx AWS_BUCKET_NAME=xxxxx ./s3-cleaner start

Development

This project requires below tools while developing:

• Golang 1.20
• pre-commit
• golangci-lint - required by pre-commit
• gocyclo - required by pre-commit

Simply run below command to prepare your development environment:

$ python3 -m venv venv
$ source venv/bin/activate
$ pip3 install pre-commit
$ pre-commit install -c build/ci/.pre-commit-config.yaml

Documentation

Overview

Copyright © 2022 bilalcaliskan bilalcaliskan@protonmail.com