README
¶
Bucket Scanner
[
] (https://cjbarker.gitlab.io/bucketscanner/test-coverage.html)
Overview
Searching Cloud Storage Since 2017
Usage
The bucketscanner requires a cloud provider and action coupled with the bucket name(s).
Usage: ./bucketscanner-darwin-amd64 --cloud=CLOUD --action=ACTION [<flags>] <bucket-name>
Cloud command-line bucket (object) scanner.
Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
--version Show application version.
--cloud=CLOUD Cloud provider to scan: aws, gcp, azure. Defaults to all.
--action=ACTION Scan action to invoke against bucket: (r)ead, (w)rite, all. Defaults to all.
--throttle=THROTTLE Time in milliseconds to throttle subsequent requests sent to a given provider.
--download Download bucket content(s).
--output=OUTPUT Download bucket content(s) destination directory. Defaults to current user's directory if
none passed.
--JSON Output results in JSON.
--verbose Verbose output messages. Defaults to quiet.
Args:
<bucket-name> Bucket(s) name(s) to scan. Does support comma separated for multiple buckets.
Example searching one bucket on AWS for read-access:
./bucketscanner --cloud=aws --action=read --JSON listing-test
{
"files": [
{
"Body": null,
"directory": true,
"files": null,
"name": "empty folder/",
"size": 0
},
{
"Body": null,
"directory": true,
"files": null,
"name": "empty folder/empty folder/",
"size": 0
},
{
"Body": null,
"directory": false,
"files": null,
"name": "index-bucketname.html",
"size": 362
},
{
"Body": null,
"directory": false,
"files": null,
"name": "index-null.html",
"size": 323
},
{
"Body": null,
"directory": false,
"files": null,
"name": "index-path.html",
"size": 385
},
{
"Body": null,
"directory": false,
"files": null,
"name": "index-vh.html",
"size": 385
}
],
"name": "listing-test",
"noFiles": 6,
"provider": "Amazon Simple Storage Service (S3)",
"scanned": "2018-04-11T11:31:16.78290151-07:00",
"state": 3,
"totalSize": 1455,
"uri": "https://listing-test.s3.amazonaws.com"
}
Developer
Bucketscanner supports multiple platform builds via GNU Make. It does assume and rely on Glide for GoLang package management including dependencies. Please ensure glide is installed and available in your path before continuing.
To build the binary and library you'll need to clone the repo, setup GoLang and run make.
The default make target builds both components command line binary and library (bucketscanner and libbucketscanner).
# Assumes GOPATH exists and golang installed with tools in path
# export PATH=${GOPATH}/bin:${PATH}
cd ${GOPATH}/src
mkdir -p gitlab.com/cjbarker/
cd gitlab.com/cjbarker
git clone git@gitlab.com:cjbarker/bucketscanner.git
cd bucketscanner
make
# Built Binary & Library
ls bin/
bucketscanner libbucketscanner
Continous Integration
GitLab's CI Pipelines handle the continuos integration (CI) and eventually will also handle the continuous deployment (CD) to cloud provider (TBA).
All management of CI/CD is handled via the .gitlab-ci.yml file. For more details on GitLab CI and job configuration consult: https://docs.gitlab.com/ce/ci/yaml/README.html
Any commit to a branch will trigger the CI. If you do not want the pipeline's job(s) to trigger you can add a skip to your git commit message.
git add <file>
git commit -m "[skip ci] will not trigger GitLab CI job"
git push
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( Version string Build string )
Variables to identify the build
Functions ¶
This section is empty.
Types ¶
type AwsScanner ¶
type AwsScanner struct {
// contains filtered or unexported fields
}
AwsScanner is struct for cloud scanner of Amazon Web Services
func (AwsScanner) GetProviderName ¶
func (a AwsScanner) GetProviderName() (cloudProviderName string)
GetProviderName returns the given Cloud Provider's name for the scanner
type AzureScanner ¶
type AzureScanner struct {
}
AzureScanner is struct for cloud scanner of Azure
func (AzureScanner) GetProviderName ¶
func (a AzureScanner) GetProviderName() (cloudProviderName string)
GetProviderName returns the given Cloud Provider's name for the scanner
type Bucket ¶
type Bucket struct {
Provider string `json:"provider"`
Name string `json:"name"`
Scanned time.Time `json:"scanned"`
URI string `json:"uri"`
State BucketState `json:"state"`
NoFiles int64 `json:"noFiles"`
TotalSize int64 `json:"totalSize"`
Files []file `json:"files"`
// contains filtered or unexported fields
}
Bucket structure is the results of a given bucket including its meta-data
type BucketState ¶
type BucketState int
BucketState denotes an integer defining its given state
const ( Unknown BucketState = iota Invalid // Bucket does not exists e.g. 404 Not Found Private // Bucket exists but is not accessible e.g. 403 Forbidden Public // Bucket exists and is available e.g. 200 OK RateLimited // Unable to determine due to rate limiting e.g. 503 Slow Down )
Bucket states
type Contents ¶
type Contents struct {
XMLName xml.Name `xml:"Contents"`
Key string
LastModified string
Etag string `xml:"ETag"`
Size int
StorageClass string
}
Contents are the XML contents of the bucket (one per file)
type GcpScanner ¶
type GcpScanner struct {
}
GcpScanner is struct for cloud scanner of Google Cloud Platform
func (GcpScanner) GetProviderName ¶
func (g GcpScanner) GetProviderName() (cloudProviderName string)
GetProviderName returns the given Cloud Provider's name for the scanner
Source Files
Directories