xstore

README

xStore CLI

Simple, secure, and flexible CLI configuration management.

xStore CLI provides six basic commands to manage configuration files.

Cmd	Action	Example	Flags
push	synchronize	$ xstore push	-f -s -t -c
pull	download	$ xstore pull	-f -s -t -o
purge	delete remote	$ xstore purge	-f -s -t
clean	delete local	$ xstore clean	-f -s -t
list	list catalog	$ xstore list	-f -s -t
tag	tag files	$ xstore tag	-f -s -t, -a, -d

Synced files are replaced by an xstore.yml file that can be shared and checked into source control safely. xstore.yml catalogs the local file path, remote storage details, and file encryption making local restoration or consumption by a service as simple as $ xstore pull.

Install / Upgrade

OS	CMD	Notes
Mac	$ sudo curl -L -o /usr/local/bin/xstore https://github.com/dabblebox/xstore/releases/download/v0.1.0-rc/xstore_darwin_amd64 && sudo chmod +x /usr/local/bin/xstore
Linux	$ sudo curl -L -o /usr/local/bin/xstore https://github.com/dabblebox/xstore/releases/download/v0.1.0-rc/xstore_linux_386 && sudo chmod +x /usr/local/bin/xstore
Windows	C:\> mkdir %HOMEPATH%\xstore\bin & wget -O %HOMEPATH%\xstore\bin\xstore.exe https://github.com/dabblebox/xstore/releases/download/v0.1.0-rc/xstore_windows_amd64.exe (add %HOMEPATH%\xstore\bin to the PATH to make xstore executable from anywhere)	install requires wget v1.20

AWS Storage Solutions

Store	Supported File Types
Secrets Manager	.env, .json, .yml, .xml, .sql, .cert
Parameter Store	.env
S3 Storage	.*

CLI Authentication

$ export AWS_REGION=us-east-1
$ export AWS_PROFILE=user-profile-devops

CLI Commands

$ xstore push

Sync single file with remote store

$ xstore push service/dev/.env

Sync multiple files with remote store

$ xstore push service/dev/.env service/qa/.env

Auto find files in service folder and sync with remote store

$ xstore push $(find service -name '*.env')

Flag	Examples	Description
-f	xstore.yml	catalog name
-c	app-name	remote storage key prefix
-s	secrets-manager, paramter-store, s3	remote storage location
-t	service,dev,app	tagging for quick file reference

$ xstore pull

Download files

$ xstore pull

$ xstore pull service/dev/.env service/qa/.env 

Download files by custom tags

$ xstore pull -t service,dev # match all tags

Download files by remote store

$ xstore pull -s s3

Send raw data to stdout

$ xstore pull -o

Send JSON objects to stdout

$ xstore pull service/dev/.env -o json-object

Export environment variables

$ eval $( xstore pull service/dev/.env -o terminal-export )

Flag	Examples	Description
-f	xstore.yml	catalog name
-o	terminal-export, json-object	file output format
-s	secrets-manager, paramter-store, s3	remote storage location
-t	service,dev,app	tagging for quick file reference

$ xstore purge

Delete all tracked files

$ xstore purge

Delete specific tracked files

$ xstore purge service/dev/.env service/qa/.env 

Delete tagged tracked files

$ xstore purge -t service,dev # match all tags

Delete files by remote store

$ xstore purge -s s3

Flag	Examples	Description
-f	xstore.yml	catalog name
-s	secrets-manager, paramter-store, s3	remote storage location
-t	service,dev,app	tagging for quick file reference

$ xstore clean

Delete local copies of tracked files

$ xstore clean

Delete specific local copies of tracked files

$ xstore clean service/dev/.env service/qa/.env 

Delete tagged local copies of tracked files

$ xstore clean -t service,dev # match all tags

Delete local copies by remote store

$ xstore clean -s s3

Flag	Examples	Description
-f	xstore.yml	catalog name
-s	secrets-manager, paramter-store, s3	remote storage location
-t	service,dev,app	tagging for quick file reference

$ xstore list

List tracked files

$ xstore list

List specific tracked files

$ xstore list service/dev/.env service/qa/.env 

List tagged tracked files

$ xstore list -t service,dev

List tracked files by store

$ xstore list -s s3

Flag	Examples	Description
-f	xstore.yml	catalog name
-s	secrets-manager, paramter-store, s3	remote storage location
-t	service,dev,app	tagging for quick file reference

$ xstore tag

Overwrite tags on specific tracked files

$ xstore tag service/dev/.env service/qa/.env -t app,non-prod

Add tag to tracked files with specic tags

$ xstore tag -t service,dev -a non-prod

Delete tag from tracked files with specic tags

$ xstore tag -t service,dev -d service

Add tag to tracked files for remote store

$ xstore tag -s s3 -a config

Flag	Examples	Description
-f	xstore.yml	catalog name
-s	secrets-manager, paramter-store, s3	remote storage location
-t	service,dev,app	tagging for quick file reference
-a	app,non-prod	add tags
-d	app,non-prod	delete tags

Environment Variables

Any field in the xstore.yml can be overwritten when pulling configuration. Use _ to access children.

$ export XSTORE_FILES_SAMPLE_DEV__ENV_OPT_S3_BUCKET=configs

Setting environment variables will default values and eliminate prompts when pushing new files.

Variable	Examples	Description
XSTORE_CATALOG	xstore.yml	catalog name
XSTORE_KMS_KEY_ID	aws/secretsmanager	KMS key id or default key alias
XSTORE_S3_BUCKET	configs	S3 bucket name
XSTORE_STORE	s3	remote store name

Application Integration

Go (go1.13.0)

package main

import (
  "log"
  "fmt"
  
  "github.com/dabblebox/kendavis2/xstore"
  "github.com/dabblebox/kendavis2/xstore/action"
)

data, err := xstore.Pull(action.PullOpt{
    Output: "json-object",
    Tags:   []string{"dev"},
})
if err != nil {
    log.Fatal(err)
}

m := map[string]string{}
if err := json.Unmarshal(data[0].Data, &m); err != nil {
    log.Fatal(err)
}

for k, v := range m {
    fmt.Printf("\n%s=%s", k, v)
}

BuildNRun

$ ./build
$ ./xstore push .env

Documentation

Index

• func Pull(opt action.PullOpt) ([]action.DownloadedFile, error)

Functions

func Pull

func Pull(opt action.PullOpt) ([]action.DownloadedFile, error)

Pull downloads files from a remote store.