sopstool

README

sopstool

sopstool is a multi-file wrapper around sops. It uses the sops binary to encrypt and decrypt files, and piggybacks off the .sops.yaml configuration file.

sopstool provides functionality to manage multiple secret files at once, and even use as an entrypoint to decrypt at startup, for container images. Much of this behavior is inspired by the great blackbox project.

1.0.0 Release and Breaking Changes

1.0.0 release of sopstool introduces M1 / darwin-arm64 support. We also want to match build artifacts produced by GoReleaser to what sops produces. Therefore, this version introduces a breaking change where we no longer produce artifacts like sopstool_linux.(deb|rpm|tar.gz) and sopstool_darwin.tar.gz. Instead, you'll see artifacts like sopstool_darwin_(arm64|amd64)_(deb|rpm|tar.gz) and sopstool_linux_(arm64|amd64)_(deb|rpm|tar.gz) in future releases.

Installation

The most direct install uses a shell script hosted in this repository. This script will install the latest sops (if the command does not exist) and sopstool to ./bin by default.

curl https://raw.githubusercontent.com/Ibotta/sopstool/master/install.sh | bash

• Override the sops version with the environment variable SOPS_VERSION
• Override the sopstool version with the environment variable SOPSTOOL_VERSION
• Override the binary install location with the first shell argument
  • remember, you may need sudo or root access if you are installing to /usr/*

Example with overrides:

curl https://raw.githubusercontent.com/Ibotta/sopstool/master/install.sh | SOPS_VERSION=3.0.0 SOPSTOOL_VERSION=0.3.0 bash -s /usr/local/bin

Docker

Note: We currently only build a docker image for Linux - amd64.

To use sopstool in your docker container, you can use the direct install method above, but since Docker 1.13, there is a better way by using build stages!

In your Dockerfile:

COPY --from=ibotta/sopstool:latest usr/local/bin/sops usr/local/bin/sopstool /usr/local/bin/

Images are tagged with the same version numbering as the releases, and latest always gets the latest release. Note that your image will need root CA certificates (typically installed with curl, or a ca-certificates package).

To use sopstool in a docker container in other contexts (avoiding doing binary installs):

docker run --rm -v $(pwd):/work -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_REGION -e AWS_SECURITY_TOKEN -e AWS_SESSION_TOKEN ibotta/sopstool:latest $COMMAND

• sopstool is the entrypoint, so any sopstool subcommand can be run.
• /work is the default WORKDIR - this should be mounted to the root where .sops.yml is stored.
• the commands need access to your AWS credentials session to authenticate KMS.

Homebrew

Ibotta maintains a tap for their open-source projects, which includes sopstool. This will install sops as a requirement

brew install Ibotta/public/sopstool

Installing sops manually

Since sopstool requires sops, install it first. You can use one of the following methods:

Installing the sops binary with our script installer

The install script above uses a separate script to download sops

curl https://raw.githubusercontent.com/Ibotta/sopstool/master/sopsinstall.sh | bash

• Override the tag with the first shell argument (defaults to latest)
• Override the binary install location with the -b flag (defaults to /.bin)

(This script was generated by godownloader)

Download sops from our https mirror

To avoid needing to find the 'latest' binary by hand or by script, use our https server to download the binary. The latest binary is uploaded automatically whenever sopstool is deployed.

• Always the Latest
  • Linux: https://oss-pkg.ibotta.com/sops/sops_linux.tar.gz
  • MacOS: https://oss-pkg.ibotta.com/sops/sops_darwin.tar.gz
• Versions: always the pattern: https://oss-pkg.ibotta.com/sops/$TAG/sops_$OS.tar.gz

Download sops from github

You can install it by hand from a github release.

Installing sops using go (master branch)

go get -u go.mozilla.org/sops/cmd/sops

Installing sopstool manually

Following the lead of sops, we only build 64bit binaries.

Installing the sopstool binary using our script installer

The install script above uses a separate script to download sopstool

curl https://raw.githubusercontent.com/Ibotta/sopstool/master/sopstoolinstall.sh | bash

• Override the tag with the first shell argument (defaults to latest)
• Override the binary install location with the -b flag (defaults to /.bin)

(This script was generated by godownloader)

Download sopstool from our https mirror

To avoid needing to find the 'latest' binary by hand or by script, use our https server to download the binary. The latest binary is uploaded automatically whenever sopstool is deployed.

• Always the Latest
  • Linux: https://oss-pkg.ibotta.com/sopstool/sopstool_linux.tar.gz
  • MacOS: https://oss-pkg.ibotta.com/sopstool/sopstool_darwin.tar.gz
• Versions: always the pattern: https://oss-pkg.ibotta.com/sopstool/$TAG/sopstool_$OS.tar.gz

Download sopstool from github

Download the latest version for your platform from a github release.

Installing sopstool using go (master branch)

go get -u github.com/Ibotta/sopstool

Usage

This is a package that builds a single binary (per architecture) for wrapping sops with multi-file capabilities.

for more details, use the built-in documentation on commands:

sopstool -h

to get the shell completion helpers:

#!/usr/bin/env bash
sopstool completion

#!/usr/bin/env zsh
sopstool completion --sh zsh

Configuration

1. use a .sops.yaml file

   • this will be at the root of your project. this file is used to both configure keys as well as hold the list of files managed.

   • it needs to specify at least one KMS key accessible by your environment

     creation_rules:
       - kms: arn:aws:kms:REGION:ACCOUNT:key/KEY_ID
     

   • it can specify more complex cases of patterns vs keys too (see link)

How-To

1. Create a KMS Key.
2. Follow along the Configuration Steps, and place the .sops.yaml file at the root directory where your scripts will run.
   • All files added to SOPS are relative, or in child directories to the .sops.yaml configuration file.
3. Create a file to encrypt(any extension other than .yaml if you wish to do the ENTIRE file), or create a yaml file with key: value pairs(and make sure it's extension is .yaml). Sops will encrypt the values, but not it's keys.
   • You can read more about SOPS Here.
4. At this point, sopstool is ready and you can now sopstool add filename. You'll notice it will create a filename.sops.extension. This is your newly encrypted file.
   • When your files are properly encyrepted, you can run sopstool clean to remove the original plain text secret files.
5. Now, you can interact via the command line in various ways.
   • Editing an encrypted file - sopstool edit filename.sops.extension. You can also use your original filename too! sopstool edit filename.extension
   • Listing all encrypted files - sopstool list
   • Removing encrypted file - sopstool remove filename.extension
   • Display the contents of encrypted file - sopstool cat filename.extension

Walkthrough

In this walkthrough, we will go through the steps required to get a secure yaml configuration file running.

1. Configure your .sops.yaml

   # .sops.yaml
   creation_rules:
     - kms: arn:aws:kms:REGION:ACCOUNT:key/KEY_ID
   

2. Create a secrets yaml configuration file

   # credentials.yaml
   database.password: supersecretdb
   database.user: supersecretpassword
   redshift:
     user: my.user.name
     password: my.password
   

3. Encrypt the newly created file

   sopstool add credentials.yaml
   

4. Create a sample script

   # myscript.py
   import yaml
   with open('credentials.yaml', 'r') as f:
       credentials = yaml.load(f)
   
   print credentials["database.user"]
   print credentials["database.password"]
   print credentials["redshift"]["user"]
   print credentials["redshift"]["password"]
   

5. Here is what your folder structure would look like to this point(after deleting the unencrypted credentials.yaml file)

   my-project/
   ├── .sops.yaml
   ├── credentials.sops.yaml
   └── myscript.py
   

6. Accessing credentials

   The flow should be as follows: unencrypt credentials -> run script -> destroy credentials. You can use the sopstool entrypoint to achieve this.

   sopstool entrypoint python myscript.py
   

Contributing

Bug reports and pull requests are welcome at https://github.com/Ibotta/sopstool

docs

Generate markdown docs for the commands via

sopstool docs