sup

package module
v0.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 4, 2016 License: MIT Imports: 16 Imported by: 0

README

Stack Up

Stack Up is a simple deployment tool that performs given set of commands on multiple hosts in parallel. It reads Supfile, a YAML configuration file, which defines networks (groups of hosts), commands and targets.

Demo

Sup

Note: Demo is based on this example Supfile.

Installation

$ go get -u github.com/pressly/sup/cmd/sup

Usage

$ sup [OPTIONS] NETWORK COMMAND [...]
Options
Option Description
-f Supfile Custom path to Supfile
-e, --env=[] Set environment variables
--only REGEXP Filter hosts matching regexp
--except REGEXP Filter out hosts matching regexp
--debug, -D Enable debug/verbose mode
--disable-prefix Disable hostname prefix
--help, -h Show help/usage
--version, -v Print version

Network

A group of hosts.

# Supfile

networks:
    production:
        hosts:
            - api1.example.com
            - api2.example.com
            - api3.example.com
    staging:
        # fetch dynamic list of hosts
        inventory: curl http://example.com/latest/meta-data/hostname

$ sup production COMMAND will run COMMAND on api1, api2 and api3 hosts in parallel.

Command

A shell command(s) to be run remotely.

# Supfile

commands:
    restart:
        desc: Restart example Docker container
        run: sudo docker restart example
    tail-logs:
        desc: Watch tail of Docker logs from all hosts
        run: sudo docker logs --tail=20 -f example

$ sup staging restart will restart all staging Docker containers in parallel.

$ sup production tail-logs will tail Docker logs from all production containers in parallel.

Serial command (a.k.a. Rolling Update)

serial: N constraints a command to be run on N hosts at a time at maximum. Rolling Update for free!

# Supfile

commands:
    restart:
        desc: Restart example Docker container
        run: sudo docker restart example
        serial: 2

$ sup production restart will restart all Docker containers, two at a time at maximum.

Once command (one host only)

once: true constraints a command to be run only on one host. Useful for one-time tasks.

# Supfile

commands:
    build:
        desc: Build Docker image and push to registry
        run: sudo docker build -t image:latest . && sudo docker push image:latest
        once: true # one host only
    pull:
        desc: Pull latest Docker image from registry
        run: sudo docker pull image:latest

$ sup production build pull will build Docker image on one production host only and spread it to all hosts.

Local command

Runs command always on localhost.

# Supfile

commands:
    prepare:
        desc: Prepare to upload
        local: npm run build
Upload command

Uploads files/directories to all remote hosts. Uses tar under the hood.

# Supfile

commands:
    upload:
        desc: Upload dist files to all hosts
        upload:
          - src: ./dist
            dst: /tmp/
Interactive Bash on all hosts

Do you want to interact with multiple hosts at once? Sure!

# Supfile

commands:
    bash:
        desc: Interactive Bash on all hosts
        stdin: true
        run: bash

$ sup production bash
#
# type in commands and see output from all hosts!
# ^C

Passing prepared commands to all hosts:

$ echo 'sudo apt-get update -y' | sup production bash

# or:
$ sup production bash <<< 'sudo apt-get update -y'

# or:
$ cat <<EOF | sup production bash
sudo apt-get update -y
date
uname -a
EOF
Interactive Docker Exec on all hosts
# Supfile

commands:
    exec:
        desc: Exec into Docker container on all hosts
        stdin: true
        run: sudo docker exec -i $CONTAINER bash

$ sup production exec
ps aux
strace -p 1 # trace system calls and signals on all your production hosts

Target

Target is an alias for multiple commands. Each command will be run on all hosts in parallel, sup will check return status from all hosts, and run subsequent commands on success only (thus any error on any host will interrupt the process).

# Supfile

targets:
    deploy:
        - build
        - pull
        - migrate-db-up
        - stop-rm-run
        - health
        - slack-notify
        - airbrake-notify

$ sup production deploy

is equivalent to

$ sup production build pull migrate-db-up stop-rm-run health slack-notify airbrake-notify

Supfile

See example Supfile.

Basic structure
# Supfile
---
version: 0.4

# Global environment variables
env:
  NAME: api
  IMAGE: example/api

networks:
  local:
    hosts:
      - localhost
  staging:
    hosts:
      - stg1.example.com
  production:
    hosts:
      - api1.example.com
      - api2.example.com

commands:
  echo:
    desc: Print some env vars
    run: echo $NAME $IMAGE $SUP_NETWORK
  date:
    desc: Print OS name and current date/time
    run: uname -a; date

targets:
  all:
    - echo
    - date
Default environment variables available in Supfile
  • $SUP_HOST - Current host.
  • $SUP_NETWORK - Current network.
  • $SUP_USER - User who invoked sup command.
  • $SUP_TIME - Date/time of sup command invocation.
  • $SUP_ENV - Environment variables provided on sup command invocation. You can pass $SUP_ENV to another sup or docker commands in your Supfile.

Running sup from Supfile

Supfile doesn't let you import another Supfile. Instead, it lets you run sup sub-process from inside your Supfile. This is how you can structure larger projects:

./Supfile
./database/Supfile
./services/scheduler/Supfile

Top-level Supfile calls sup with Supfiles from sub-projects:

 restart-scheduler:
    desc: Restart scheduler
    run: >
      sup -f ./services/scheduler/Supfile $SUP_ENV $SUP_NETWORK restart
 db-up:
    desc: Migrate database
    run: >
      sup -f ./database/Supfile $SUP_ENV $SUP_NETWORK up

Development

fork it, hack it..

$ make build

create new Pull Request

We'll be happy to review & accept new Pull Requests!

License

Licensed under the MIT License.

Documentation

Index

Constants

View Source 
const VERSION = "0.4"

Variables

View Source 
var (
	Colors = []string{
		"\033[32m",
		"\033[33m",
		"\033[36m",
		"\033[35m",
		"\033[31m",
		"\033[34m",
	}
	ResetColor = "\033[0m"
)

Functions

func LocalTarCmdArgs 

func LocalTarCmdArgs(path, exclude string) []string

func NewTarStreamReader 

func NewTarStreamReader(cwd, path, exclude string) (io.Reader, error)

NewTarStreamReader creates a tar stream reader from a local path. TODO: Refactor. Use "archive/tar" instead.

func RemoteTarCommand 

func RemoteTarCommand(dir string) string

RemoteTarCommand returns command to be run on remote SSH host to properly receive the created TAR stream. TODO: Check for relative directory.

func ResolveLocalPath 

func ResolveLocalPath(cwd, path, env string) (string, error)

Types

type Client 

type Client interface {
	Connect(host string) error
	Run(task *Task) error
	Wait() error
	Close() error
	Prefix() (string, int)
	Write(p []byte) (n int, err error)
	WriteClose() error
	Stdin() io.WriteCloser
	Stderr() io.Reader
	Stdout() io.Reader
	Signal(os.Signal) error
}

type Command 

type Command struct {
	Name   string   `yaml:"-"`      // Command name.
	Desc   string   `yaml:"desc"`   // Command description.
	Local  string   `yaml:"local"`  // Command(s) to be run locally.
	Run    string   `yaml:"run"`    // Command(s) to be run remotelly.
	Script string   `yaml:"script"` // Load command(s) from script and run it remotelly.
	Upload []Upload `yaml:"upload"` // See Upload struct.
	Stdin  bool     `yaml:"stdin"`  // Attach localhost STDOUT to remote commands' STDIN?
	Once   bool     `yaml:"once"`   // The command should be run "once" (on one host only).
	Serial int      `yaml:"serial"` // Max number of clients processing a task in parallel.

	// API backward compatibility. Will be deprecated in v1.0.
	RunOnce bool `yaml:"run_once"` // The command should be run once only.
}

Command represents command(s) to be run remotely.

type EnvList 

type EnvList []EnvVar

EnvList is a list of environment variables that maps to a YAML map, but maintains order, enabling late variables to reference early variables.

func (*EnvList) Set 

func (e *EnvList) Set(key, value string)

Set key to be equal value in this list.

func (*EnvList) UnmarshalYAML 

func (e *EnvList) UnmarshalYAML(unmarshal func(interface{}) error) error

type EnvVar 

type EnvVar struct {
	Key   string
	Value string
}

EnvVar represents an environment variable

func (EnvVar) AsExport 

func (e EnvVar) AsExport() string

AsExport returns the environment variable as a bash export statement

func (EnvVar) String 

func (e EnvVar) String() string

type ErrConnect 

type ErrConnect struct {
	User   string
	Host   string
	Reason string
}

func (ErrConnect) Error 

func (e ErrConnect) Error() string

type ErrMustUpdate 

type ErrMustUpdate struct {
	Msg string
}

func (ErrMustUpdate) Error 

func (e ErrMustUpdate) Error() string

type ErrTask 

type ErrTask struct {
	Task   *Task
	Reason string
}

func (ErrTask) Error 

func (e ErrTask) Error() string

type LocalhostClient 

type LocalhostClient struct {
	// contains filtered or unexported fields
}

Client is a wrapper over the SSH connection/sessions.

func (*LocalhostClient) Close 

func (c *LocalhostClient) Close() error

func (*LocalhostClient) Connect 

func (c *LocalhostClient) Connect(_ string) error

func (*LocalhostClient) Prefix 

func (c *LocalhostClient) Prefix() (string, int)

func (*LocalhostClient) Run 

func (c *LocalhostClient) Run(task *Task) error

func (*LocalhostClient) Signal 

func (c *LocalhostClient) Signal(sig os.Signal) error

func (*LocalhostClient) Stderr 

func (c *LocalhostClient) Stderr() io.Reader

func (*LocalhostClient) Stdin 

func (c *LocalhostClient) Stdin() io.WriteCloser

func (*LocalhostClient) Stdout 

func (c *LocalhostClient) Stdout() io.Reader

func (*LocalhostClient) Wait 

func (c *LocalhostClient) Wait() error

func (*LocalhostClient) Write 

func (c *LocalhostClient) Write(p []byte) (n int, err error)

func (*LocalhostClient) WriteClose 

func (c *LocalhostClient) WriteClose() error

type Network 

type Network struct {
	Env       EnvList  `yaml:"env"`
	Inventory string   `yaml:"inventory"`
	Hosts     []string `yaml:"hosts"`
	Bastion   string   `yaml:"bastion"` // Jump host for the environment
}

Network is group of hosts with extra custom env vars.

func (Network) ParseInventory 

func (n Network) ParseInventory() ([]string, error)

ParseInventory runs the inventory command, if provided, and appends the command's output lines to the manually defined list of hosts.

type SSHClient 

type SSHClient struct {
	// contains filtered or unexported fields
}

Client is a wrapper over the SSH connection/sessions.

func (*SSHClient) Close 

func (c *SSHClient) Close() error

Close closes the underlying SSH connection and session.

func (*SSHClient) Connect 

func (c *SSHClient) Connect(host string) error

Connect creates SSH connection to a specified host. It expects the host of the form "[ssh://]host[:port]".

func (*SSHClient) ConnectWith 

func (c *SSHClient) ConnectWith(host string, dialer SSHDialFunc) error

ConnectWith creates a SSH connection to a specified host. It will use dialer to establish the connection. TODO: Split Signers to its own method.

func (*SSHClient) DialThrough 

func (sc *SSHClient) DialThrough(net, addr string, config *ssh.ClientConfig) (*ssh.Client, error)

DialThrough will create a new connection from the ssh server sc is connected to. DialThrough is an SSHDialer.

func (*SSHClient) Prefix 

func (c *SSHClient) Prefix() (string, int)

func (*SSHClient) Run 

func (c *SSHClient) Run(task *Task) error

Run runs the task.Run command remotely on c.host.

func (*SSHClient) Signal 

func (c *SSHClient) Signal(sig os.Signal) error

func (*SSHClient) Stderr 

func (c *SSHClient) Stderr() io.Reader

func (*SSHClient) Stdin 

func (c *SSHClient) Stdin() io.WriteCloser

func (*SSHClient) Stdout 

func (c *SSHClient) Stdout() io.Reader

func (*SSHClient) Wait 

func (c *SSHClient) Wait() error

Wait waits until the remote command finishes and exits. It closes the SSH session.

func (*SSHClient) Write 

func (c *SSHClient) Write(p []byte) (n int, err error)

func (*SSHClient) WriteClose 

func (c *SSHClient) WriteClose() error

type SSHDialFunc 

type SSHDialFunc func(net, addr string, config *ssh.ClientConfig) (*ssh.Client, error)

SSHDialFunc can dial an ssh server and return a client

type Stackup 

type Stackup struct {
	// contains filtered or unexported fields
}

func New 

func New(conf *Supfile) (*Stackup, error)

func (*Stackup) Debug 

func (sup *Stackup) Debug(value bool)

func (*Stackup) Prefix 

func (sup *Stackup) Prefix(value bool)

func (*Stackup) Run 

func (sup *Stackup) Run(network *Network, commands ...*Command) error

Run runs set of commands on multiple hosts defined by network sequentially. TODO: This megamoth method needs a big refactor and should be split 

to multiple smaller methods.

type Supfile 

type Supfile struct {
	Networks map[string]Network  `yaml:"networks"`
	Commands map[string]Command  `yaml:"commands"`
	Targets  map[string][]string `yaml:"targets"`
	Env      EnvList             `yaml:"env"`
	Version  string              `yaml:"version"`
}

Supfile represents the Stack Up configuration YAML file.

func NewSupfile 

func NewSupfile(file string) (*Supfile, error)

NewSupfile parses configuration file and returns Supfile or error.

type Task 

type Task struct {
	Run     string
	Input   io.Reader
	Clients []Client
	TTY     bool
}

Task represents a set of commands to be run.

type Upload 

type Upload struct {
	Src string `yaml:"src"`
	Dst string `yaml:"dst"`
	Exc string `yaml:"exclude"`
}

Upload represents file copy operation from localhost Src path to Dst path of every host in a given Network.

Source Files

View all Source files

Directories

Path Synopsis
cmd
sup

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.