tsa

README

tsa

controls worker authentication within concourse

by stuckincustoms

reporting issues and requesting features

please report all issues and feature requests in concourse/concourse

about

TSA is the way workers securely register to join a Concourse deployment. It provides authentication and transport encryption (if required). Worker machines can ssh into TSA with a custom command to register or have traffic forwarded to them. Once an SSH session has been established then TSA begins to automatically heartbeat information about the worker into the ATC's pool.

The main advantage that this provides over the old style of registration is that Workers no longer need to be internet routable in order to have the ATC reach them. They open a reverse tunnel through the TSA which, when collocated with ATC, is far more likely to be easily routable. This also allows for simpler setup and better security as before you either had to expose your Garden server publicly or set up some interesting custom security if the workers and ATC were not in the same private network.

usage

First, create two new SSH keys:

$ ssh-keygen -t rsa -f host_key
$ ssh-keygen -t rsa -f worker_key

Next, let's create an authorized keys file so that our workers are able to authenticate with us without providing a password:

cat worker_key.pub > authorized_keys

Now to start tsa itself:

tsa \
  --peer-address $PEER_ADDRESS \
  --host-key ./host_key \
  --authorized-keys ./authorized_keys \
  --session-signing-key $SIGNING_KEY \
  --atc-url $ATC_URL

The variables here should be set to:

Variable	Description
$PEER_ADDRESS	The host or IP where this machine can be reached by the ATC for the purpose of forwarding traffic to remote workers.
$SIGNING_KEY	RSA key used to sign the tokens used when communicating to the ATC.
$ATC_URL	ATC URL reachable by the TSA (e.g. https://ci.concourse-ci.org).

forwarding workers

In order to have a worker on a remote network register with tsa and have its traffic forwarded you can run the following command:

ssh -p 2222 $TSA_HOST \
  -i worker_key \
  -o UserKnownHostsFile=host_key.pub \
  -R0.0.0.0:7777:127.0.0.1:7777 \
  -R0.0.0.0:7788:127.0.0.1:7788 \
  forward-worker \
    --garden 0.0.0.0:7777 \
    --baggageclaim 0.0.0.0:7788 \
  < worker.json

Note that in this case you should always have Garden and BaggageClaim listen on 127.0.0.1 so that they're not exposed to the outside world. For this reason there is no $GARDEN_ADDR or $BAGGAGECLAIM_URL.

The worker.json file should contain the following:

{
    "platform": "linux",
    "tags": []
}

Documentation

Index

• Constants
• Variables
• func KeepAlive(ctx context.Context, sshClient *ssh.Client, tcpConn *net.TCPConn, ...)
• type Client
  • func (client *Client) ContainersToDestroy(ctx context.Context) ([]string, error)
  • func (client *Client) Delete(ctx context.Context) error
  • func (client *Client) Land(ctx context.Context) error
  • func (client *Client) Register(ctx context.Context, opts RegisterOptions) error
  • func (client *Client) ReportContainers(ctx context.Context, handles []string) error
  • func (client *Client) ReportVolumes(ctx context.Context, handles []string) error
  • func (client *Client) Retire(ctx context.Context) error
  • func (client *Client) VolumesToDestroy(ctx context.Context) ([]string, error)
• type Deleter
  • func (l *Deleter) Delete(ctx context.Context, worker atc.Worker) error
• type EndpointPicker
  • func NewRandomATCEndpointPicker(atcURLFlags []flag.URL) EndpointPicker
• type Event
• type EventReader
  • func NewEventReader(src io.Reader) EventReader
  • func (r EventReader) Next() (Event, error)
• type EventType
• type EventWriter
  • func NewEventWriter(dest io.Writer) EventWriter
  • func (w EventWriter) Heartbeated() error
  • func (w EventWriter) Registered() error
• type HandshakeError
  • func (err *HandshakeError) Error() string
• type HeartbeatStatus
• type Heartbeater
  • func NewHeartbeater(clock clock.Clock, interval time.Duration, cprInterval time.Duration, ...) *Heartbeater
  • func (heartbeater *Heartbeater) Heartbeat(ctx context.Context) error
• type Lander
  • func (l *Lander) Land(ctx context.Context, worker atc.Worker) error
• type RegisterOptions
• type Retirer
  • func (l *Retirer) Retire(ctx context.Context, worker atc.Worker) error
• type Sweeper
  • func (l *Sweeper) Sweep(ctx context.Context, worker atc.Worker, resourceAction string) ([]byte, error)
• type WorkerStatus
  • func (l *WorkerStatus) WorkerStatus(ctx context.Context, worker atc.Worker, resourceAction string) error

Constants

const (
	ForwardWorker = "forward-worker"

	LandWorker   = "land-worker"
	RetireWorker = "retire-worker"
	DeleteWorker = "delete-worker"

	ReportContainers      = "report-containers"
	ReportVolumes         = "report-volumes"
	ResourceActionMissing = "resource-type-missing"
)

const (
	HeartbeatStatusUnhealthy = iota
	HeartbeatStatusLanded
	HeartbeatStatusGoneAway
	HeartbeatStatusHealthy
)

const (
	SweepContainers = "sweep-containers"
	SweepVolumes    = "sweep-volumes"
)

Variables

var ErrAllGatewaysUnreachable = errors.New("all worker SSH gateways unreachable")

ErrAllGatewaysUnreachable is returned when all hosts reject the connection.

var ErrConnectionDrainTimeout = errors.New("timeout draining connections")

ErrConnectionDrainTimeout is returned when the connection underlying a registration has been idle for the configured ConnectionDrainTimeout.

Functions

func KeepAlive

func KeepAlive(ctx context.Context, sshClient *ssh.Client, tcpConn *net.TCPConn, interval time.Duration, timeout time.Duration)

Types

type Client

type Client struct {
	Hosts    []string
	HostKeys []ssh.PublicKey

	PrivateKey *rsa.PrivateKey

	Worker atc.Worker
}

Client is used to communicate with a pool of remote SSH gateways.

func (*Client) ContainersToDestroy

func (client *Client) ContainersToDestroy(ctx context.Context) ([]string, error)

ContainersToDestroy invokes the 'sweep-containers' command, returning a list of handles to be destroyed.

func (*Client) Delete

func (client *Client) Delete(ctx context.Context) error

Delete invokes the 'delete-worker' command, which will immediately unregister the worker without draining, causing any existing registrations to exit.

func (*Client) Land

func (client *Client) Land(ctx context.Context) error

Land invokes the 'land-worker' command, which will initiate the landing process for the worker. The worker will transition to 'landing' and finally to 'landed' when it is fully drained, causing any existing registrations to exit.

func (*Client) Register

func (client *Client) Register(ctx context.Context, opts RegisterOptions) error

Register invokes the 'forward-worker' command, proxying traffic through the tunnel and to the configured Garden/Baggageclaim addresses. It will also continuously keep the connection alive. The SSH gateway will continuously heartbeat the worker.

If the context is canceled, heartbeating is immediately stopped and the remote SSH gateway will wait for connections to drain. If a ConnectionDrainTimeout is configured, the connection will be terminated after no data has gone to/from the SSH gateway for the configured duration.

func (*Client) ReportContainers

func (client *Client) ReportContainers(ctx context.Context, handles []string) error

ReportContainers invokes the 'report-containers' command, sending a list of the worker's container handles to Concourse.

func (*Client) ReportVolumes

func (client *Client) ReportVolumes(ctx context.Context, handles []string) error

ReportVolumes invokes the 'report-volumes' command, sending a list of the worker's container handles to Concourse.

func (*Client) Retire

func (client *Client) Retire(ctx context.Context) error

Retire invokes the 'retire-worker' command, which will initiate the retiring process for the worker. The worker will transition to 'retiring' and disappear when it is fully drained, causing any existing registrations to exit.

func (*Client) VolumesToDestroy

func (client *Client) VolumesToDestroy(ctx context.Context) ([]string, error)

VolumesToDestroy invokes the 'sweep-volumes' command, returning a list of handles to be destroyed.

type Deleter

type Deleter struct {
	ATCEndpoint *rata.RequestGenerator
	HTTPClient  *http.Client
}

func (*Deleter) Delete

func (l *Deleter) Delete(ctx context.Context, worker atc.Worker) error

type EndpointPicker

type EndpointPicker interface {
	Pick() *rata.RequestGenerator
}

func NewRandomATCEndpointPicker

func NewRandomATCEndpointPicker(atcURLFlags []flag.URL) EndpointPicker

type Event

type Event struct {
	Type EventType `json:"event"`
}

type EventReader

type EventReader struct {
	// contains filtered or unexported fields
}

func NewEventReader

func NewEventReader(src io.Reader) EventReader

func (EventReader) Next

func (r EventReader) Next() (Event, error)

type EventType

type EventType string

const (
	EventTypeRegistered  EventType = "registered"
	EventTypeHeartbeated EventType = "heartbeated"
)

type EventWriter

type EventWriter struct {
	// contains filtered or unexported fields
}

func NewEventWriter

func NewEventWriter(dest io.Writer) EventWriter

func (EventWriter) Heartbeated

func (w EventWriter) Heartbeated() error

func (EventWriter) Registered

func (w EventWriter) Registered() error

type HandshakeError

type HandshakeError struct {
	Err error
}

HandshakeError is returned when the client fails to establish an SSH connection, possibly due to bad credentials.

func (*HandshakeError) Error

func (err *HandshakeError) Error() string

type HeartbeatStatus

type HeartbeatStatus int

type Heartbeater

type Heartbeater struct {
	// contains filtered or unexported fields
}

func NewHeartbeater

func NewHeartbeater(
	clock clock.Clock,
	interval time.Duration,
	cprInterval time.Duration,
	gardenClient gclient.Client,
	baggageclaimClient baggageclaim.Client,
	atcEndpointPicker EndpointPicker,
	httpClient *http.Client,
	worker atc.Worker,
	eventWriter EventWriter,
) *Heartbeater

func (*Heartbeater) Heartbeat

func (heartbeater *Heartbeater) Heartbeat(ctx context.Context) error

type Lander

type Lander struct {
	ATCEndpoint *rata.RequestGenerator
	HTTPClient  *http.Client
}

func (*Lander) Land

func (l *Lander) Land(ctx context.Context, worker atc.Worker) error

type RegisterOptions

type RegisterOptions struct {
	// The local Garden network and address to forward through the SSH gateway.
	LocalGardenNetwork string
	LocalGardenAddr    string

	// The local Baggageclaim network and address to forward through the SSH
	// gateway.
	LocalBaggageclaimNetwork string
	LocalBaggageclaimAddr    string

	// Under normal circumstances, the connection is kept alive by continuously
	// sending a keepalive request to the SSH gateway. When the context is
	// canceled, the keepalive loop is stopped, and the connection will break
	// after it has been idle for this duration, if configured.
	ConnectionDrainTimeout time.Duration

	// RegisteredFunc is called when the initial registration has completed.
	//
	// The function must be careful not to take too long or become deadlocked, or
	// else the SSH connection can starve.
	RegisteredFunc func()

	// HeartbeatedFunc is called on each heartbeat after registration.
	//
	// The function must be careful not to take too long or become deadlocked, or
	// else the SSH connection can starve.
	HeartbeatedFunc func()
}

RegisterOptions contains required configuration for the registration.

type Retirer

type Retirer struct {
	ATCEndpoint *rata.RequestGenerator
	HTTPClient  *http.Client
}

func (*Retirer) Retire

func (l *Retirer) Retire(ctx context.Context, worker atc.Worker) error

type Sweeper

type Sweeper struct {
	ATCEndpoint *rata.RequestGenerator
	HTTPClient  *http.Client
}

func (*Sweeper) Sweep

func (l *Sweeper) Sweep(ctx context.Context, worker atc.Worker, resourceAction string) ([]byte, error)

type WorkerStatus

type WorkerStatus struct {
	ATCEndpoint      *rata.RequestGenerator
	HTTPClient       *http.Client
	ContainerHandles []string
	VolumeHandles    []string
}

func (*WorkerStatus) WorkerStatus

func (l *WorkerStatus) WorkerStatus(ctx context.Context, worker atc.Worker, resourceAction string) error