knox

package module
v0.0.0-...-3492e0a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 16, 2022 License: Apache-2.0 Imports: 20 Imported by: 0

README

Knox -- the high level overview

Knox is a service for storing and rotation of secrets, keys, and passwords used by other services.

The Problem Knox is Meant to Solve

Pinterest has a plethora of keys or secrets doing things like signing cookies, encrypting data, protecting our network via TLS, accessing our AWS machines, communicating with our third parties, and many more. If these keys become compromised, rotating (or changing our keys) used to be a difficult process generally involving a deploy and likely a code change. Keys/secrets within Pinterest were stored in git repositories. This means they were copied all over our company's infrastructure and present on many of our employees laptops. There was no way to audit who accessed or who has access to the keys. Knox was built to solve these problems.

The goals of Knox are:

  • Ease of use for developers to access/use confidential secrets, keys, and credentials
  • Confidentiality for secrets, keys, and credentials
  • Provide mechanisms for key rotation in case of compromise
  • Create audit log to keep track of what systems and users access confidential data

Read more at https://github.com/pinterest/knox/wiki

Getting knox set up

The first step is to install Go (or use Docker, see below). We require Go >= 1.6 or Go 1.5 with the vendor flag enabled (GO15VENDOREXPERIMENT=1). For instructions on setting up Go, please visit https://golang.org/doc/install

After Go is set up (including a $GOPATH directory that will store your workspace), please run go get -d github.com/pinterest/knox to get the latest version of the knox code.

To compile the devserver and devclient binaries, run go install github.com/pinterest/knox/cmd/dev_server and go install github.com/pinterest/knox/cmd/dev_client. These can be directly executed, the dev_client expects the server to be running on a localhost. By default, the client uses mTLS with a hardcoded signed cert given for example.com for machine authentication and had github authentication enabled for users.

To start your server run:

$GOPATH/bin/dev_server

For using this client as a user, generate a token via these instructions https://help.github.com/articles/creating-an-access-token-for-command-line-use/ with read:org permissions. This token will be able to get your username and the organization you belong to. With the dev_server running you can now create your first knox key.

export KNOX_USER_AUTH=<insert generated github token here>
echo -n "My first knox secret" | $GOPATH/bin/dev_client create test_service:first_secret

You can retrieve the secret using:

$GOPATH/bin/dev_client get test_service:first_secret

You can see all key IDs using:

$GOPATH/bin/dev_client keys

To see all available commands run:

$GOPATH/bin/dev_client help

For production usage, I recommend making your own client, renaming it knox, and moving it into you $PATH for ease of use.

For more information on interacting with knox, use knox help or go to https://github.com/pinterest/knox/wiki/Knox-Client

Knox with Docker

You can run a Docker container to get knox set up, instead of installing Go on your host.

git clone https://github.com/pinterest/knox.git
cd knox
docker run --name knox --rm -v "$PWD":/go/src/github.com/pinterest/knox -it golang /bin/bash

This will run a bash shell into the container, mounting a local copy of knox in the go source path.

You can refer back to the section "Getting knox set up" to set up knox.

Documentation

Overview

Package knox is a package to provide the basic types to be used across client and server.

Index

Constants

View Source 
const (
	// Unknown represents a bad PrincipalType that cannot be marshaled
	Unknown PrincipalType = -1
	// User represents a single LDAP User.
	User = iota
	// UserGroup represents an LDAP security group.
	UserGroup
	// Machine represents the host of a machine.
	Machine
	// MachinePrefix represents a prefix to match multiple machines.
	MachinePrefix
	// Service represents a service via SPIFFE ID.
	Service
	// ServicePrefix represents a prefix to match multiple SPIFFE IDs.
	ServicePrefix
)
View Source 
const (
	OKCode = iota
	InternalServerErrorCode
	KeyIdentifierExistsCode
	KeyVersionDoesNotExistCode
	KeyIdentifierDoesNotExistCode
	UnauthenticatedCode
	UnauthorizedCode
	NotYetImplementedCode
	NotFoundCode
	NoKeyIDCode
	NoKeyDataCode
	BadRequestDataCode
	BadKeyFormatCode
	BadPrincipalIdentifier
)

These are the error codes for use in server responses.

Variables

View Source 
var (
	ErrACLDuplicateEntries = fmt.Errorf("Duplicate entries in ACL")
	ErrACLContainsNone     = fmt.Errorf("ACL contains None access")
	ErrACLEmptyPrincipal   = fmt.Errorf("Principals of type user, user group, machine, or machine prefix may not be empty.")

	ErrACLInvalidService               = fmt.Errorf("Service is invalid, must conform to 'spiffe://<domain>/<path>' format.")
	ErrACLInvalidServicePrefixURL      = fmt.Errorf("Service prefix is invalid URL, must conform to 'spiffe://<domain>/<path>/' format.")
	ErrACLInvalidServicePrefixNoSlash  = fmt.Errorf("Service prefix had no trailing slash, must conform to 'spiffe://<domain>/<path>/' format.")
	ErrACLInvalidServicePrefixTooShort = fmt.Errorf("Service prefix too short, path of namespace for prefix needs to be longer.")

	ErrInvalidKeyID       = fmt.Errorf("KeyID can only contain alphanumeric characters, colons, and underscores.")
	ErrInvalidVersionHash = fmt.Errorf("Hash does not match")

	ErrInactiveToPrimary = fmt.Errorf("Version must be Active to promote to Primary")
	ErrPrimaryToActive   = fmt.Errorf("Primary Key can not be demoted. Specify Active key to promote.")
	ErrPrimaryToInactive = fmt.Errorf("Version must be Active to demote to Inactive")

	ErrMulitplePrimary = fmt.Errorf("More than one Primary key")
	ErrSameVersionID   = fmt.Errorf("Repeated Version ID")

	ErrInvalidStatus      = fmt.Errorf("Invalid Status")
	ErrKeyVersionNotFound = fmt.Errorf("Key version not found")
	ErrKeyIDNotFound      = fmt.Errorf("KeyID not found")
	ErrKeyExists          = fmt.Errorf("Key Exists")
)

Functions

func GetBackoffDuration 

func GetBackoffDuration(attempt int) time.Duration

GetBackoffDuration returns a time duration to sleep based on the attempt #.

func Register 

func Register(keyID string) ([]byte, error)

Register registers the given keyName with knox. If the operation fails, it returns an error.

Types

type ACL 

type ACL []Access

ACL is a list of access information that provides authorization information for a specific key.

func (ACL) Add 

func (acl ACL) Add(a Access) ACL

Add appends an access to the ACL. It does so by overwriting any existing access that principal or group may have had.

func (ACL) Validate 

func (acl ACL) Validate() error

Validate ensures the ACL is of valid form. Not specifying the same group or id more than once.

type APIClient 

type APIClient interface {
	GetKey(keyID string) (*Key, error)
	CreateKey(keyID string, data []byte, acl ACL) (uint64, error)
	GetKeys(keys map[string]string) ([]string, error)
	DeleteKey(keyID string) error
	GetACL(keyID string) (*ACL, error)
	PutAccess(keyID string, acl ...Access) error
	AddVersion(keyID string, data []byte) (uint64, error)
	UpdateVersion(keyID, versionID string, status VersionStatus) error
	CacheGetKey(keyID string) (*Key, error)
	NetworkGetKey(keyID string) (*Key, error)
	GetKeyWithStatus(keyID string, status VersionStatus) (*Key, error)
	CacheGetKeyWithStatus(keyID string, status VersionStatus) (*Key, error)
	NetworkGetKeyWithStatus(keyID string, status VersionStatus) (*Key, error)
}

APIClient is an interface that talks to the knox server for key management.

func NewClient 

func NewClient(host string, client HTTP, authHandler func() string, keyFolder, version string) APIClient

NewClient creates a new client to connect to talk to Knox.

type Access 

type Access struct {
	Type       PrincipalType `json:"type"`
	ID         string        `json:"id"`
	AccessType AccessType    `json:"access"`
}

Access is a specific access grant as a part of an ACL specifying one principal's or a group of principals' granted acccess.

type AccessType 

type AccessType int

AccessType represents what kind of Access is granted in a key's ACL. 

const (
	// None denotes no access.
	None AccessType = iota
	// Read denotes the ability to read key data.
	Read
	// Write denotes the ability to add key versions and perform rotation.
	Write
	// Admin denotes the ability to delete the key and modify the ACL.
	Admin
)

func (AccessType) CanAccess 

func (s AccessType) CanAccess(resource AccessType) bool

CanAccess uses a principal's AccessType to determine if the principal can access a given resource.

func (AccessType) MarshalJSON 

func (s AccessType) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON representation of an AccessType.

func (*AccessType) UnmarshalJSON 

func (s *AccessType) UnmarshalJSON(b []byte) error

UnmarshalJSON parses JSON input to set an AccessType.

type Client 

type Client interface {
	// GetPrimary returns the primary key version for the knox key.
	// This should be used for sending relationships like signing, encrypting, or api secrets
	GetPrimary() string
	// GetActive returns all of the active key versions for the knox key.
	// This should be used for receiving relationships like verifying or decrypting.
	GetActive() []string
	// GetKeyObject returns the full key object, including versions, ACLs, and other attributes.
	GetKeyObject() Key
}

Client is an interface for interacting with a specific knox key

func NewFileClient 

func NewFileClient(keyID string) (Client, error)

NewFileClient creates a file watcher knox client for the keyID given (it refreshes every ten seconds). This client calls `knox register` to cache the key locally on the file system.

func NewMock 

func NewMock(primary string, active []string) Client

NewMock is a knox Client to be used for testing.

type HTTP 

type HTTP interface {
	Do(req *http.Request) (*http.Response, error)
}

type HTTPClient 

type HTTPClient struct {
	// Host is used as the host for http connections
	Host string
	//AuthHandler returns the authorization string for authenticating to knox. Users should be prefixed by 0u, machines by 0m. On fail, return empty string.
	AuthHandler func() string
	// KeyFolder is the location of cached keys on the file system. If empty, does not check for cached keys.
	KeyFolder string
	// Client is the http client for making network calls
	Client HTTP
	// Version is the current client version, useful for debugging and sent as a header
	Version string
}

HTTPClient is a client that uses HTTP to talk to Knox.

func MockClient 

func MockClient(host string) *HTTPClient

MockClient builds a client that ignores certs and talks to the given host.

func (*HTTPClient) AddVersion 

func (c *HTTPClient) AddVersion(keyID string, data []byte) (uint64, error)

AddVersion adds a key version to a specific key.

func (*HTTPClient) CacheGetKey 

func (c *HTTPClient) CacheGetKey(keyID string) (*Key, error)

CacheGetKey gets the key from file system cache.

func (*HTTPClient) CacheGetKeyWithStatus 

func (c *HTTPClient) CacheGetKeyWithStatus(keyID string, status VersionStatus) (*Key, error)

CacheGetKeyWithStatus gets the key with status from file system cache.

func (*HTTPClient) CreateKey 

func (c *HTTPClient) CreateKey(keyID string, data []byte, acl ACL) (uint64, error)

CreateKey creates a knox key with given keyID data and ACL.

func (HTTPClient) DeleteKey 

func (c HTTPClient) DeleteKey(keyID string) error

DeleteKey deletes a key from Knox.

func (*HTTPClient) GetACL 

func (c *HTTPClient) GetACL(keyID string) (*ACL, error)

GetACL gets a knox key by keyID.

func (*HTTPClient) GetKey 

func (c *HTTPClient) GetKey(keyID string) (*Key, error)

GetKey gets a knox key by keyID.

func (*HTTPClient) GetKeyWithStatus 

func (c *HTTPClient) GetKeyWithStatus(keyID string, status VersionStatus) (*Key, error)

GetKeyWithStatus gets a knox key by keyID and status (leverages cache).

func (*HTTPClient) GetKeys 

func (c *HTTPClient) GetKeys(keys map[string]string) ([]string, error)

GetKeys gets all Knox (if empty map) or gets all keys in map that do not match key version hash.

func (*HTTPClient) NetworkGetKey 

func (c *HTTPClient) NetworkGetKey(keyID string) (*Key, error)

NetworkGetKey gets a knox key by keyID and only uses network without the caches.

func (*HTTPClient) NetworkGetKeyWithStatus 

func (c *HTTPClient) NetworkGetKeyWithStatus(keyID string, status VersionStatus) (*Key, error)

NetworkGetKeyWithStatus gets a knox key by keyID and given version status (always calls network).

func (*HTTPClient) PutAccess 

func (c *HTTPClient) PutAccess(keyID string, a ...Access) error

PutAccess will add an ACL rule to a specific key.

func (*HTTPClient) UpdateVersion 

func (c *HTTPClient) UpdateVersion(keyID, versionID string, status VersionStatus) error

UpdateVersion either promotes or demotes a specific key version.

type Key 

type Key struct {
	ID          string         `json:"id"`
	ACL         ACL            `json:"acl"`
	VersionList KeyVersionList `json:"versions"`
	VersionHash string         `json:"hash"`
	Path        string         `json:"path,omitempty"`
}

Key represents the Primary element of Knox.

func (Key) Validate 

func (k Key) Validate() error

Validate calls makes sure all attributes of key are in good state.

type KeyVersion 

type KeyVersion struct {
	ID           uint64        `json:"id"`
	Data         []byte        `json:"data"`
	Status       VersionStatus `json:"status"`
	CreationTime int64         `json:"ts"`
}

KeyVersion is a specific version of a Key. All attributes should be immutable except status.

func NewMockKeyVersion 

func NewMockKeyVersion(keydata []byte, status VersionStatus) KeyVersion

NewMockKeyVersion creates a Knox KeyVersion to be used for testing

type KeyVersionList 

type KeyVersionList []KeyVersion

KeyVersionList represents the list of versions of a key. This will grow as the key is rotated.

func (KeyVersionList) GetActive 

func (kvl KeyVersionList) GetActive() KeyVersionList

GetActive returns the active keys in a KeyVersionList.

func (KeyVersionList) GetPrimary 

func (kvl KeyVersionList) GetPrimary() *KeyVersion

GetPrimary returns the primary key in a KeyVersionList.

func (KeyVersionList) Hash 

func (kvl KeyVersionList) Hash() string

Hash computes the Sha256 hash of the ordered key versions. The hash ordering is the Primary version id followed by all Active version id in numeric order.

func (KeyVersionList) Len 

func (kvl KeyVersionList) Len() int

Len returns the length of the key version list.

func (KeyVersionList) Less 

func (kvl KeyVersionList) Less(i, j int) bool

Less determines where a key version is in an ordered list.

func (KeyVersionList) Swap 

func (kvl KeyVersionList) Swap(i, j int)

Swap swaps two elements in the list

func (KeyVersionList) Update 

func (kvl KeyVersionList) Update(versionID uint64, s VersionStatus) (KeyVersionList, error)

Update changes the status of a particular key version. It also updates any other key versions that need to be updated. Acceptable changes are Active -> Primary, Active -> Inactive, and Inactive -> Active.

func (KeyVersionList) Validate 

func (kvl KeyVersionList) Validate() error

Validate checks that key versions are unique and that there is exactly one Primary key.

type Principal 

type Principal interface {
	CanAccess(ACL, AccessType) bool
	GetID() string
	Type() string
}

Principal is a person, machine, or process that accesses an object. This interface is currently defined for people and machines.

func NewPrincipalMux 

func NewPrincipalMux(defaultPrincipal Principal, allPrincipals map[string]Principal) Principal

NewPrincipalMux returns a Principal that represents many principals.

type PrincipalMux 

type PrincipalMux struct {
	// contains filtered or unexported fields
}

PrincipalMux provides a Principal Interface over multiple Principals.

func (PrincipalMux) CanAccess 

func (p PrincipalMux) CanAccess(acl ACL, accessType AccessType) bool

CanAccess will check the principals in order of adding, and the first Principal that provides at least the AccessType requested will be used.

func (PrincipalMux) Default 

func (p PrincipalMux) Default() Principal

Default returns the first registered Principal.

func (PrincipalMux) GetID 

func (p PrincipalMux) GetID() string

GetID returns the ID of the default principal.

func (PrincipalMux) GetIDs 

func (p PrincipalMux) GetIDs() []string

GetIDs returns all registered IDs from the principals that are muxed.

func (PrincipalMux) Type 

func (p PrincipalMux) Type() string

Type returns the underlying type of a principal, for logging/debugging purposes.

type PrincipalType 

type PrincipalType int

PrincipalType is an attribute of ACLs that specifies what type of Principal is represented. This allows for users and machines to be bucketed together.

func (PrincipalType) IsValidPrincipal 

func (s PrincipalType) IsValidPrincipal(id string, extraValidators []PrincipalValidator) error

IsValidPrincipal verifies that the given id string matches our expectations for what a principal should look like given the principal type. For example, a service principal should be a valid SPIFFE ID.

func (PrincipalType) MarshalJSON 

func (s PrincipalType) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON representation of an PrincipalType.

func (*PrincipalType) UnmarshalJSON 

func (s *PrincipalType) UnmarshalJSON(b []byte) error

UnmarshalJSON parses JSON input to set an PrincipalType.

type PrincipalValidator 

type PrincipalValidator func(pt PrincipalType, id string) error

A PrincipalValidator is a function that applies to a principal type and string, and validates that the string is a valid principal for the given type.

func ServicePrefixPathComponentsValidator 

func ServicePrefixPathComponentsValidator(minPathComponents int) PrincipalValidator

ServicePrefixPathComponentsValidator is an extra validator that can be applied to ensure that service prefixes have a certain minimum length, e.g. to prevent allow the prefix to be a full domain.

type Response 

type Response struct {
	Status    string      `json:"status"`
	Code      int         `json:"code"`
	Host      string      `json:"host"`
	Timestamp int64       `json:"ts"`
	Message   string      `json:"message"`
	Data      interface{} `json:"data"`
}

Response is the format for responses from the api server.

type VersionStatus 

type VersionStatus int

VersionStatus is an enum to determine that state of a single Key Version. This is related to key rotation. 

const (
	// Primary is the main key version. There is exactly one in a given KeyVersionList.
	Primary VersionStatus = iota
	// Active represents Key Versions still in use, but not Primary.
	Active
	// Inactive represents Key Versions no longer in use.
	Inactive
)

func (VersionStatus) MarshalJSON 

func (s VersionStatus) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON representation of an VersionStatus.

func (*VersionStatus) UnmarshalJSON 

func (s *VersionStatus) UnmarshalJSON(b []byte) error

UnmarshalJSON parses JSON input to set an VersionStatus.

Source Files

View all Source files

Directories

Path Synopsis
cmd
dev_client
dev_server
migrate_db
Package log implements a simple logging package.
 Package log implements a simple logging package.
auth
keydb

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.