natsbackend

package module
v1.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 3, 2024 License: MPL-2.0 Imports: 20 Imported by: 0

README

Contributors Forks Stargazers Issues MPL-2.0 License


Logo

vault-plugin-secrets-nats

vault-plugin-secrets-nats extends Hashicorp Vault with a secrets engine for NATS.

About The Project

vault-plugin-secrets-nats is a Hashicorp Vault plugin that extends Vault with a secrets engine for NATS for Nkey/JWT auth. It is capable of generating NATS credentials for operators, accounts and users. The generated credentials are stored in Vault and can be revoked at any time. The plugin is also able to push the generated credentials to a NATS account server.

A similar project is the nsc tool by NATS but the nsc tool doesn't provide a way to store the generated credentials other than file based. The nsc tool is not automated and heavily relies on manual steps.

Features

  • Manage NATS nkey and jwt for operators, accounts and users
  • Give access to user creds files
  • Push generated credentials to a NATS account server

Getting Started

The nats secrets engine generates NATS credentials dynamically. The plugin supports several resources, including: operators, accounts, users, NKeys, JWTs and creds, as well as signing keys for operators an accounts.

There is a command structure to create, read update and delete operators, accounts, users and permissions based on entity paths. Please read the official NATS documentation to understand the concepts of operators, accounts and users as well as the authentication process. A hand full of resources can be defined within the vault plugin:

The resource of type issue represent entities that result in generation of nkey and JWTs.

Entity path Description Operations
issue/operator List operator issues list
issue/operator/<operator>/account List account issues list
issue/operator/<operator>/account/<account>/user List user issues within an account list
issue/operator/<operator> Manage operator issues. See the operator section for more information. write, read, delete
issue/operator/<operator>/account/<account> Manage account issues. See the account section for more information. write, read, delete
issue/operator/<operator>/account/<account>/user/<name> Manage user issues within an account. See the user section for more information. write, read, delete

The resources of type creds represent user credentials that can be used to authenticate against a NATS server.

Entity path Description Operations
creds/operator/<operator>account/<account>/user List user creds List
creds/operator/<operator>account/<account>/user/<user> Manage user creds write, read, update

Resouces of type nkey are either be generated by issues or are imported and referenced by issues during their creation.

Entity path Description Operations
nkey/operator List operator nkeys list
nkey/operator/<operator>/signing List operators' signing nkeys list
nkey/operator/<operator>/account List account nkeys list
nkey/operator/<operator>/account/<account>/signing List accounts' signing nkeys list
nkey/operator/<operator>/account/<account>/user List user nkeys list
nkey/operator/<operator> Manage operator nkey write, read, delete
nkey/operator/<operator>/signing/<key> Manage operator signing nkeys write, read, delete
nkey/operator/<operator>account/<account> Manage accounts' nkey write, read, delete
nkey/operator/<operator>account/<account>/signing/<key> Manage accounts' signing nkeys write, read, delete
nkey/operator/<operator>account/<account>/user/<user> Manage user nkey write, read, delete

Resource of type 'jwt' are either be generated by issues or are imported and referenced by issues during their creation.

Entity path Description Operations
jwt/operator List operator JWTs list
jwt/operator/<operator>/account List account JWTs list
jwt/operator/<operator>/account/<account>/user List user JWTs list
jwt/operator/<operator> Manage operator JWT write, read, delete
jwt/operator/<operator>account/<account> Manage accounts' JWTs write, read, delete
jwt/operator/<operator>account/<account>/user/<user> Manage user JWT write, read, delete

✔️ Prerequisites

ex TODO

⚙️ Configuration

There are arguments that can be passed to the paths for issue/ (operator, account, user), creds/, jwt/ and nkey/.

Issues

Issues can be created with an imported nkey. If the nkey is not present during the creation of the issue, a new nkey will be generated. Note: if you don't provide any claims for an operator, account or user, the plugin will generate a default set of claims. The default claims are set to "you are not allowed to do anything".

Operator
Key Type Required Default Description
syncAccountServer bool false false If set to true, the plugin will push the generated credentials to the configured account server.
claims json string false {} Claims to be added to the operator's JWT. See pkg/claims/operator/v1alpha1/api.go
Account
Key Type Required Default Description
useSigningKey string false "" Operator signing key's name, e.g. "opsk1"
claims json string false {} Claims to be added to the account's JWT. See pkg/claims/account/v1alpha1/api.go
User
Key Type Required Default Description
useSigningKey bool false false Account signing key's name, e.g. "opsk1"
claims json string false {} Claims to be added to the user's JWT. See pkg/claims/user/v1alpha1/api.go
Nkey
Key Type Required Default Description
seed string false "" Seed to import. If not set, then a new one is created
JWT
Key Type Required Default Description
jwt string false "" JWT to import. If not set, then a new one is created
Creds
Key Type Required Default Description
creds string false "" Creds file to import. If not set, then a new one is created
📤 System account specific configuration

This section describes the configuration options that are specific to the system account.

The default name of the system account is sys. If you want to use a different name, you can set the systemAccount configuration option in the operator. Within the sys account the only user that is capable of pushing credentials to the account server is the default-push user.

See the example/sysaccount directory for an example configuration of both sys account and default-push user.

🎯 Installation and Setup

In order to use this plugin you need to register it with Vault. Configure your vault server to have a valid plugins_directory configuration.

Note: you might want to set api_addr to your listening address and disable_mlock to true in the vault configuration to be able to use the plugin.

Install from release

Download the latest stable release from the release page and put it into the plugins_directory of your vault server.

To use a vault plugin you need the plugin's sha256 sum. You can download the file vault-plugin-secrets-nats.sha256 file from the release, obtain it with sha256sum vault-plugin-secrets-nats or look within the OCI image at /etc/vault/vault_plugins_checksums/vault-plugin-secrets-nats.sha256.

Example how to register the plugin:

SHA256SUM=$(sha256sum vault-plugin-secrets-nats | cut -d' ' -f1)
vault plugin register -sha256 ${SHA256SUM} secret vault-plugin-secrets-nats
vault secrets enable -path=nats-secrets vault-plugin-secrets-nats

Note: you might use the -tls-skip-verify flag if you are using a self-signed certificate.

Install from OCI image using bank-vaults in Kubernetes

This project provides a custom built vault OCI image that includes the vault-plugin-secrets-nats plugin. See here for available versions. The plugins_directory must be set to /etc/vault/vault_plugins in the vault configuration.

This describes the steps to install the plugin using the bank-vaults operator. See here for more information. Define the custom vault image in the Vault custom resource and configure

apiVersion: "vault.banzaicloud.com/v1alpha1"
kind: "Vault"
metadata:
  name: "myVault"
spec:
  size: 1
  # Use the custom vault image containing the NATS secrets plugin
  image: ghcr.io/edgefarm/vault-plugin-secrets-nats/vault-with-nats-secrets:1.6.0
  config:
    disable_mlock: true
    plugin_directory: "/etc/vault/vault_plugins"
    listener:
      tcp:
        address: "0.0.0.0:8200"
    api_addr: "https://0.0.0.0:8200"
  externalConfig:
    plugins:
    - plugin_name: vault-plugin-secrets-nats
      command: vault-plugin-secrets-nats --tls-skip-verify --ca-cert=/vault/tls/ca.crt
      sha256: e03f5771e7806296e9f77ccffde203ce2ea5ce0728df6d88054bb7939c6dc392
      type: secret
    secrets:
    - path: nats-secrets
      type: plugin
      plugin_name: vault-plugin-secrets-nats
      description: NATS secrets backend
  # ...

See the fule dev/manifests/vault/vault.yaml for a full example of a Vault custom resource that can be used by the vault-operator.

🧪 Testing

To test the plugin in a production like environment you can spin up a local kind cluster that runs a production vault server with the plugin enabled and a NATS server the plugin writes account information to.

Note: you need to have kind and devspace installed.

The first step is to spin up the cluster with everything installed.

# Create the cluster
$ devspace run create-kind-cluster

# Deploy initial stuff like ingress and cert-manager
$ devspace run-pipeline init 

# Deploy the vault-operator and vault instance
$ devspace run-pipeline deploy-vault

# Wait for the vault pods get ready
$ kubectl get pods -n vault 

# Check if the plugin is successfully loaded
$ kubectl port-forward -n vault svc/vault 8200:8200 &
$ PID=$!
$ export VAULT_ADDR=https://127.0.0.1:8200
$ VAULT_TOKEN=$(kubectl get secrets bank-vaults -n vault -o jsonpath='{.data.vault-root}' | base64 -d)
$ echo $VAULT_TOKEN | vault login -
$ vault secrets list
Handling connection for 8200
Path             Type                         Accessor                              Description
----             ----                         --------                              -----------
cubbyhole/       cubbyhole                    cubbyhole_ec217496                    per-token private secret storage
identity/        identity                     identity_9123b895                     identity store
nats-secrets/    vault-plugin-secrets-nats    vault-plugin-secrets-nats_d8584dcc    NATS secrets backend
sys/             system                       system_5bd0e10f                       system endpoints used for control, policy and debugging
$ pkill $PID

# Deploy the NATS server
$ devspace run-pipeline deploy-nats

# Wait for the NATS server to be ready
$ kubectl get pods -n nats

Once this is working create a account and a user and act as a third party that uses the creds outside the cluster.

# Create the account and user and get the creds for the user
$ devspace run-pipeline create-custom-nats-account
$ kubectl port-forward -n nats svc/nats 4222:4222 &
$ PID=$!

# Publish and subscribe using the creds previously fetched
$ docker run -it -d --rm --name nats-subscribe --network host -v $(pwd)/.devspace/creds/creds:/creds natsio/nats-box:0.13.4 nats sub -s nats://localhost:4222 --creds /creds foo 
$ docker run --rm -d -it --name nats-publish --network host -v $(pwd)/.devspace/creds/creds:/creds natsio/nats-box:0.13.4 nats pub -s nats://localhost:4222 --creds /creds foo --count 3 "Message {{Count}} @ {{Time}}"

# Log output shows that authenticating with the creds file works for pub and sub
$ docker logs nats-subscribe
14:49:35 Subscribing on foo 
[#1] Received on "foo"
Message 1 @ 2:49PM

[#2] Received on "foo"
Message 2 @ 2:49PM

[#3] Received on "foo"
Message 3 @ 2:49PM

# Cleanup
$ docker kill nats-subscribe
$ pkill $PID

💡 Example

Read this section to learn how to use vault-plugin-secrets-nats by trying out the example. See the example directory for a full example. The example runs a locally running Vault server and a NATS server.

An operator and a sys account is created. Both are using signing keys. A sys account user called default-push is created that is used to push the credentials to the NATS account server. The NATS server is configured to use the generated credentials. After the NATS server is up and running a new "normal" account and a user is created and pushed to the NATS server. The user is then able to connect to the NATS server.

Note: please make sure that you have docker installed as the example starts a local NATS server using docker.

🛠️ Setup

To use the plugin, you must first enable it with Vault. This example mounts the plugin at the path nats-secrets:

First run make to start a Vault server in dev mode that is pre-configured to use the plugin.

$ make

Then, enable the plugin:

$ export VAULT_ADDR='http://127.0.0.1:8200'
$ vault secrets enable -path=nats-secrets vault-plugin-secrets-nats
Success! Enabled the vault-plugin-secrets-nats secrets engine at: nats-secrets/

🏁 Run the example

$ cd examples
$ ./config.sh
> Creating NATS resources (operator and sysaccount)
Success! Data written to: nats-secrets/issue/operator/myop
Success! Data written to: nats-secrets/issue/operator/myop/account/sys
Success! Data written to: nats-secrets/issue/operator/myop/account/sys/user/default-push
> Generate NATS server config with preloaded operator and sys account settings
> Starting up NATS server
9402e7608bfe8bc391c862eb01f4dbac19e16210a431fb9d84384e009f013a3d
a5bd1e08562382aaf6b40f35203afd479bfa847fddf72a617dbd083446863071
> Creating normal account and user
Success! Data written to: nats-secrets/issue/operator/myop/account/myaccount
Success! Data written to: nats-secrets/issue/operator/myop/account/myaccount/user/user
> Exporting user creds file
> Publishing using user creds file
12:57:09 Published 3 bytes to "foo"
> Cleaning up...
nats
nats
> done.

🐞 Debugging

The recommended way to debug this plugin is to use write unit tests and debug them as standard go tests. If you like to debug the plugin in a running Vault instance you can use the following steps:

  1. make
  2. make enable
  3. Attach to the process using your favorite debugger
  4. Use vault CLI to interact with the plugin
  5. Debug the plugin

🤝🏽 Contributing

Code contributions are very much welcome.

  1. Fork the Project
  2. Create your Branch (git checkout -b AmazingFeature)
  3. Commit your Changes (`git commit -m 'Add some AmazingFeature")
  4. Push to the Branch (git push origin AmazingFeature)
  5. Open a Pull Request targetting the main branch.

🫶 Acknowledgements

Thanks for the NATS developers for providing a really great way of solving many problems with communication.

Also, thanks to the Vault developers for providing a great way of managing secrets and a great plugin system.

Documentation

Index

Constants

View Source 
const (
	// DefaultSysAccountName is the name of the system account
	DefaultSysAccountName = "sys"

	// DefaultSysUser is the name of the system user
	DefaultPushUser = "default-push"
)
View Source 
const (

	// Parameters
	InvalidParametersError = "invalid parameters"
	DecodeFailedError      = "could not decode parameters"

	// ISSUE
	AddingIssueFailedError  = "adding issue failed"
	ReadingIssueFailedError = "reading issue failed"
	IssueNotFoundError      = "issue not found"
	DeleteIssueFailedError  = "deleting issue failed"
	ListIssuesFailedError   = "listing issues failed"

	// JWT
	AddingJWTFailedError  = "adding jwt failed"
	ReadingJWTFailedError = "reading jwt failed"
	ListJWTsFailedError   = "listing jwts failed"
	DeleteJWTFailedError  = "deleting jwt failed"
	JwtNotFoundError      = "jwt not found"

	// NKEY
	AddingNkeyFailedError  = "adding nkey failed"
	ReadingNkeyFailedError = "reading nkey failed"
	ListNkeysFailedError   = "listing nkeys failed"
	DeleteNkeyFailedError  = "deleting nkey failed"
	NkeyNotFoundError      = "nkey not found"

	// CREDS
	AddingCredsFailedError  = "adding creds failed"
	ReadingCredsFailedError = "reading creds failed"
	ListCredsFailedError    = "listing credss failed"
	DeleteCredsFailedError  = "deleting creds failed"
	CredsNotFoundError      = "creds not found"
)

Variables

This section is empty.

Functions

func Factory 

func Factory(ctx context.Context, conf *logical.BackendConfig) (logical.Backend, error)

func IsNatsUrl 

func IsNatsUrl(url string) bool

Types

type AccountResolverAction added in v1.5.0 

type AccountResolverAction string
const (
	AccountResolverActionPush   AccountResolverAction = "push"
	AccountResolverActionDelete AccountResolverAction = "delete"
)

type AccountServerStatus 

type AccountServerStatus struct {
	Synced   bool  `json:"synced"`
	LastSync int64 `json:"lastSync"`
}

type CredsData 

type CredsData struct {
	CredsStorage
}

CredsData represents the the data returned by a Creds operation

type CredsParameters 

type CredsParameters struct {
	Operator string `json:"operator,omitempty"`
	Account  string `json:"account,omitempty"`
	User     string `json:"user,omitempty"`
	CredsStorage
}

CredsParameters represents the parameters for a Creds operation

type CredsStorage 

type CredsStorage struct {
	Creds string `json:"creds"`
}

JWTStorage represents a Creds stored in the backend

type IssueAccountData 

type IssueAccountData struct {
	Operator      string                 `json:"operator"`
	Account       string                 `json:"account"`
	UseSigningKey string                 `json:"useSigningKey"`
	Claims        v1alpha1.AccountClaims `json:"claims"`
	Status        IssueAccountStatus     `json:"status"`
}

type IssueAccountParameters 

type IssueAccountParameters struct {
	Operator      string                 `json:"operator"`
	Account       string                 `json:"account"`
	UseSigningKey string                 `json:"useSigningKey,omitempty"`
	Claims        v1alpha1.AccountClaims `json:"claims,omitempty"`
}

IssueAccountParameters is the user facing interface for configuring an account issue. Using pascal case on purpose. +k8s:deepcopy-gen=true

func (*IssueAccountParameters) DeepCopy 

func (in *IssueAccountParameters) DeepCopy() *IssueAccountParameters

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new IssueAccountParameters.

func (*IssueAccountParameters) DeepCopyInto 

func (in *IssueAccountParameters) DeepCopyInto(out *IssueAccountParameters)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type IssueAccountStatus 

type IssueAccountStatus struct {
	Account       IssueStatus         `json:"account"`
	AccountServer AccountServerStatus `json:"accountServer"`
}

type IssueAccountStorage 

type IssueAccountStorage struct {
	Operator      string                 `json:"operator"`
	Account       string                 `json:"account"`
	UseSigningKey string                 `json:"useSigningKey"`
	Claims        v1alpha1.AccountClaims `json:"claims"`
	Status        IssueAccountStatus     `json:"status"`
}

type IssueOperatorData 

type IssueOperatorData struct {
	Operator            string                    `json:"operator"`
	CreateSystemAccount bool                      `json:"createSystemAccount"`
	SyncAccountServer   bool                      `json:"syncAccountServer"`
	Claims              operatorv1.OperatorClaims `json:"claims"`
	Status              IssueOperatorStatus       `json:"status"`
}

type IssueOperatorParameters 

type IssueOperatorParameters struct {
	Operator            string                    `json:"operator"`
	CreateSystemAccount bool                      `json:"createSystemAccount,omitempty"`
	SyncAccountServer   bool                      `json:"syncAccountServer,omitempty"`
	Claims              operatorv1.OperatorClaims `json:"claims,omitempty"`
}

IssueOperatorParameters +k8s:deepcopy-gen=true

func (*IssueOperatorParameters) DeepCopy 

func (in *IssueOperatorParameters) DeepCopy() *IssueOperatorParameters

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new IssueOperatorParameters.

func (*IssueOperatorParameters) DeepCopyInto 

func (in *IssueOperatorParameters) DeepCopyInto(out *IssueOperatorParameters)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type IssueOperatorStatus 

type IssueOperatorStatus struct {
	Operator          IssueStatus `json:"operator"`
	SystemAccount     IssueStatus `json:"systemAccount"`
	SystemAccountUser IssueStatus `json:"systemAccountUser"`
}

type IssueOperatorStorage 

type IssueOperatorStorage struct {
	Operator            string                    `json:"operator"`
	CreateSystemAccount bool                      `json:"createSystemAccount"`
	SyncAccountServer   bool                      `json:"syncAccountServer"`
	Claims              operatorv1.OperatorClaims `json:"claims"`
}

type IssueStatus 

type IssueStatus struct {
	Nkey bool `json:"nkey"`
	JWT  bool `json:"jwt"`
}

type IssueUserData 

type IssueUserData struct {
	Operator      string              `json:"operator"`
	Account       string              `json:"account"`
	User          string              `json:"user"`
	UseSigningKey string              `json:"useSigningKey"`
	Claims        v1alpha1.UserClaims `json:"claims"`
	Status        IssueUserStatus     `json:"status"`
}

type IssueUserParameters 

type IssueUserParameters struct {
	Operator      string              `json:"operator"`
	Account       string              `json:"account"`
	User          string              `json:"user"`
	UseSigningKey string              `json:"useSigningKey,omitempty"`
	Claims        v1alpha1.UserClaims `json:"claims,omitempty"`
}

IssueUserParameters is the user facing interface for configuring a user issue. Using pascal case on purpose. +k8s:deepcopy-gen=true

func (*IssueUserParameters) DeepCopy 

func (in *IssueUserParameters) DeepCopy() *IssueUserParameters

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new IssueUserParameters.

func (*IssueUserParameters) DeepCopyInto 

func (in *IssueUserParameters) DeepCopyInto(out *IssueUserParameters)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type IssueUserStatus 

type IssueUserStatus struct {
	User IssueStatus `json:"user"`
}

type IssueUserStorage 

type IssueUserStorage struct {
	Operator      string              `json:"operator"`
	Account       string              `json:"account"`
	User          string              `json:"user"`
	UseSigningKey string              `json:"useSigningKey"`
	Claims        v1alpha1.UserClaims `json:"claims"`
	Status        IssueUserStatus     `json:"status"`
}

type JWTData 

type JWTData struct {
	JWTStorage
}

JWTData represents the the data returned by a JWT operation

type JWTParameters 

type JWTParameters struct {
	Operator string `json:"operator,omitempty"`
	Account  string `json:"account,omitempty"`
	User     string `json:"user,omitempty"`
	JWTStorage
}

JWTParameters represents the parameters for a JWT operation

type JWTStorage 

type JWTStorage struct {
	JWT string `json:"jwt"`
}

JWTStorage represents a JWT stored in the backend

type NKeyStorage 

type NKeyStorage struct {
	Seed []byte `json:"seed,omitempty"`
}

NkeySorage represents a Nkey stored in the backend

type NatsBackend 

type NatsBackend struct {
	*framework.Backend
	// contains filtered or unexported fields
}

natsBackend defines an object that extends the Vault backend and stores the target API's client.

type NatsClient 

type NatsClient struct {
	*nats.Conn
}

natsClient creates an object storing the client.

type NkeyData 

type NkeyData struct {
	PublicKey  string `json:"publicKey,omitempty"`
	PrivateKey string `json:"privateKey,omitempty"`
	Seed       string `json:"seed,omitempty"`
}

NkeyData represents the the data returned by a Nkey operation

type NkeyParameters 

type NkeyParameters struct {
	Operator string `json:"operator,omitempty"`
	Account  string `json:"account,omitempty"`
	Signing  string `json:"signing,omitempty"`
	User     string `json:"user,omitempty"`
	Seed     string `json:"seed,omitempty"`
}

NkeyParameters represents the parameters for a Nkey operation

Source Files

View all Source files

Directories

Path Synopsis
cmd
vault-plugin-secrets-nats
pkg
claims/account/v1alpha1
+k8s:deepcopy-gen=package
 +k8s:deepcopy-gen=package
claims/common
+kubebuilder:object:generate=true
 +kubebuilder:object:generate=true
claims/operator/v1alpha1
+k8s:deepcopy-gen=package
 +k8s:deepcopy-gen=package
claims/user/v1alpha1
+k8s:deepcopy-gen=package
 +k8s:deepcopy-gen=package
resolver
stm
Package stm provides methods for converting a golang struct to a map[string]interface{} and vice versa using JSON tags Therefore the name: stm -> *S*truct *T*o *M*ap
 Package stm provides methods for converting a golang struct to a map[string]interface{} and vice versa using JSON tags Therefore the name: stm -> *S*truct *T*o *M*ap
validate

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.