bootstrap

README

BOOTSTRAP SERVICE

New devices need to be configured properly and connected to the Magistrala. Bootstrap service is used in order to accomplish that. This service provides the following features:

1. Creating new Magistrala Clients
2. Providing basic configuration for the newly created Clients
3. Enabling/disabling Clients

Pre-provisioning a new Client is as simple as sending Configuration data to the Bootstrap service. Once the Client is online, it sends a request for initial config to Bootstrap service. Bootstrap service provides an API for enabling and disabling Clients. Only enabled Clients can exchange messages over Magistrala. Bootstrapping does not implicitly enable Clients, it has to be done manually.

In order to bootstrap successfully, the Client needs to send bootstrapping request to the specific URL, as well as a secret key. This key and URL are pre-provisioned during the manufacturing process. If the Client is provisioned on the Bootstrap service side, the corresponding configuration will be sent as a response. Otherwise, the Client will be saved so that it can be provisioned later.

Client Configuration Entity

Client Configuration consists of two logical parts: the custom configuration that can be interpreted by the Client itself and Magistrala-related configuration. Magistrala config contains:

1. corresponding Magistrala Client ID
2. corresponding Magistrala Client key
3. list of the Magistrala channels the Client is connected to

Note: list of channels contains IDs of the Magistrala channels. These channels are pre-provisioned on the Magistrala side and, unlike corresponding Magistrala Client, Bootstrap service is not able to create Magistrala Channels.

Enabling and disabling Client (adding Client to/from whitelist) is as simple as connecting corresponding Magistrala Client to the given list of Channels. Configuration keeps state of the Client:

State	What it means
Inactive	Client is created, but isn't enabled
Active	Client is able to communicate using Magistrala

Switching between states Active and Inactive enables and disables Client, respectively.

Client configuration also contains the so-called external ID and external key. An external ID is a unique identifier of corresponding Client. For example, a device MAC address is a good choice for external ID. External key is a secret key that is used for authentication during the bootstrapping procedure.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable	Description	Default
SMQ_BOOTSTRAP_LOG_LEVEL	Log level for Bootstrap (debug, info, warn, error)	info
SMQ_BOOTSTRAP_DB_HOST	Database host address	localhost
SMQ_BOOTSTRAP_DB_PORT	Database host port	5432
SMQ_BOOTSTRAP_DB_USER	Database user	magistrala
SMQ_BOOTSTRAP_DB_PASS	Database password	magistrala
SMQ_BOOTSTRAP_DB_NAME	Name of the database used by the service	bootstrap
SMQ_BOOTSTRAP_DB_SSL_MODE	Database connection SSL mode (disable, require, verify-ca, verify-full)	disable
SMQ_BOOTSTRAP_DB_SSL_CERT	Path to the PEM encoded certificate file	""
SMQ_BOOTSTRAP_DB_SSL_KEY	Path to the PEM encoded key file	""
SMQ_BOOTSTRAP_DB_SSL_ROOT_CERT	Path to the PEM encoded root certificate file	""
SMQ_BOOTSTRAP_ENCRYPT_KEY	Secret key for secure bootstrapping encryption	12345678910111213141516171819202
SMQ_BOOTSTRAP_HTTP_HOST	Bootstrap service HTTP host	""
SMQ_BOOTSTRAP_HTTP_PORT	Bootstrap service HTTP port	9013
SMQ_BOOTSTRAP_HTTP_SERVER_CERT	Path to server certificate in pem format	""
SMQ_BOOTSTRAP_HTTP_SERVER_KEY	Path to server key in pem format	""
SMQ_BOOTSTRAP_EVENT_CONSUMER	Bootstrap service event source consumer name	bootstrap
SMQ_ES_URL	Event store URL	nats://localhost:4222
SMQ_AUTH_GRPC_URL	Auth service Auth gRPC URL	localhost:8181
SMQ_AUTH_GRPC_TIMEOUT	Auth service Auth gRPC request timeout in seconds	1s
SMQ_AUTH_GRPC_CLIENT_CERT	Path to the PEM encoded auth service Auth gRPC client certificate file	""
SMQ_AUTH_GRPC_CLIENT_KEY	Path to the PEM encoded auth service Auth gRPC client key file	""
SMQ_AUTH_GRPC_SERVER_CERTS	Path to the PEM encoded auth server Auth gRPC server trusted CA certificate file	""
SMQ_CLIENTS_URL	Base URL for Magistrala Clients	http://localhost:9000
SMQ_JAEGER_URL	Jaeger server URL	http://localhost:4318/v1/traces
SMQ_JAEGER_TRACE_RATIO	Jaeger sampling ratio	1.0
SMQ_SEND_TELEMETRY	Send telemetry to magistrala call home server	true
SMQ_BOOTSTRAP_INSTANCE_ID	Bootstrap service instance ID	""

Deployment

The service itself is distributed as Docker container. Check the bootstrap service section in docker-compose file to see how service is deployed.

To start the service outside of the container, execute the following shell script:

# download the latest version of the service
git clone https://github.com/absmach/magistrala

cd magistrala

# compile the servic e
make bootstrap

# copy binary to bin
make install

# set the environment variables and run the service
SMQ_BOOTSTRAP_LOG_LEVEL=info \
SMQ_BOOTSTRAP_DB_HOST=localhost \
SMQ_BOOTSTRAP_DB_PORT=5432 \
SMQ_BOOTSTRAP_DB_USER=magistrala \
SMQ_BOOTSTRAP_DB_PASS=magistrala \
SMQ_BOOTSTRAP_DB_NAME=bootstrap \
SMQ_BOOTSTRAP_DB_SSL_MODE=disable \
SMQ_BOOTSTRAP_DB_SSL_CERT="" \
SMQ_BOOTSTRAP_DB_SSL_KEY="" \
SMQ_BOOTSTRAP_DB_SSL_ROOT_CERT="" \
SMQ_BOOTSTRAP_HTTP_HOST=localhost \
SMQ_BOOTSTRAP_HTTP_PORT=9013 \
SMQ_BOOTSTRAP_HTTP_SERVER_CERT="" \
SMQ_BOOTSTRAP_HTTP_SERVER_KEY="" \
SMQ_BOOTSTRAP_EVENT_CONSUMER=bootstrap \
SMQ_ES_URL=nats://localhost:4222 \
SMQ_AUTH_GRPC_URL=localhost:8181 \
SMQ_AUTH_GRPC_TIMEOUT=1s \
SMQ_AUTH_GRPC_CLIENT_CERT="" \
SMQ_AUTH_GRPC_CLIENT_KEY="" \
SMQ_AUTH_GRPC_SERVER_CERTS="" \
SMQ_CLIENTS_URL=http://localhost:9000 \
SMQ_JAEGER_URL=http://localhost:14268/api/traces \
SMQ_JAEGER_TRACE_RATIO=1.0 \
SMQ_SEND_TELEMETRY=true \
SMQ_BOOTSTRAP_INSTANCE_ID="" \
$GOBIN/magistrala-bootstrap

Setting SMQ_BOOTSTRAP_HTTP_SERVER_CERT and SMQ_BOOTSTRAP_HTTP_SERVER_KEY will enable TLS against the service. The service expects a file in PEM format for both the certificate and the key.

Setting SMQ_AUTH_GRPC_CLIENT_CERT and SMQ_AUTH_GRPC_CLIENT_KEY will enable TLS against the auth service. The service expects a file in PEM format for both the certificate and the key. Setting SMQ_AUTH_GRPC_SERVER_CERTS will enable TLS against the auth service trusting only those CAs that are provided. The service expects a file in PEM format of trusted CAs.

Usage

For more information about service capabilities and its usage, please check out the API documentation.

Documentation

Overview

Package bootstrap contains the domain concept definitions needed to support SuperMQ bootstrap service functionality.

Index

• Variables
• type Channel
• type Config
• type ConfigReader
  • func NewConfigReader(encKey []byte) ConfigReader
• type ConfigRepository
• type ConfigsPage
• type Filter
• type Service
  • func New(policyService policies.Service, configs ConfigRepository, sdk mgsdk.SDK, ...) Service
• type State
  • func (s State) String() string

Variables

var (
	// ErrClients indicates failure to communicate with SuperMQ Clients service.
	// It can be due to networking error or invalid/unauthenticated request.
	ErrClients = errors.New("failed to receive response from Clients service")

	// ErrExternalKey indicates a non-existent bootstrap configuration for given external key.
	ErrExternalKey = errors.New("failed to get bootstrap configuration for given external key")

	// ErrExternalKeySecure indicates error in getting bootstrap configuration for given encrypted external key.
	ErrExternalKeySecure = errors.New("failed to get bootstrap configuration for given encrypted external key")

	// ErrBootstrap indicates error in getting bootstrap configuration.
	ErrBootstrap = errors.New("failed to read bootstrap configuration")

	// ErrAddBootstrap indicates error in adding bootstrap configuration.
	ErrAddBootstrap = errors.New("failed to add bootstrap configuration")
)

Types

type Channel

type Channel struct {
	ID          string                 `json:"id"`
	Name        string                 `json:"name,omitempty"`
	Metadata    map[string]interface{} `json:"metadata,omitempty"`
	DomainID    string                 `json:"domain_id"`
	Parent      string                 `json:"parent_id,omitempty"`
	Description string                 `json:"description,omitempty"`
	CreatedAt   time.Time              `json:"created_at"`
	UpdatedAt   time.Time              `json:"updated_at,omitempty"`
	UpdatedBy   string                 `json:"updated_by,omitempty"`
	Status      clients.Status         `json:"status"`
}

Channel represents SuperMQ channel corresponding SuperMQ Client is connected to.

type Config

type Config struct {
	ClientID     string    `json:"client_id"`
	ClientSecret string    `json:"client_secret"`
	DomainID     string    `json:"domain_id,omitempty"`
	Name         string    `json:"name,omitempty"`
	ClientCert   string    `json:"client_cert,omitempty"`
	ClientKey    string    `json:"client_key,omitempty"`
	CACert       string    `json:"ca_cert,omitempty"`
	Channels     []Channel `json:"channels,omitempty"`
	ExternalID   string    `json:"external_id"`
	ExternalKey  string    `json:"external_key"`
	Content      string    `json:"content,omitempty"`
	State        State     `json:"state"`
}

Config represents Configuration entity. It wraps information about external entity as well as info about corresponding SuperMQ entities. MGClient represents corresponding SuperMQ Client ID. MGKey is key of corresponding SuperMQ Client. MGChannels is a list of SuperMQ Channels corresponding SuperMQ Client connects to.

type ConfigReader

type ConfigReader interface {
	ReadConfig(Config, bool) (interface{}, error)
}

ConfigReader is used to parse Config into format which will be encoded as a JSON and consumed from the client side. The purpose of this interface is to provide convenient way to generate custom configuration response based on the specific Config which will be consumed by the client.

func NewConfigReader

func NewConfigReader(encKey []byte) ConfigReader

NewConfigReader return new reader which is used to generate response from the config.

type ConfigRepository

type ConfigRepository interface {
	// Save persists the Config. Successful operation is indicated by non-nil
	// error response.
	Save(ctx context.Context, cfg Config, chsConnIDs []string) (string, error)

	// RetrieveByID retrieves the Config having the provided identifier, that is owned
	// by the specified user.
	RetrieveByID(ctx context.Context, domainID, id string) (Config, error)

	// RetrieveAll retrieves a subset of Configs that are owned
	// by the specific user, with given filter parameters.
	RetrieveAll(ctx context.Context, domainID string, clientIDs []string, filter Filter, offset, limit uint64) ConfigsPage

	// RetrieveByExternalID returns Config for given external ID.
	RetrieveByExternalID(ctx context.Context, externalID string) (Config, error)

	// Update updates an existing Config. A non-nil error is returned
	// to indicate operation failure.
	Update(ctx context.Context, cfg Config) error

	// UpdateCerts updates and returns an existing Config certificate and domainID.
	// A non-nil error is returned to indicate operation failure.
	UpdateCert(ctx context.Context, domainID, clientID, clientCert, clientKey, caCert string) (Config, error)

	// UpdateConnections updates a list of Channels the Config is connected to
	// adding new Channels if needed.
	UpdateConnections(ctx context.Context, domainID, id string, channels []Channel, connections []string) error

	// Remove removes the Config having the provided identifier, that is owned
	// by the specified user.
	Remove(ctx context.Context, domainID, id string) error

	// ChangeState changes of the Config, that is owned by the specific user.
	ChangeState(ctx context.Context, domainID, id string, state State) error

	// ListExisting retrieves those channels from the given list that exist in DB.
	ListExisting(ctx context.Context, domainID string, ids []string) ([]Channel, error)

	// RemoveClient removes Config of the Client with the given ID.
	RemoveClient(ctx context.Context, id string) error

	// UpdateChannel updates channel with the given ID.
	UpdateChannel(ctx context.Context, c Channel) error

	// RemoveChannel removes channel with the given ID.
	RemoveChannel(ctx context.Context, id string) error

	// ConnectClient changes state of the Config when the corresponding Client is connected to the Channel.
	ConnectClient(ctx context.Context, channelID, clientID string) error

	// DisconnectClient changes state of the Config when the corresponding Client is disconnected from the Channel.
	DisconnectClient(ctx context.Context, channelID, clientID string) error
}

ConfigRepository specifies a Config persistence API.

type ConfigsPage

type ConfigsPage struct {
	Total   uint64   `json:"total"`
	Offset  uint64   `json:"offset"`
	Limit   uint64   `json:"limit"`
	Configs []Config `json:"configs"`
}

ConfigsPage contains page related metadata as well as list of Configs that belong to this page.

type Filter

type Filter struct {
	FullMatch    map[string]string
	PartialMatch map[string]string
}

Filter is used for the search filters.

type Service

type Service interface {
	// Add adds new Client Config to the user identified by the provided token.
	Add(ctx context.Context, session smqauthn.Session, token string, cfg Config) (Config, error)

	// View returns Client Config with given ID belonging to the user identified by the given token.
	View(ctx context.Context, session smqauthn.Session, id string) (Config, error)

	// Update updates editable fields of the provided Config.
	Update(ctx context.Context, session smqauthn.Session, cfg Config) error

	// UpdateCert updates an existing Config certificate and token.
	// A non-nil error is returned to indicate operation failure.
	UpdateCert(ctx context.Context, session smqauthn.Session, clientID, clientCert, clientKey, caCert string) (Config, error)

	// UpdateConnections updates list of Channels related to given Config.
	UpdateConnections(ctx context.Context, session smqauthn.Session, token, id string, connections []string) error

	// List returns subset of Configs with given search params that belong to the
	// user identified by the given token.
	List(ctx context.Context, session smqauthn.Session, filter Filter, offset, limit uint64) (ConfigsPage, error)

	// Remove removes Config with specified token that belongs to the user identified by the given token.
	Remove(ctx context.Context, session smqauthn.Session, id string) error

	// Bootstrap returns Config to the Client with provided external ID using external key.
	Bootstrap(ctx context.Context, externalKey, externalID string, secure bool) (Config, error)

	// ChangeState changes state of the Client with given client ID and domain ID.
	ChangeState(ctx context.Context, session smqauthn.Session, token, id string, state State) error

	// UpdateChannelHandler updates Channel with data received from an event.
	UpdateChannelHandler(ctx context.Context, channel Channel) error

	// RemoveConfigHandler removes Configuration with id received from an event.
	RemoveConfigHandler(ctx context.Context, id string) error

	// RemoveChannelHandler removes Channel with id received from an event.
	RemoveChannelHandler(ctx context.Context, id string) error

	// ConnectClientHandler changes state of the Config to active when connect event occurs.
	ConnectClientHandler(ctx context.Context, channelID, clientID string) error

	// DisconnectClientHandler changes state of the Config to inactive when disconnect event occurs.
	DisconnectClientHandler(ctx context.Context, channelID, clientID string) error
}

Service specifies an API that must be fulfilled by the domain service implementation, and all of its decorators (e.g. logging & metrics).

func New

func New(policyService policies.Service, configs ConfigRepository, sdk mgsdk.SDK, encKey []byte, idp supermq.IDProvider) Service

New returns new Bootstrap service.

type State

type State int

State represents corresponding SuperMQ Client state. The possible Config States as well as description of what that State represents are given in the table: | State | What it means | |----------+--------------------------------------------------------------------------------| | Inactive | Client is created, but isn't able to communicate over SuperMQ | | Active | Client is able to communicate using SuperMQ |.

const (
	// Inactive Client is created, but not able to exchange messages using SuperMQ.
	Inactive State = iota
	// Active Client is created, configured, and whitelisted.
	Active
)

func (State) String

func (s State) String() string

String returns string representation of State.