bootstrap

README

BOOTSTRAP SERVICE

New devices need to be configured properly and connected to the Magistrala. Bootstrap service is used in order to accomplish that. This service provides the following features:

1. Creating new Magistrala Things
2. Providing basic configuration for the newly created Things
3. Enabling/disabling Things

Pre-provisioning a new Thing is as simple as sending Configuration data to the Bootstrap service. Once the Thing is online, it sends a request for initial config to Bootstrap service. Bootstrap service provides an API for enabling and disabling Things. Only enabled Things can exchange messages over Magistrala. Bootstrapping does not implicitly enable Things, it has to be done manually.

In order to bootstrap successfully, the Thing needs to send bootstrapping request to the specific URL, as well as a secret key. This key and URL are pre-provisioned during the manufacturing process. If the Thing is provisioned on the Bootstrap service side, the corresponding configuration will be sent as a response. Otherwise, the Thing will be saved so that it can be provisioned later.

Thing Configuration Entity

Thing Configuration consists of two logical parts: the custom configuration that can be interpreted by the Thing itself and Magistrala-related configuration. Magistrala config contains:

1. corresponding Magistrala Thing ID
2. corresponding Magistrala Thing key
3. list of the Magistrala channels the Thing is connected to

Note: list of channels contains IDs of the Magistrala channels. These channels are pre-provisioned on the Magistrala side and, unlike corresponding Magistrala Thing, Bootstrap service is not able to create Magistrala Channels.

Enabling and disabling Thing (adding Thing to/from whitelist) is as simple as connecting corresponding Magistrala Thing to the given list of Channels. Configuration keeps state of the Thing:

State	What it means
Inactive	Thing is created, but isn't enabled
Active	Thing is able to communicate using Magistrala

Switching between states Active and Inactive enables and disables Thing, respectively.

Thing configuration also contains the so-called external ID and external key. An external ID is a unique identifier of corresponding Thing. For example, a device MAC address is a good choice for external ID. External key is a secret key that is used for authentication during the bootstrapping procedure.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable	Description	Default
MG_BOOTSTRAP_LOG_LEVEL	Log level for Bootstrap (debug, info, warn, error)	info
MG_BOOTSTRAP_DB_HOST	Database host address	localhost
MG_BOOTSTRAP_DB_PORT	Database host port	5432
MG_BOOTSTRAP_DB_USER	Database user	magistrala
MG_BOOTSTRAP_DB_PASS	Database password	magistrala
MG_BOOTSTRAP_DB_NAME	Name of the database used by the service	bootstrap
MG_BOOTSTRAP_DB_SSL_MODE	Database connection SSL mode (disable, require, verify-ca, verify-full)	disable
MG_BOOTSTRAP_DB_SSL_CERT	Path to the PEM encoded certificate file	""
MG_BOOTSTRAP_DB_SSL_KEY	Path to the PEM encoded key file	""
MG_BOOTSTRAP_DB_SSL_ROOT_CERT	Path to the PEM encoded root certificate file	""
MG_BOOTSTRAP_ENCRYPT_KEY	Secret key for secure bootstrapping encryption	12345678910111213141516171819202
MG_BOOTSTRAP_HTTP_HOST	Bootstrap service HTTP host	""
MG_BOOTSTRAP_HTTP_PORT	Bootstrap service HTTP port	9013
MG_BOOTSTRAP_HTTP_SERVER_CERT	Path to server certificate in pem format	""
MG_BOOTSTRAP_HTTP_SERVER_KEY	Path to server key in pem format	""
MG_BOOTSTRAP_EVENT_CONSUMER	Bootstrap service event source consumer name	bootstrap
MG_ES_URL	Event store URL	nats://localhost:4222
MG_AUTH_GRPC_URL	Auth service Auth gRPC URL	localhost:8181
MG_AUTH_GRPC_TIMEOUT	Auth service Auth gRPC request timeout in seconds	1s
MG_AUTH_GRPC_CLIENT_CERT	Path to the PEM encoded auth service Auth gRPC client certificate file	""
MG_AUTH_GRPC_CLIENT_KEY	Path to the PEM encoded auth service Auth gRPC client key file	""
MG_AUTH_GRPC_SERVER_CERTS	Path to the PEM encoded auth server Auth gRPC server trusted CA certificate file	""
MG_THINGS_URL	Base url for Magistrala Things	http://localhost:9000
MG_JAEGER_URL	Jaeger server URL	http://localhost:14268/api/traces
MG_JAEGER_TRACE_RATIO	Jaeger sampling ratio	1.0
MG_SEND_TELEMETRY	Send telemetry to magistrala call home server	true
MG_BOOTSTRAP_INSTANCE_ID	Bootstrap service instance ID	""

Deployment

The service itself is distributed as Docker container. Check the bootstrap service section in docker-compose file to see how service is deployed.

To start the service outside of the container, execute the following shell script:

# download the latest version of the service
git clone https://github.com/andychao217/magistrala

cd magistrala

# compile the servic e
make bootstrap

# copy binary to bin
make install

# set the environment variables and run the service
MG_BOOTSTRAP_LOG_LEVEL=info \
MG_BOOTSTRAP_DB_HOST=localhost \
MG_BOOTSTRAP_DB_PORT=5432 \
MG_BOOTSTRAP_DB_USER=magistrala \
MG_BOOTSTRAP_DB_PASS=magistrala \
MG_BOOTSTRAP_DB_NAME=bootstrap \
MG_BOOTSTRAP_DB_SSL_MODE=disable \
MG_BOOTSTRAP_DB_SSL_CERT="" \
MG_BOOTSTRAP_DB_SSL_KEY="" \
MG_BOOTSTRAP_DB_SSL_ROOT_CERT="" \
MG_BOOTSTRAP_HTTP_HOST=localhost \
MG_BOOTSTRAP_HTTP_PORT=9013 \
MG_BOOTSTRAP_HTTP_SERVER_CERT="" \
MG_BOOTSTRAP_HTTP_SERVER_KEY="" \
MG_BOOTSTRAP_EVENT_CONSUMER=bootstrap \
MG_ES_URL=nats://localhost:4222 \
MG_AUTH_GRPC_URL=localhost:8181 \
MG_AUTH_GRPC_TIMEOUT=1s \
MG_AUTH_GRPC_CLIENT_CERT="" \
MG_AUTH_GRPC_CLIENT_KEY="" \
MG_AUTH_GRPC_SERVER_CERTS="" \
MG_THINGS_URL=http://localhost:9000 \
MG_JAEGER_URL=http://localhost:14268/api/traces \
MG_JAEGER_TRACE_RATIO=1.0 \
MG_SEND_TELEMETRY=true \
MG_BOOTSTRAP_INSTANCE_ID="" \
$GOBIN/magistrala-bootstrap

Setting MG_BOOTSTRAP_HTTP_SERVER_CERT and MG_BOOTSTRAP_HTTP_SERVER_KEY will enable TLS against the service. The service expects a file in PEM format for both the certificate and the key.

Setting MG_AUTH_GRPC_CLIENT_CERT and MG_AUTH_GRPC_CLIENT_KEY will enable TLS against the auth service. The service expects a file in PEM format for both the certificate and the key. Setting MG_AUTH_GRPC_SERVER_CERTS will enable TLS against the auth service trusting only those CAs that are provided. The service expects a file in PEM format of trusted CAs.

Usage

For more information about service capabilities and its usage, please check out the API documentation.

Documentation

Overview

Package bootstrap contains the domain concept definitions needed to support Magistrala bootstrap service functionality.

Index

• Variables
• type Channel
• type Config
• type ConfigReader
  • func NewConfigReader(encKey []byte) ConfigReader
• type ConfigRepository
• type ConfigsPage
• type Filter
• type Service
  • func New(auth magistrala.AuthServiceClient, configs ConfigRepository, sdk mgsdk.SDK, ...) Service
• type State
  • func (s State) String() string

Variables

var (
	// ErrThings indicates failure to communicate with Magistrala Things service.
	// It can be due to networking error or invalid/unauthenticated request.
	ErrThings = errors.New("failed to receive response from Things service")

	// ErrExternalKey indicates a non-existent bootstrap configuration for given external key.
	ErrExternalKey = errors.New("failed to get bootstrap configuration for given external key")

	// ErrExternalKeySecure indicates error in getting bootstrap configuration for given encrypted external key.
	ErrExternalKeySecure = errors.New("failed to get bootstrap configuration for given encrypted external key")

	// ErrBootstrap indicates error in getting bootstrap configuration.
	ErrBootstrap = errors.New("failed to read bootstrap configuration")
)

Types

type Channel

type Channel struct {
	ID          string                 `json:"id"`
	Name        string                 `json:"name,omitempty"`
	Metadata    map[string]interface{} `json:"metadata,omitempty"`
	Owner       string                 `json:"owner_id"`
	Parent      string                 `json:"parent_id,omitempty"`
	Description string                 `json:"description,omitempty"`
	CreatedAt   time.Time              `json:"created_at"`
	UpdatedAt   time.Time              `json:"updated_at,omitempty"`
	UpdatedBy   string                 `json:"updated_by,omitempty"`
	Status      clients.Status         `json:"status"`
}

Channel represents Magistrala channel corresponding Magistrala Thing is connected to.

type Config

type Config struct {
	ThingID     string    `json:"thing_id"`
	Owner       string    `json:"owner,omitempty"`
	Name        string    `json:"name,omitempty"`
	ClientCert  string    `json:"client_cert,omitempty"`
	ClientKey   string    `json:"client_key,omitempty"`
	CACert      string    `json:"ca_cert,omitempty"`
	ThingKey    string    `json:"thing_key"`
	Channels    []Channel `json:"channels,omitempty"`
	ExternalID  string    `json:"external_id"`
	ExternalKey string    `json:"external_key"`
	Content     string    `json:"content,omitempty"`
	State       State     `json:"state"`
}

Config represents Configuration entity. It wraps information about external entity as well as info about corresponding Magistrala entities. MGThing represents corresponding Magistrala Thing ID. MGKey is key of corresponding Magistrala Thing. MGChannels is a list of Magistrala Channels corresponding Magistrala Thing connects to.

type ConfigReader

type ConfigReader interface {
	ReadConfig(Config, bool) (interface{}, error)
}

ConfigReader is used to parse Config into format which will be encoded as a JSON and consumed from the client side. The purpose of this interface is to provide convenient way to generate custom configuration response based on the specific Config which will be consumed by the client.

func NewConfigReader

func NewConfigReader(encKey []byte) ConfigReader

NewConfigReader return new reader which is used to generate response from the config.

type ConfigRepository

type ConfigRepository interface {
	// Save persists the Config. Successful operation is indicated by non-nil
	// error response.
	Save(ctx context.Context, cfg Config, chsConnIDs []string) (string, error)

	// RetrieveByID retrieves the Config having the provided identifier, that is owned
	// by the specified user.
	RetrieveByID(ctx context.Context, owner, id string) (Config, error)

	// RetrieveAll retrieves a subset of Configs that are owned
	// by the specific user, with given filter parameters.
	RetrieveAll(ctx context.Context, owner string, filter Filter, offset, limit uint64) ConfigsPage

	// RetrieveByExternalID returns Config for given external ID.
	RetrieveByExternalID(ctx context.Context, externalID string) (Config, error)

	// Update updates an existing Config. A non-nil error is returned
	// to indicate operation failure.
	Update(ctx context.Context, cfg Config) error

	// UpdateCerts updates and returns an existing Config certificate and owner.
	// A non-nil error is returned to indicate operation failure.
	UpdateCert(ctx context.Context, owner, thingID, clientCert, clientKey, caCert string) (Config, error)

	// UpdateConnections updates a list of Channels the Config is connected to
	// adding new Channels if needed.
	UpdateConnections(ctx context.Context, owner, id string, channels []Channel, connections []string) error

	// Remove removes the Config having the provided identifier, that is owned
	// by the specified user.
	Remove(ctx context.Context, owner, id string) error

	// ChangeState changes of the Config, that is owned by the specific user.
	ChangeState(ctx context.Context, owner, id string, state State) error

	// ListExisting retrieves those channels from the given list that exist in DB.
	ListExisting(ctx context.Context, owner string, ids []string) ([]Channel, error)

	// RemoveThing removes Config of the Thing with the given ID.
	RemoveThing(ctx context.Context, id string) error

	// UpdateChannel updates channel with the given ID.
	UpdateChannel(ctx context.Context, c Channel) error

	// RemoveChannel removes channel with the given ID.
	RemoveChannel(ctx context.Context, id string) error

	// DisconnectHandler changes state of the Config when the corresponding Thing is
	// disconnected from the Channel.
	DisconnectThing(ctx context.Context, channelID, thingID string) error
}

ConfigRepository specifies a Config persistence API.

type ConfigsPage

type ConfigsPage struct {
	Total   uint64   `json:"total"`
	Offset  uint64   `json:"offset"`
	Limit   uint64   `json:"limit"`
	Configs []Config `json:"configs"`
}

ConfigsPage contains page related metadata as well as list of Configs that belong to this page.

type Filter

type Filter struct {
	FullMatch    map[string]string
	PartialMatch map[string]string
}

Filter is used for the search filters.

type Service

type Service interface {
	// Add adds new Thing Config to the user identified by the provided token.
	Add(ctx context.Context, token string, cfg Config) (Config, error)

	// View returns Thing Config with given ID belonging to the user identified by the given token.
	View(ctx context.Context, token, id string) (Config, error)

	// Update updates editable fields of the provided Config.
	Update(ctx context.Context, token string, cfg Config) error

	// UpdateCert updates an existing Config certificate and token.
	// A non-nil error is returned to indicate operation failure.
	UpdateCert(ctx context.Context, token, thingID, clientCert, clientKey, caCert string) (Config, error)

	// UpdateConnections updates list of Channels related to given Config.
	UpdateConnections(ctx context.Context, token, id string, connections []string) error

	// List returns subset of Configs with given search params that belong to the
	// user identified by the given token.
	List(ctx context.Context, token string, filter Filter, offset, limit uint64) (ConfigsPage, error)

	// Remove removes Config with specified token that belongs to the user identified by the given token.
	Remove(ctx context.Context, token, id string) error

	// Bootstrap returns Config to the Thing with provided external ID using external key.
	Bootstrap(ctx context.Context, externalKey, externalID string, secure bool) (Config, error)

	// ChangeState changes state of the Thing with given ID and owner.
	ChangeState(ctx context.Context, token, id string, state State) error

	// UpdateChannelHandler updates Channel with data received from an event.
	UpdateChannelHandler(ctx context.Context, channel Channel) error

	// RemoveConfigHandler removes Configuration with id received from an event.
	RemoveConfigHandler(ctx context.Context, id string) error

	// RemoveChannelHandler removes Channel with id received from an event.
	RemoveChannelHandler(ctx context.Context, id string) error

	// DisconnectHandler changes state of the Config when connect/disconnect event occurs.
	DisconnectThingHandler(ctx context.Context, channelID, thingID string) error
}

Service specifies an API that must be fulfilled by the domain service implementation, and all of its decorators (e.g. logging & metrics).

func New

func New(auth magistrala.AuthServiceClient, configs ConfigRepository, sdk mgsdk.SDK, encKey []byte, idp magistrala.IDProvider) Service

New returns new Bootstrap service.

type State

type State int

State represents corresponding Magistrala Thing state. The possible Config States as well as description of what that State represents are given in the table: | State | What it means | |----------+--------------------------------------------------------------------------------| | Inactive | Thing is created, but isn't able to communicate over Magistrala | | Active | Thing is able to communicate using Magistrala |.

const (
	// Inactive Thing is created, but not able to exchange messages using Magistrala.
	Inactive State = iota
	// Active Thing is created, configured, and whitelisted.
	Active
)

func (State) String

func (s State) String() string

String returns string representation of State.