lohpi

README

Lohpi

Lohpi is a distributed system for sharing and enforcing data policies. By bootstrapping storage nodes close to existing infrastructure, data access is enforced and tracked by employing Azure AD identities and PostgresQL. It uses the underlying Ifrit library for message passing and secure membership management.

Lohpi consists of multiple components, each of which is explained below. In addition, each component exposes and API that can be consumed.

Directory server

The directory server is a network entity that serves client requests and forwards them to the correct storage node. The directory server keeps track of the datasets that are stored at the nodes and which datasets that have been checked out by clients.

Storage node

A storage node is a node that assigns policies with dataset identifiers. In addition, it enforces accesses and keeps track of the datasets being checked out by the clients. There can exist multiple nodes, each is assigned a data repository to safeguard data.

Policy store

The policy store is a component that accepts policy changes through a web interface and disseminates the policies in batches using multicasting and gossiping.

CA

The certificate authority is an entity that distributes X.509 certificates upon requests from the storage nodes and the underlying Ifrit network.

Production environment

To deploy the system at the diggi-2 server, run docker-compose up. This, however, whould not be necessary because the Github runner will run it whenever changes are pushed to the master branch. When a new deployment is done, run docker ps to check if any containers try to restart. If one or more tries to restart, something went wrong.

Local development

These are the steps needed to create a local development database on your own machine. The database will be used by any number of nodes. This allows you to develop and run Lohpi with minimal overhead. Please You need to connect to VPN if you are outside of the university domain.

1. Create a local directory to mount the postgres volume. $ mkdir postgres-data

2. Start a PostgreSQL DBMS instance running in Docker $ docker run -d --name lohpi-postgres-dev -e POSTGRES_PASSWORD=password! -v postgres-data:/var/lib/postgresql/data -p 5432:5432 postgres

3. Run docker ps to see that it runs. Note the container ID to tear it down later using docker stop <id>.

4. Copy dev-initialize.sql into the Docker image: $ docker cp setup/dev-initialize.sql lohpi-postgres-dev:/dev-initialize.sql

5. Enter the shell: $ docker exec -it lohpi-postgres-dev bash

6. Setup the development database: $ psql -h localhost -U postgres -f dev-initialize.sql

7. When you need to login to the database using psql, run $ psql -h localhost -U lohpi_dev_user -d dataset_policy_db. Now you can operate directly on the DBMS using the psql shell, if needed.

8. Tear down the database by running docker stop <id> as explained above.

Docker-compose

You can deploy the system locally on your system using docker-compose -f docker-compose.dev.yml build and docker-compose -f docker-compose.dev.yml up. This will build the CA, directory server, policy store and one storage node. If you want muliple nodes, you can build additional images by running docker build -f cmd/dataverse/node/Dockerfile -t some_tag and then docker run --network=host -t some_tag. Remember to supply a unique value to the -name flag in the node executable.

Go executables

You can compile the programs separately by running go build in the cmd subdirectories. Each subdirectory contains an application that implements the handlers whenever client requests arrive.

Lohpi roadmap

To be completed psql "host=lohpi-demo-temp.postgres.database.azure.com port=5432 dbname=postgres user=lohpiadmin@lohpi-demo password=wiGeeT3sNodes sslmode=require"

Documentation

Index

• type AzureKeyVaultClient
  • func NewAzureKeyVaultClient(config *AzureKeyVaultClientConfig) (*AzureKeyVaultClient, error)
  • func (k *AzureKeyVaultClient) GetSecret(vaultBaseURL, secretName string) (*AzureKeyVaultSecretResponse, error)
• type AzureKeyVaultClientConfig
• type AzureKeyVaultSecretResponse
• type DatasetIndexingOptions
• type DirectoryServer
  • func NewDirectoryServer(config *DirectoryServerConfig, new bool) (*DirectoryServer, error)
  • func (d *DirectoryServer) Start()
  • func (d *DirectoryServer) Stop()
• type DirectoryServerConfig
• type Node
  • func NewNode(config *NodeConfig, createNew bool) (*Node, error)
  • func (n *Node) HandshakeNetwork(directoryServerAddress, policyStoreAddress string) error
  • func (n *Node) IfritAddress() string
  • func (n *Node) IndexDataset(datasetId string, indexOptions *DatasetIndexingOptions) error
  • func (n *Node) Name() string
  • func (n *Node) RegisterDatasetHandler(f func(datasetId string, w http.ResponseWriter, r *http.Request))
  • func (n *Node) RegisterMetadataHandler(f func(datasetId string, w http.ResponseWriter, r *http.Request))
  • func (n *Node) RemoveDataset(id string)
  • func (n *Node) Shutdown()
  • func (n *Node) Start()
  • func (n *Node) StartDatasetSyncing(remoteAddr string) error
  • func (n *Node) String() string
• type NodeConfig
• type PolicyStore
  • func NewPolicyStore(config *PolicyStoreConfig, new bool) (*PolicyStore, error)
  • func (ps *PolicyStore) IfritAddress() string
  • func (ps *PolicyStore) RunPolicyBatcher()
  • func (ps *PolicyStore) Start() error
  • func (ps *PolicyStore) Stop()
  • func (ps *PolicyStore) StopPolicyBatcher()
• type PolicyStoreConfig

Types

type AzureKeyVaultClient

type AzureKeyVaultClient struct {
	// contains filtered or unexported fields
}

func NewAzureKeyVaultClient

func NewAzureKeyVaultClient(config *AzureKeyVaultClientConfig) (*AzureKeyVaultClient, error)

func (*AzureKeyVaultClient) GetSecret

func (k *AzureKeyVaultClient) GetSecret(vaultBaseURL, secretName string) (*AzureKeyVaultSecretResponse, error)

GetSecret retrieves a secret from keyvault

type AzureKeyVaultClientConfig

type AzureKeyVaultClientConfig struct {
	AzureKeyVaultClientID     string
	AzureKeyVaultClientSecret string
	AzureKeyVaultTenantID     string
}

Configuration required to use Azure Key Vault

type AzureKeyVaultSecretResponse

type AzureKeyVaultSecretResponse struct {
	Value string `json:"value"`
	Id    string `json:"id"`
}

type DatasetIndexingOptions

type DatasetIndexingOptions struct {
	AllowMultipleCheckouts bool
}

type DirectoryServer

type DirectoryServer struct {
	// contains filtered or unexported fields
}

func NewDirectoryServer

func NewDirectoryServer(config *DirectoryServerConfig, new bool) (*DirectoryServer, error)

Returns a new DirectoryServer using the given directory server options. Returns a non-nil error, if any.

func (*DirectoryServer) Start

func (d *DirectoryServer) Start()

Starts the directory server by running the Ifrit server, gRPC server and HTTP server. The call will return when these services have been started.

func (*DirectoryServer) Stop

func (d *DirectoryServer) Stop()

Performs a graceful shutdown of the directory server.

type DirectoryServerConfig

type DirectoryServerConfig struct {
	// The address of the CA. Default value is "127.0.0.1:8301"
	CaAddress string

	// The name of this node
	Name string

	// The database connection string. Default value is "". If it is not set, the database connection
	// will not be used. This means that only the in-memory maps will be used for storage.
	SQLConnectionString string

	// Backup retention time. Default value is 0. If it is zero, backup retentions will not be issued.
	// NOT USED
	BackupRetentionTime time.Time

	// Hostname of the node. Default value is "127.0.1.1".
	HostName string

	// Output directory of gossip observation unit. Default value is the current working directory.
	PolicyObserverWorkingDirectory string

	// HTTP port used by the server. Default value is 8080.
	HTTPPort int

	// TCP port used by the gRPC server. Default value is 8081.
	GRPCPort int

	// Path used to store X.509 certificate and private key
	CryptoUnitWorkingDirectory string

	// Ifrit's TCP port. Default value is 5000.
	IfritTCPPort int

	// Ifrit's UDP port. Default value is 6000.
	IfritUDPPort int
}

type Node

type Node struct {
	// contains filtered or unexported fields
}

func NewNode

func NewNode(config *NodeConfig, createNew bool) (*Node, error)

TODO: consider using intefaces

func (*Node) HandshakeNetwork

func (n *Node) HandshakeNetwork(directoryServerAddress, policyStoreAddress string) error

func (*Node) IfritAddress

func (n *Node) IfritAddress() string

Returns the underlying Ifrit address.

func (*Node) IndexDataset

func (n *Node) IndexDataset(datasetId string, indexOptions *DatasetIndexingOptions) error

IndexDataset registers a dataset, given with its unique identifier. The call is blocking; it will return when policy requests to the policy store finish.

func (*Node) Name

func (n *Node) Name() string

Returns the name of the node.

func (*Node) RegisterDatasetHandler

func (n *Node) RegisterDatasetHandler(f func(datasetId string, w http.ResponseWriter, r *http.Request))

Registers a handler that processes the client request of datasets. The handler is only invoked if the same id was registered with 'func (n *Node) IndexDataset()' method. It is the caller's responsibility to close the request after use.

func (*Node) RegisterMetadataHandler

func (n *Node) RegisterMetadataHandler(f func(datasetId string, w http.ResponseWriter, r *http.Request))

Registers a handler that processes the client request of metadata. The handler is only invoked if the same id was registered with 'func (n *Node) IndexDataset()' method. It is the caller's responsibility to close the request after use.

func (*Node) RemoveDataset

func (n *Node) RemoveDataset(id string)

Removes the dataset policy from the node. The dataset will no longer be available to clients.

func (*Node) Shutdown

func (n *Node) Shutdown()

Shuts down the node

func (*Node) Start

func (n *Node) Start()

func (*Node) StartDatasetSyncing

func (n *Node) StartDatasetSyncing(remoteAddr string) error

func (*Node) String

func (n *Node) String() string

Returns the string representation of the node.

type NodeConfig

type NodeConfig struct {
	// The address of the CA. Default value is "127.0.0.1:8301"
	CaAddress string

	// The name of this node
	Name string

	// The database connection string. Default value is "". If it is not set, the database connection
	// will not be used. This means that only the in-memory maps will be used for storage.
	SQLConnectionString string

	// Backup retention time. Default value is 0. If it is zero, backup retentions will not be issued.
	// NOT USED
	BackupRetentionTime time.Time

	// Hostname of the node. Default value is "127.0.1.1".
	Hostname string

	// Output directory of gossip observation unit. Default value is the current working directory.
	PolicyObserverWorkingDirectory string

	// HTTP port number. Default value is 9000
	Port int

	// Synchronization interval. Default value is 60 seconds.
	SyncInterval time.Duration

	// Path used to store X.509 certificate and private key
	CryptoUnitWorkingDirectory string
}

type PolicyStore

type PolicyStore struct {
	// contains filtered or unexported fields
}

func NewPolicyStore

func NewPolicyStore(config *PolicyStoreConfig, new bool) (*PolicyStore, error)

func (*PolicyStore) IfritAddress

func (ps *PolicyStore) IfritAddress() string

func (*PolicyStore) RunPolicyBatcher

func (ps *PolicyStore) RunPolicyBatcher()

func (*PolicyStore) Start

func (ps *PolicyStore) Start() error

func (*PolicyStore) Stop

func (ps *PolicyStore) Stop()

func (*PolicyStore) StopPolicyBatcher

func (ps *PolicyStore) StopPolicyBatcher()

type PolicyStoreConfig

type PolicyStoreConfig struct {
	// The address of the CA. Default value is "127.0.0.1:8301"
	CaAddress string

	// The name of this node
	Name string

	// The location of the Git repository. Default value is "./policy_store_repository".
	PolicyStoreGitRepository string

	// Hostname of the policy store. Default value is "127.0.1.1".
	Hostname string

	// Gossip interval in seconds. Default value is 60 seconds.
	GossipInterval time.Duration

	// HTTP port used by the http server. Default value is 8083
	HTTPPort int

	// TCP port used by the gRPC server. Default value is 8084
	GRPCPort int

	// Mutlicast acceptance level. Default value is 0.5.
	MulticastAcceptanceLevel float64

	// Number of direct recipients. Default value is 1.
	NumDirectRecipients int

	// Directory server address. Default value is "127.0.1.1:8081".
	DirectoryServerAddress string

	// The address of the CA. Default value is "127.0.0.1:8301"
	LohpiCaAddr string

	// The database connection string. Default value is "". If it is not set, the database connection
	// will not be used. This means that only the in-memory maps will be used for storage.
	SQLConnectionString string

	// Path used to store X.509 certificate and private key
	CryptoUnitWorkingDirectory string

	// Ifrit's TCP port. Default value is 5000.
	IfritTCPPort int

	// Ifrit's UDP port. Default value is 6000.
	IfritUDPPort int

	// Ifrit's X.509 certificate path. An error is returned if the string is empty.
	IfritCertPath string
}