harp

README

Harp

Harp is for Harpocrates (Ancient Greek: Ἁρποκράτης) the god of silence, secrets and confidentiality in the Hellenistic religion. - Wikipedia

Why harp?

• Secret management is in essence a collection of processes that must be auditable, executable and reproducible for infosec and operation requirements;
• Secret provisioning must be designed with secret rotation as a day one task, due to the fact that secret data must be rotated periodically to keep its secret property;
• Developers should negotiate secret value for the secret consumer they are currently developing, by the contract based on a path (reference to the secret) and a value specification (for code contract) without the knowledge of the final deployed value;
• Secret Operators use different set of tools to achieve secret management operation which increases the error/secret exposure probability due to tool count involved in the process (incompatibility, changes, etc.);
• Without a defined secret naming convention, the secret storage becomes difficult to handle in time (naming is hard) and secret naming could not be helped to get a consistent, reliable and flexible secret tree;
• Secret storage backend can use various implementations in different environments and should be provisioned consistently.

How does it work?

Like a Data pipeline but for secret

harp allows you to handle secrets using deterministic pipelines expressed using an atomic series of CLI operations applied to a commonly shared container immutable and standalone filesystem used to store secret collection (Bundle) generated from a template engine via user specification, or external secret value coming from files or external secret storage.

These pipelines use the immutable container filesystem as a data exchange protocol and could be extended for new input, intermediary operation or output via plugins created with the harp SDK.

Immutable transformation

Each applied transformation create a container with transformed data inside. This will enforce container reproducibility by eliminating cumulative side-effects applied to the same container.

The container handles for you the confidentiality and integrity protection applied to the secret collection stored inside and manipulated by copy during the pipeline execution.

What can I do?

New to harp, let's start with onboarding tutorial ! TL;DR - Features overview

Harp provides :

• A methodology to design your secret management;
  • Secret naming convention (CSO);
  • A defined common language and complete processes to achieve secret management operations;
• A SDK to create your own tools to orchestrate your secret management pipelines;
  • A container manipulation library exposed as github.com/elastic/harp/pkg/container;
  • A secret bundle specification to store and manipulate secrets exposed as github.com/elastic/harp/pkg/bundle;
  • An on-steroid template engine exposed as github.com/elastic/harp/pkg/template
  • A path name validation library exposed as github.com/elastic/harp/pkg/cso
• A CLI for secret management implementation
  • CI/CD integration;
  • Based on human readable definitions (YAML);
  • In order to create auditable and reproducible pipelines.
  • An extensible tool which can be enhanced via plugins.

And allows :

• Bundle level operations
  • Create a bundle from scratch / template / json (more via plugins);
  • Generate a complete bundle using a YAML Descriptor (BundleTemplate) to describe secret and their usages;
  • Read value stored in the K/V virtual filesystem;
  • Update the K/V virtual filesystem;
  • Reproductible patch applied on immutable container (copy-on-write);
  • Import / Export to Vault.
• Immutable container level operations
  • Seal / Unseal a container for integrity and confidentiality property conservation to enforce at-rest encryption (aes256-gcm96 or chacha20-poly1305);
  • Multiple identities sealing algorithm;

License

harp artifacts and source code is released under Apache 2.0 Software License.

Homebrew install

Download a release or build from source.

For stable version

brew tap elastic/harp
brew install elastic/harp/harp

Build instructions

Download a release or build from source.

First time

Check your go version

Only last 2 minor versions of a major are supported.

Harp is compiled with :

$ go version
go version go1.16.7 linux/amd64

Simple go version manager - https://github.com/stefanmaric/g

Install mage

Mage is an alternative to Make where language used is Go. You can install it using 2 different methods.

From source

# Install mage
git clone https://github.com/magefile/mage
cd mage
go run bootstrap.go

From brew formula

brew install mage

Clone repository

git clone git@github.com:elastic/harp.git
# Go to tools submodule
cd harp/tools
# Pull tools sources, compile them and install executable in tools/bin
mage

Daily

export PATH=$HARP_REPO/tools/bin:$PATH
# Build harp in bin folder
mage

Docker

For Tools

You have to build this image once before executing artifact pipelines.

mage docker:tools

For CLI

# or docker image [distroless:static, rootless, noshell]
mage docker:harp
# To execute in the container
docker run --rm -ti --read-only elastic/harp:<version>

Plugins

You can find more Harp feature extensions - https://github.com/elastic/harp-plugins

Community

Here is the list of external projects used as inspiration :

• Kubernetes
• Helm
• Open Policy Agent ConfTest
• SaltPack
• Hashicorp Vault
• AWS SDK Go

Documentation

Overview

Package harp contains harp SDK