pii

README

PII

A pluggable Go library to protect Personal Identifiable Information at the struct field level.

TLDR; PII simplifies encryption and cryptographic erasure.

Motivation

PII may considerably help if you are:

• Following Privacy By Design principles.
• Looking for a solution to comply with privacy standards (ex: GDPR) while using Event Sourcing or an immutable store.

Project Status

The library is experimental; breaking changes may occur based on developer experience.

v1.0.0 aims to be the first stable version.

Getting Started

Installation:

    $ go get github.com/ln80/pii

    import "github.com/ln80/pii"

At your struct level:

type Person struct {
    UserID   string `pii:"subjectID,prefix=account-"`
    Fullname string `pii:"data,replace=forgotten user"`
    Gender   string `pii:"data"`
    Country  string
}

• Tag the field representing the Subject ID (ex: UserID)
• Tag Personal data fields to encrypt (only string fields are supported at the moment)

prefix option is added to the field value to define the subject ID.

replace option is used to replace the crypto-erased field value. Otherwise, the field value will be empty.

At the root level (ex: main func):

Initiate the Factory service:

func main() {
    ctx := context.Background()

    // newProt func used by factory service to instantiate protector service per namespace
    newProt := func(namespace string) pii.Protector {
        // engine handles encryption keys storage and lifecycle
        engine := memory.NewKeyEngine()

        return pii.NewProtector(namespace, enigne, func(pc *pii.ProtectorConfig) {
            pc.CacheEnabled = true
            pc.CacheTTL = 10 * time.Minute
            pc.GracefulMode = true
        })
    }

    // Factory must be injected as dependency in the functional code (ex: HTTP handlers)
    f := pii.NewFactory(newProt)

    // In a separated Goroutine, supervise and regularly clear resources
    f.Monitor(ctx)
}

At the functional level (ex: HTTP handler):

• Instantiate Protector service by passing the namespace (ex: tenant ID)
• Use the Protector to Encrypt/Decrypt structs that contain Personal data:

func MakeSignupHandler(f pii.Factory, store UserStore) http.HandlerFunc {

        return func(w http.ResponseWriter, r *http.Request) {
        ctx := r.Context()

        var per Person
        err := json.NewDecoder(r.Body).Decode(&per)
        if err != nil {
            http.Error(w, err.Error(), http.StatusBadRequest)
            return
        }

        // Get the protector service for the given namespace
        nspace := r.Header.Get("Tenant-ID")
        prot, clear := f.Instance(nspace)
        
        // Optional, Force clearing cache of encryption materials (related to namespace)
        defer clear()

        // Encrypt Person struct which contains PII.
        if err := prot.Encrypt(ctx, &per); err != nil {
            http.Error(w, err.Error(), http.StatusBadRequest)
            return
        }

        ...

        if err := store.Save(ctx, per); err != nil {
            http.Error(w, err.Error(), http.StatusBadRequest)
            return
        }

        ...
    }
}

Note that Factory service maintains a single thread-safe Protector service per namespace.

Under the hood, the Protector service generates a single encryption key per Subject ID and securely saves it in a Database.

Crypto Erasure:

Allows to Forget a subject's Personal data by first disabling, then deleting the associated encryption materials.

    ...

    if err := prot.Forget(ctx, subjectID); err != nil {
        return err
    }

    ...

    if err := prot.Recover(ctx, subjectID); err != nil {
        if errors.Is(err, pii.ErrCannotRecoverSubject) {
            fmt.Print("Sorry it's late. Good bye forever")
        }

        return err
    }

Forgetting a subject means we can't decrypt nor encrypt any of its old or new Personal data.

Forgetting the encryption key and not being able to decrypt personal is likely accepted by most Privacy Standards as deletion of PII. Nonetheless, you may need to seek legal advice related to your specific context.

Depending on Graceful Mode config, a subject encryption materials can be recovered within a grace period (to define) or not.

Plugins

Protector service uses plugins to manage encryption keys and the encryption algorithm.

You can use your own implementation for each pluggin:

    b := func(namespace string) pii.Protector {
        return pii.NewProtector(namespace, nil, func(pc *pii.ProtectorConfig) {

            pc.Encrypter = MyCustomAlgorithm()

            pc.Engine = MyCustomWrapper(
                MyCustomKeyEngine(),
            )
        })
    }

    f := NewFactory(b)

Encryption algorithm:

By default, PII uses AES 256 GCM for encryption.

By implementing core.Encrypter interface, you take responsibility, and you use your favorite algorithm (likely to respond to security standards requirements).

Key Engine:

Responsible for storing encryption keys and managing their life cycle.

PII comes with two basic implementations:

• Dynamodb: keys are saved in plain text in an AWS Dynamodb table. (server-side encryption can be applied).

• In-memory: used for test purposes.

and two wrappers:

KMS Wrapper: uses AWS KMS service to allow Envelope Encryption; client-side encryption of subjects' keys using a KMS Key as a Master Key.

Memory Cache: saves keys in memory for a limited period to enhance performance and reduce costs.

Use your custom logic by implementing core.KeyEngine, core.KeyEngineWrapper or core.KeyEngineCache.

Limitations

// TODO

Usage and documentation

Please see https://pkg.go.dev/github.com/ln80/pii for detailed usage docs.

License

Distributed under MIT License.

Documentation

Overview

Package PII offers tools to deal with Personal Identified Information in struct-field level

Index

• Constants
• Variables
• func MustTruncateIPv4Addr(ip string, n uint8) string
• func TruncateIPv4Addr(ip string, n uint8) (string, error)
• type Error
  • func (e Error) Error() string
  • func (e Error) Is(err error) bool
  • func (e Error) Message() string
  • func (e Error) Namespace() string
  • func (e Error) Subject() string
  • func (e Error) Unwrap() error
• type Factory
  • func NewFactory(newProt FactoryNewFunc, opts ...func(*FactoryConfig)) Factory
• type FactoryClearFunc
• type FactoryConfig
• type FactoryNewFunc
• type Protector
  • func NewProtector(namespace string, engine core.KeyEngine, opts ...func(*ProtectorConfig)) Protector
• type ProtectorConfig

Constants

const VERSION version = "v0.3.0"

VERSION is the current version of the PII Go Module.

Variables

var (
	ErrEncryptDecryptFailure = newErr("failed to encrypt/decrypt")
	ErrForgetSubjectFailure  = newErr("failed to forget subject")
	ErrRecoverSubjectFailure = newErr("failed to recover subject")
	ErrClearCacheFailure     = newErr("failed to clear cache")
	ErrCannotRecoverSubject  = newErr("cannot recover subject")
	ErrSubjectForgotten      = newErr("subject is forgotten")
)

Errors returned by Protector service

var (
	ErrUnsupportedType         = errors.New("unsupported type must be a struct pointer")
	ErrUnsupportedFieldType    = errors.New("unsupported field type must be exported string")
	ErrInvalidTagConfiguration = errors.New("invalid tag configuration")
)

Errors related to struct PII tag configuration

var (
	ErrInvalidIPAddress = errors.New("invalid IP address")
)

Functions

func MustTruncateIPv4Addr

func MustTruncateIPv4Addr(ip string, n uint8) string

MustTruncateIPv4Addr ,similar to TruncateIPv4Addr function, truncates the last "n" bytes from the IP v4 address, but it panics in case of error instead.

func TruncateIPv4Addr

func TruncateIPv4Addr(ip string, n uint8) (string, error)

TruncateIPv4Addr takes an IP v4 address and a number "n" of least bytes to remove and replace with zeros.

It returns the truncated IP address or returns an error if the given IPv4 address is invalid.

It helps to partially pseudonymize the IP address while preserving a prefix.

Types

type Error

type Error struct {
	// Err is the base err
	Err error
	// contains filtered or unexported fields
}

Error is a developer-friendly error wrapper that speaks privacy language. Its base error contains more technical details, and it can be enriched with meta-data, e.g., namespace and subject.

func (Error) Error

func (e Error) Error() string

Error implements error interface.

func (Error) Is

func (e Error) Is(err error) bool

Is compares only messages of Errors to decide whether they are equal. Otherwise, the wrapped error will decide.

func (Error) Message

func (e Error) Message() string

Message returns a short and primary message of the error.

In contrast to Error() method, It doesn't include base error or meta-data in the return.

func (Error) Namespace

func (e Error) Namespace() string

Namespace returns the associated namespace to the error if it exists.

func (Error) Subject

func (e Error) Subject() string

Subject returns the associated subject to the error if it exists.

func (Error) Unwrap

func (e Error) Unwrap() error

type Factory

type Factory interface {

	// Instance creates a new Protector instance for the given namespace or returns the existing one.
	Instance(namespace string) (Protector, FactoryClearFunc)

	// Monitor starts a long-running process in a separate Goroutine.
	// It checks Protectors' activities and removes inactive ones,
	// and clears their caches based on their cache TTL config.
	Monitor(ctx context.Context)
}

Factory manages and maintains a registry of Protector services.

It monitors each Protector service to track its activity and regularly clears encryption materials caches.

func NewFactory

func NewFactory(newProt FactoryNewFunc, opts ...func(*FactoryConfig)) Factory

NewFactory returns a thread-safe factory service instance. It panics if builderFunc is nil. Options params allow overwriting the default configuration.

type FactoryClearFunc

type FactoryClearFunc func()

FactoryClearFunc presents the function returned by Factory.Instance method. It tells the associated Protector instance to immediately clear the cache of encryption materials.

type FactoryConfig

type FactoryConfig struct {

	// IDLE is the duration used to define whether a Protector service is inactive.
	IDLE time.Duration

	// MonitorPeriod is the frequency of the regular checks made by the monitoring process.
	MonitorPeriod time.Duration
}

FactoryConfig presents the configuration of Factory service

type FactoryNewFunc

type FactoryNewFunc func(namespace string) Protector

FactoryNewFunc is used by the Factory service to create Perotector instance per namespace.

type Protector

type Protector interface {

	// Encrypt encrypts Personal data fields of the given structs pointers.
	// It does its best to ensure atomicity in case of multiple structs pointers.
	// It ensures idempotency and only encrypts fields once.
	Encrypt(ctx context.Context, structPts ...interface{}) error

	// Decrypt decrypts Personal data fields of the given structs pointers.
	// It does its best to ensure in case of multiple structs pointers.
	// It ensures idempotency and only decrypts fields once.
	//
	// It replaces the field value with a replacement message, defined in the tag,
	// if the subject is forgotten. Otherwise, the field will be kept empty.
	Decrypt(ctx context.Context, structPts ...interface{}) error

	// Forget removes the associated encryption materials of the given subject,
	// and crypto-erases its Personal data.
	Forget(ctx context.Context, subID string) error

	// Recover allows to recover encryption materials of the given subject.
	// It will fail if the grace period was exceeded, and encryption materials were hard deleted.
	Recover(ctx context.Context, subID string) error

	// Clear clears encryption materials' cache based on cache-related configuration.
	Clear(ctx context.Context, force bool) error
}

Protector presents the service's interface that encrypts, decrypts, and crypto-erases subjects' Personal data.

func NewProtector

func NewProtector(namespace string, engine core.KeyEngine, opts ...func(*ProtectorConfig)) Protector

NewProtector returns a Protector service instance. It requires a Key engine and accepts options to overwrite the default configuration.

It panics if the given engine is nil. It uses a default namespace if the given namespace is empty.

By default, Cache and Graceful mode options are enabled and 'AES 256 GCM' encrypter is used.

type ProtectorConfig

type ProtectorConfig struct {

	// Engine presents an implementation of core.KeyEngine.
	// It manages encryption materials' life-cycle.
	Engine core.KeyEngine

	// Encrypter presents an implementation of core.Encrypter.
	// It allows using a specific encryption algorithm.
	Encrypter core.Encrypter

	// CacheEnabled used to enable/disable cache.
	CacheEnabled bool

	// CacheTTL defines the cache's time to live duration.
	CacheTTL time.Duration

	// GracefulMode allows first to disable the encryption materials during a graceful period.
	// Therefore recovery may succeed. Otherwise, encryption materials are immediately deleted.
	GracefulMode bool
}

ProtectorConfig presents the configuration of Protector service