storage

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 7, 2016 License: Apache-2.0 Imports: 12 Imported by: 269

Documentation

Overview

Package storage exposes the policy engine's storage layer.

Index

Examples

Constants

View Source 
const (
	// InternalErr indicates an unknown, internal error has occurred.
	InternalErr ErrCode = iota

	// NotFoundErr indicates the path used in the storage operation does not
	// locate a document.
	NotFoundErr = iota

	// InvalidPatchErr indicates an invalid patch/write was issued. The patch
	// was rejected.
	InvalidPatchErr = iota

	// MountConflictErr indicates a mount attempt was made on a path that is
	// already used for a mount.
	MountConflictErr = iota

	// IndexNotFoundErr indicates the caller attempted to use indexing on a
	// reference that has not been indexed.
	IndexNotFoundErr = iota

	// IndexingNotSupportedErr indicates the caller attempted to index a
	// reference provided by a store that does not support indexing.
	IndexingNotSupportedErr = iota

	// TriggersNotSupportedErr indicates the caller attempted to register a
	// trigger against a store that does not support them.
	TriggersNotSupportedErr = iota

	// WritesNotSupportedErr indicate the caller attempted to perform a write
	// against a store that does not support them.
	WritesNotSupportedErr = iota
)
View Source 
const (
	AddOp     PatchOp = iota
	RemoveOp          = iota
	ReplaceOp         = iota
)

Patch supports add, remove, and replace operations.

Variables

This section is empty.

Functions

func DeletePolicy added in v0.2.0 

func DeletePolicy(store *Storage, id string) error

DeletePolicy removes a policy module from storage inside a new transaction.

func GetPolicy added in v0.2.0 

func GetPolicy(store *Storage, id string) (*ast.Module, []byte, error)

GetPolicy returns a policy module from storage inside a new transaction.

func InsertPolicy added in v0.2.0 

func InsertPolicy(store *Storage, id string, mod *ast.Module, raw []byte, persist bool) error

InsertPolicy upserts a policy module into storage inside a new transaction.

func IsInvalidPatch added in v0.2.0 

func IsInvalidPatch(err error) bool

IsInvalidPatch returns true if this error is a InvalidPatchErr.

func IsNotFound 

func IsNotFound(err error) bool

IsNotFound returns true if this error is a NotFoundErr.

func ReadOrDie added in v0.2.0 

func ReadOrDie(store *Storage, path ast.Ref) interface{}

ReadOrDie is a helper function to read the path from storage. If the read fails for any reason, this function will panic. This function should only be used for tests.

Types

type Config added in v0.2.0 

type Config struct {
	Builtin   Store
	PolicyDir string
}

Config represents the configuration for the policy engine's storage layer.

func InMemoryConfig added in v0.2.0 

func InMemoryConfig() Config

InMemoryConfig returns a new Config for an in-memory storage layer.

func InMemoryWithJSONConfig added in v0.2.0 

func InMemoryWithJSONConfig(data map[string]interface{}) Config

InMemoryWithJSONConfig returns a new Config for an in-memory storage layer using existing JSON data. This is primarily for test purposes.

func (Config) WithPolicyDir added in v0.2.0 

func (c Config) WithPolicyDir(dir string) Config

WithPolicyDir returns a new Config with the policy directory configured.

type DataStore 

type DataStore struct {
	// contains filtered or unexported fields
}

DataStore is the backend containing rule references and data.

func NewDataStore 

func NewDataStore() *DataStore

NewDataStore returns an empty DataStore.

func NewDataStoreFromJSONObject 

func NewDataStoreFromJSONObject(data map[string]interface{}) *DataStore

NewDataStoreFromJSONObject returns a new DataStore containing the supplied documents. This is mostly for test purposes.

func NewDataStoreFromReader added in v0.2.0 

func NewDataStoreFromReader(r io.Reader) *DataStore

NewDataStoreFromReader returns a new DataStore from a reader that produces a JSON serialized object. This function is for test purposes.

func (*DataStore) Begin added in v0.2.0 

func (ds *DataStore) Begin(txn Transaction, refs []ast.Ref) error

Begin is called when a new transaction is started.

func (*DataStore) Close added in v0.2.0 

func (ds *DataStore) Close(txn Transaction)

Close is called when a transaction is finished.

func (*DataStore) ID added in v0.2.0 

func (ds *DataStore) ID() string

ID returns a unique identifier for the in-memory store.

func (*DataStore) Read added in v0.2.0 

func (ds *DataStore) Read(txn Transaction, path ast.Ref) (interface{}, error)

Read fetches a value from the in-memory store.

func (*DataStore) Register added in v0.2.0 

func (ds *DataStore) Register(id string, config TriggerConfig) error

Register adds a trigger.

func (*DataStore) SetMountPath added in v0.2.0 

func (ds *DataStore) SetMountPath(ref ast.Ref)

SetMountPath updates the data store's mount path. This is the path the data store expects all references to be prefixed with.

func (*DataStore) String 

func (ds *DataStore) String() string

func (*DataStore) Unregister added in v0.2.0 

func (ds *DataStore) Unregister(id string)

Unregister removes a trigger.

func (*DataStore) Write added in v0.2.0 

func (ds *DataStore) Write(txn Transaction, op PatchOp, path ast.Ref, value interface{}) error

Write modifies a document referred to by path.

type ErrCode 

type ErrCode int

ErrCode represents the collection of errors that may be returned by the storage layer.

type Error 

type Error struct {
	Code    ErrCode
	Message string
}

Error is the error type returned by the storage layer.

func (*Error) Error 

func (err *Error) Error() string

type PatchOp 

type PatchOp int

PatchOp is the enumeration of supposed modifications.

type Storage added in v0.2.0 

type Storage struct {
	// contains filtered or unexported fields
}

Storage represents the policy engine's storage layer.

func New added in v0.2.0 

func New(config Config) *Storage

New returns a new instance of the policy engine's storage layer.

func (*Storage) BuildIndex added in v0.2.0 

func (s *Storage) BuildIndex(txn Transaction, ref ast.Ref) error

BuildIndex causes the storage layer to create an index for the given reference over the snapshot identified by the transaction.

func (*Storage) Close added in v0.2.0 

func (s *Storage) Close(txn Transaction)

Close completes a transaction.

func (*Storage) DeletePolicy added in v0.2.0 

func (s *Storage) DeletePolicy(txn Transaction, id string) error

DeletePolicy removes a policy from the storage layer.

func (*Storage) GetPolicy added in v0.2.0 

func (s *Storage) GetPolicy(txn Transaction, id string) (*ast.Module, []byte, error)

GetPolicy returns the policy module with the given id. The return value includes the raw []byte representation of the policy if it was provided when inserting the policy module.

func (*Storage) Index added in v0.2.0 

func (s *Storage) Index(txn Transaction, ref ast.Ref, value interface{}, iter func(*ast.ValueMap) error) error

Index invokes the iterator with bindings for each variable in the reference that if plugged into the reference, would locate a document with a matching value.

func (*Storage) IndexExists added in v0.2.0 

func (s *Storage) IndexExists(ref ast.Ref) bool

IndexExists returns true if an index has been built for reference.

func (*Storage) InsertPolicy added in v0.2.0 

func (s *Storage) InsertPolicy(txn Transaction, id string, module *ast.Module, raw []byte, persist bool) error

InsertPolicy upserts a policy module into the storage layer. If the policy module already exists, it is replaced. If the persist flag is true, the storage layer will attempt to write the raw policy module content to disk.

func (*Storage) ListPolicies added in v0.2.0 

func (s *Storage) ListPolicies(txn Transaction) map[string]*ast.Module

ListPolicies returns a map of policy modules that have been loaded into the storage layer.

func (*Storage) Mount added in v0.2.0 

func (s *Storage) Mount(backend Store, path ast.Ref) error

Mount adds a store into the storage layer at the given path. If the path conflicts with an existing mount, an error is returned.

func (*Storage) NewTransaction added in v0.2.0 

func (s *Storage) NewTransaction(refs ...ast.Ref) (Transaction, error)

NewTransaction returns a new transcation that can be used to perform reads and writes against a consistent snapshot of the storage layer. The caller can provide a slice of references that may be read during the transaction.

func (*Storage) Open added in v0.2.0 

func (s *Storage) Open() error

Open initializes the storage layer. Open should normally be called immediately after instantiating a new instance of the storage layer. If the storage layer is configured to use in-memory storage and is not persisting policy modules to disk, the call to Open() may be omitted.

Example 
package main

import (
	"fmt"
	"io/ioutil"
	"os"
	"path/filepath"

	"github.com/open-policy-agent/opa/storage"
)

func main() {

	// Define two example modules and write them to disk in a temporary directory.
	ex1 := `

        package opa.example

        p :- q.r != 0

    `

	ex2 := `

        package opa.example

        q = {"r": 100}

    `

	path, err := ioutil.TempDir("", "")
	if err != nil {
		// Handle error.
	}

	defer os.RemoveAll(path)

	if err = ioutil.WriteFile(filepath.Join(path, "ex1.rego"), []byte(ex1), 0644); err != nil {
		// Handle error.
	}

	if err = ioutil.WriteFile(filepath.Join(path, "ex2.rego"), []byte(ex2), 0644); err != nil {
		// Handle error.
	}

	// Instantiate storage layer and configure with a directory to persist policy modules.
	store := storage.New(storage.InMemoryConfig().WithPolicyDir(path))

	if err = store.Open(); err != nil {
		// Handle error.
	}

	// Inspect one of the loaded policies.
	mod, _, err := storage.GetPolicy(store, "ex1.rego")

	if err != nil {
		// Handle error.
	}

	fmt.Println("Expr:", mod.Rules[0].Body[0])

}

Output:

Expr: neq(data.opa.example.q.r, 0)

func (*Storage) Read added in v0.2.0 

func (s *Storage) Read(txn Transaction, path ast.Ref) (interface{}, error)

Read fetches the value in storage referred to by path. The path may refer to multiple stores in which case the storage layer will fetch the values from each store and then stitch together the result.

Example 
package main

import (
	"encoding/json"
	"fmt"

	"github.com/open-policy-agent/opa/ast"
	"github.com/open-policy-agent/opa/storage"
)

func main() {

	// Define some dummy data to initialize the built-in store with.
	exampleInput := `
    {
        "users": [
            {
                "name": "alice",
                "color": "red",
                "likes": ["clouds", "ships"]
            },
            {
                "name": "burt",
                "likes": ["cheese", "wine"]
            }
        ]
    }
    `

	var data map[string]interface{}
	if err := json.Unmarshal([]byte(exampleInput), &data); err != nil {
		// Handle error.
	}

	// Instantiate the storage layer.
	store := storage.New(storage.InMemoryWithJSONConfig(data))

	txn, err := store.NewTransaction()
	if err != nil {
		// Handle error.
	}

	defer store.Close(txn)

	// Read values out of storage.
	v1, err1 := store.Read(txn, ast.MustParseRef("data.users[1].likes[1]"))
	v2, err2 := store.Read(txn, ast.MustParseRef("data.users[0].age"))

	// Inspect the return values.
	fmt.Println("v1:", v1)
	fmt.Println("err1:", err1)
	fmt.Println("v2:", v2)
	fmt.Println("err2:", err2)
	fmt.Println("err2 is not found:", storage.IsNotFound(err2))

}

Output:

v1: wine
err1: <nil>
v2: <nil>
err2: storage error (code: 1): bad path: [users 0 age], document does not exist
err2 is not found: true

func (*Storage) Unmount added in v0.2.0 

func (s *Storage) Unmount(path ast.Ref) error

Unmount removes a store from the storage layer. If the path does not locate an existing mount, an error is returned.

func (*Storage) Write added in v0.2.0 

func (s *Storage) Write(txn Transaction, op PatchOp, ref ast.Ref, value interface{}) error

Write updates a value in storage.

Example 
package main

import (
	"encoding/json"
	"fmt"

	"github.com/open-policy-agent/opa/ast"
	"github.com/open-policy-agent/opa/storage"
)

func main() {

	// Define some dummy data to initialize the DataStore with.
	exampleInput := `
    {
        "users": [
            {
                "name": "alice",
                "color": "red",
                "likes": ["clouds", "ships"]
            },
            {
                "name": "burt",
                "likes": ["cheese", "wine"]
            }
        ]
    }
    `

	var data map[string]interface{}
	if err := json.Unmarshal([]byte(exampleInput), &data); err != nil {
		// Handle error.
	}

	// Create the new DataStore with the dummy data.
	store := storage.New(storage.InMemoryWithJSONConfig(data))

	// Define dummy data to add to the DataStore.
	examplePatch := `{
        "longitude": 82.501389,
        "latitude": -62.338889
    }`

	var patch interface{}

	if err := json.Unmarshal([]byte(examplePatch), &patch); err != nil {
		// Handle error.
	}

	txn, err := store.NewTransaction()
	if err != nil {
		// Handle error.
	}

	defer store.Close(txn)

	// Write values into storage and read result.
	err0 := store.Write(txn, storage.AddOp, ast.MustParseRef("data.users[0].location"), patch)
	v1, err1 := store.Read(txn, ast.MustParseRef("data.users[0].location.latitude"))
	err2 := store.Write(txn, storage.ReplaceOp, ast.MustParseRef("data.users[1].color"), "red")

	// Inspect the return values.
	fmt.Println("err0:", err0)
	fmt.Println("v1:", v1)
	fmt.Println("err1:", err1)
	fmt.Println("err2:", err2)

	// Rollback transaction because write failed.

}

Output:

err0: <nil>
v1: -62.338889
err1: <nil>
err2: storage error (code: 1): bad path: [users 1 color], document does not exist

type Store added in v0.2.0 

type Store interface {
	Trigger

	// Returns a unique identifier for this store. The function should namespace
	// the identifier to avoid potential conflicts, e.g.,
	// com.example/foo-service.
	ID() string

	// Begin is called to indicate that a new transaction has started. The store
	// can use the call to initialize any resources that may be required for the
	// transaction. The caller will provide refs hinting the paths that may be
	// read during the transaction.
	Begin(txn Transaction, refs []ast.Ref) error

	// Read is called to fetch a document referred to by ref.
	Read(txn Transaction, ref ast.Ref) (interface{}, error)

	// Write is called to modify a document referred to by ref.
	Write(txn Transaction, op PatchOp, ref ast.Ref, value interface{}) error

	// Close indicates a transaction has finished. The store can use the call to
	// release any resources temporarily allocated for the transaction.
	Close(txn Transaction)
}

Store defines the interface for the storage layer's backend. Users can implement their own stores and mount them into the storage layer to provide the policy engine access to external data sources.

type Transaction added in v0.2.0 

type Transaction interface {

	// ID returns a unique identifier for this transaction.
	ID() uint64
}

Transaction defines the interface that identifies a consistent snapshot over the policy engine's storage layer.

func NewTransactionOrDie added in v0.2.0 

func NewTransactionOrDie(store *Storage, refs ...ast.Ref) Transaction

NewTransactionOrDie is a helper function to create a new transaction. If the storage layer cannot create a new transaction, this function will panic. This function should only be used for tests.

type Trigger added in v0.2.0 

type Trigger interface {
	Register(id string, config TriggerConfig) error

	// Unregister instructs the trigger to remove the registration.
	Unregister(id string)
}

Trigger defines the interface that stores implement to register for change notifications when data in the store changes.

type TriggerCallback added in v0.2.0 

type TriggerCallback func(txn Transaction, op PatchOp, path []interface{}, value interface{}) error

TriggerCallback defines the interface that callers can implement to handle changes in the stores.

type TriggerConfig added in v0.2.0 

type TriggerConfig struct {

	// Before is called before the change is applied to the store.
	Before TriggerCallback

	// After is called after the change is applied to the store.
	After TriggerCallback
}

TriggerConfig contains the trigger registration configuration.

type TriggersNotSupported added in v0.2.0 

type TriggersNotSupported struct{}

TriggersNotSupported provides default implementations of the Trigger interface which may be used if the backend does not support triggers.

func (TriggersNotSupported) Register added in v0.2.0 

func (TriggersNotSupported) Register(string, TriggerConfig) error

Register always returns an error indicating triggers are not supported.

func (TriggersNotSupported) Unregister added in v0.2.0 

func (TriggersNotSupported) Unregister(string)

Unregister is a no-op.

type WritesNotSupported added in v0.2.0 

type WritesNotSupported struct{}

WritesNotSupported provides a default implementation of the write interface which may be used if the backend does not support writes.

func (WritesNotSupported) Write added in v0.2.0 

func (WritesNotSupported) Write(txn Transaction, op PatchOp, ref ast.Ref, value interface{}) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.