storage

package
v0.4.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 31, 2017 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package storage exposes the policy engine's storage layer.

Index

Examples

Constants

View Source 
const (
	// InternalErr indicates an unknown, internal error has occurred.
	InternalErr = "storage_internal_error"

	// NotFoundErr indicates the path used in the storage operation does not
	// locate a document.
	NotFoundErr = "storage_not_found_error"

	// InvalidPatchErr indicates an invalid patch/write was issued. The patch
	// was rejected.
	InvalidPatchErr = "storage_invalid_patch_error"

	// MountConflictErr indicates a mount attempt was made on a path that is
	// already used for a mount.
	MountConflictErr = "storage_mount_conflict_error"

	// IndexNotFoundErr indicates the caller attempted to use indexing on a
	// reference that has not been indexed.
	IndexNotFoundErr = "storage_index_not_found_error"

	// IndexingNotSupportedErr indicates the caller attempted to index a
	// reference provided by a store that does not support indexing.
	IndexingNotSupportedErr = "storage_indexing_not_supported_error"

	// TriggersNotSupportedErr indicates the caller attempted to register a
	// trigger against a store that does not support them.
	TriggersNotSupportedErr = "storage_triggers_not_supported_error"

	// WritesNotSupportedErr indicate the caller attempted to perform a write
	// against a store that does not support them.
	WritesNotSupportedErr = "storage_writes_not_supported_error"
)
View Source 
const (
	AddOp     PatchOp = iota
	RemoveOp          = iota
	ReplaceOp         = iota
)

Patch supports add, remove, and replace operations.

Variables

This section is empty.

Functions

func DeletePolicy added in v0.2.0 

func DeletePolicy(ctx context.Context, store *Storage, id string) error

DeletePolicy removes a policy module from storage inside a new transaction.

func GetPolicy added in v0.2.0 

func GetPolicy(ctx context.Context, store *Storage, id string) (*ast.Module, []byte, error)

GetPolicy returns a policy module from storage inside a new transaction.

func InsertPolicy added in v0.2.0 

func InsertPolicy(ctx context.Context, store *Storage, id string, mod *ast.Module, raw []byte) error

InsertPolicy upserts a policy module into storage inside a new transaction.

func IsInvalidPatch added in v0.2.0 

func IsInvalidPatch(err error) bool

IsInvalidPatch returns true if this error is a InvalidPatchErr.

func IsNotFound 

func IsNotFound(err error) bool

IsNotFound returns true if this error is a NotFoundErr.

Types

type Config added in v0.2.0 

type Config struct {
	Builtin Store
}

Config represents the configuration for the policy engine's storage layer.

func InMemoryConfig added in v0.2.0 

func InMemoryConfig() Config

InMemoryConfig returns a new Config for an in-memory storage layer.

func InMemoryWithJSONConfig added in v0.2.0 

func InMemoryWithJSONConfig(data map[string]interface{}) Config

InMemoryWithJSONConfig returns a new Config for an in-memory storage layer using existing JSON data. This is primarily for test purposes.

type DataStore 

type DataStore struct {
	// contains filtered or unexported fields
}

DataStore is a simple in-memory data store that implements the storage.Store interface.

func NewDataStore 

func NewDataStore() *DataStore

NewDataStore returns an empty DataStore.

func NewDataStoreFromJSONObject 

func NewDataStoreFromJSONObject(data map[string]interface{}) *DataStore

NewDataStoreFromJSONObject returns a new DataStore containing the supplied documents. This is mostly for test purposes.

func NewDataStoreFromReader added in v0.2.0 

func NewDataStoreFromReader(r io.Reader) *DataStore

NewDataStoreFromReader returns a new DataStore from a reader that produces a JSON serialized object. This function is for test purposes.

func (*DataStore) Begin added in v0.2.0 

func (ds *DataStore) Begin(ctx context.Context, txn Transaction, params TransactionParams) error

Begin is called when a new transaction is started.

func (*DataStore) Close added in v0.2.0 

func (ds *DataStore) Close(ctx context.Context, txn Transaction)

Close is called when a transaction is finished.

func (*DataStore) ID added in v0.2.0 

func (ds *DataStore) ID() string

ID returns a unique identifier for the in-memory store.

func (*DataStore) Read added in v0.2.0 

func (ds *DataStore) Read(ctx context.Context, txn Transaction, path Path) (interface{}, error)

Read fetches a value from the in-memory store.

func (*DataStore) Register added in v0.2.0 

func (ds *DataStore) Register(id string, config TriggerConfig) error

Register adds a trigger.

func (*DataStore) String 

func (ds *DataStore) String() string

func (*DataStore) Unregister added in v0.2.0 

func (ds *DataStore) Unregister(id string)

Unregister removes a trigger.

func (*DataStore) Write added in v0.2.0 

func (ds *DataStore) Write(ctx context.Context, txn Transaction, op PatchOp, path Path, value interface{}) error

Write modifies a document referred to by path.

type Error 

type Error struct {
	Code    string `json:"code"`
	Message string `json:"message"`
}

Error is the error type returned by the storage layer.

func (*Error) Error 

func (err *Error) Error() string

type PatchOp 

type PatchOp int

PatchOp is the enumeration of supposed modifications.

type Path added in v0.3.0 

type Path []string

Path refers to a document in storage.

func MustParsePath added in v0.3.0 

func MustParsePath(s string) Path

MustParsePath returns a new Path for s. If s cannot be parsed, this function will panic. This is mostly for test purposes.

func NewPathForRef added in v0.3.0 

func NewPathForRef(ref ast.Ref) (path Path, err error)

NewPathForRef returns a new path for the given ref.

func ParsePath added in v0.3.0 

func ParsePath(str string) (path Path, ok bool)

ParsePath returns a new path for the given str.

func (Path) Compare added in v0.3.0 

func (p Path) Compare(other Path) (cmp int)

Compare performs lexigraphical comparison on p and other and returns -1 if p is less than other, 0 if p is equal to other, or 1 if p is greater than other.

func (Path) Equal added in v0.3.0 

func (p Path) Equal(other Path) bool

Equal returns true if p is the same as other.

func (Path) HasPrefix added in v0.3.0 

func (p Path) HasPrefix(other Path) bool

HasPrefix returns true if p starts with other.

func (Path) Ref added in v0.3.0 

func (p Path) Ref(head *ast.Term) (ref ast.Ref)

Ref returns a ref that represents p rooted at head.

func (Path) String added in v0.3.0 

func (p Path) String() string

type Storage added in v0.2.0 

type Storage struct {
	// contains filtered or unexported fields
}

Storage represents the policy engine's storage layer.

func New added in v0.2.0 

func New(config Config) *Storage

New returns a new instance of the policy engine's storage layer.

func (*Storage) BuildIndex added in v0.2.0 

func (s *Storage) BuildIndex(ctx context.Context, txn Transaction, ref ast.Ref) error

BuildIndex causes the storage layer to create an index for the given reference over the snapshot identified by the transaction.

func (*Storage) Close added in v0.2.0 

func (s *Storage) Close(ctx context.Context, txn Transaction)

Close completes a transaction.

func (*Storage) DeletePolicy added in v0.2.0 

func (s *Storage) DeletePolicy(txn Transaction, id string) error

DeletePolicy removes a policy from the storage layer.

func (*Storage) GetPolicy added in v0.2.0 

func (s *Storage) GetPolicy(txn Transaction, id string) (*ast.Module, []byte, error)

GetPolicy returns the policy module with the given id. The return value includes the raw []byte representation of the policy if it was provided when inserting the policy module.

func (*Storage) Index added in v0.2.0 

func (s *Storage) Index(txn Transaction, ref ast.Ref, value interface{}, iter func(*ast.ValueMap) error) error

Index invokes the iterator with bindings for each variable in the reference that if plugged into the reference, would locate a document with a matching value.

func (*Storage) IndexExists added in v0.2.0 

func (s *Storage) IndexExists(ref ast.Ref) bool

IndexExists returns true if an index has been built for reference.

func (*Storage) InsertPolicy added in v0.2.0 

func (s *Storage) InsertPolicy(txn Transaction, id string, module *ast.Module, raw []byte) error

InsertPolicy upserts a policy module into the storage layer. If the policy module already exists, it is replaced. If the persist flag is true, the storage layer will attempt to write the raw policy module content to disk.

func (*Storage) ListPolicies added in v0.2.0 

func (s *Storage) ListPolicies(txn Transaction) map[string]*ast.Module

ListPolicies returns a map of policy modules that have been loaded into the storage layer.

func (*Storage) Mount added in v0.2.0 

func (s *Storage) Mount(backend Store, path Path) error

Mount adds a store into the storage layer at the given path. If the path conflicts with an existing mount, an error is returned.

func (*Storage) NewTransaction added in v0.2.0 

func (s *Storage) NewTransaction(ctx context.Context) (Transaction, error)

NewTransaction returns a new Transaction with default parameters.

func (*Storage) NewTransactionWithParams added in v0.3.0 

func (s *Storage) NewTransactionWithParams(ctx context.Context, params TransactionParams) (Transaction, error)

NewTransactionWithParams returns a new Transaction.

func (*Storage) Open added in v0.2.0 

func (s *Storage) Open(ctx context.Context) error

Open initializes the storage layer. Open should normally be called immediately after instantiating a new instance of the storage layer. If the storage layer is configured to use in-memory storage the Open() call can be skiped.

func (*Storage) Read added in v0.2.0 

func (s *Storage) Read(ctx context.Context, txn Transaction, path Path) (interface{}, error)

Read fetches the value in storage referred to by path. The path may refer to multiple stores in which case the storage layer will fetch the values from each store and then stitch together the result.

Example 
package main

import (
	"bytes"
	"context"
	"encoding/json"
	"fmt"

	"github.com/open-policy-agent/opa/storage"
)

func main() {
	// Initialize context for the example. Normally the caller would obtain the
	// context from an input parameter or instantiate their own.
	ctx := context.Background()

	// Define some dummy data to initialize the built-in store with.
	exampleInput := `
    {
        "users": [
            {
                "name": "alice",
                "color": "red",
                "likes": ["clouds", "ships"]
            },
            {
                "name": "burt",
                "likes": ["cheese", "wine"]
            }
        ]
    }
    `

	var data map[string]interface{}

	// OPA uses Go's standard JSON library but assumes that numbers have been
	// decoded as json.Number instead of float64. You MUST decode with UseNumber
	// enabled.
	decoder := json.NewDecoder(bytes.NewBufferString(exampleInput))
	decoder.UseNumber()

	if err := decoder.Decode(&data); err != nil {
		// Handle error.
	}

	// Instantiate the storage layer.
	store := storage.New(storage.InMemoryWithJSONConfig(data))

	txn, err := store.NewTransaction(ctx)
	if err != nil {
		// Handle error.
	}

	defer store.Close(ctx, txn)

	// Read values out of storage.
	v1, err1 := store.Read(ctx, txn, storage.MustParsePath("/users/1/likes/1"))
	v2, err2 := store.Read(ctx, txn, storage.MustParsePath("/users/0/age"))

	// Inspect the return values.
	fmt.Println("v1:", v1)
	fmt.Println("err1:", err1)
	fmt.Println("v2:", v2)
	fmt.Println("err2:", err2)
	fmt.Println("err2 is not found:", storage.IsNotFound(err2))

}

Output:

v1: wine
err1: <nil>
v2: <nil>
err2: storage_not_found_error: /users/0/age: document does not exist
err2 is not found: true

func (*Storage) Unmount added in v0.2.0 

func (s *Storage) Unmount(path Path) error

Unmount removes a store from the storage layer. If the path does not locate an existing mount, an error is returned.

func (*Storage) Write added in v0.2.0 

func (s *Storage) Write(ctx context.Context, txn Transaction, op PatchOp, path Path, value interface{}) error

Write updates a value in storage.

Example 
package main

import (
	"bytes"
	"context"
	"encoding/json"
	"fmt"

	"github.com/open-policy-agent/opa/storage"
)

func main() {
	// Initialize context for the example. Normally the caller would obtain the
	// context from an input parameter or instantiate their own.
	ctx := context.Background()

	// Define some dummy data to initialize the DataStore with.
	exampleInput := `
    {
        "users": [
            {
                "name": "alice",
                "color": "red",
                "likes": ["clouds", "ships"]
            },
            {
                "name": "burt",
                "likes": ["cheese", "wine"]
            }
        ]
    }
    `

	var data map[string]interface{}

	// OPA uses Go's standard JSON library but assumes that numbers have been
	// decoded as json.Number instead of float64. You MUST decode with UseNumber
	// enabled.
	decoder := json.NewDecoder(bytes.NewBufferString(exampleInput))
	decoder.UseNumber()

	if err := decoder.Decode(&data); err != nil {
		// Handle error.
	}

	// Create the new DataStore with the dummy data.
	store := storage.New(storage.InMemoryWithJSONConfig(data))

	// Define dummy data to add to the DataStore.
	examplePatch := `{
        "longitude": 82.501389,
        "latitude": -62.338889
    }`

	var patch interface{}

	// See comment above regarding decoder usage.
	decoder = json.NewDecoder(bytes.NewBufferString(examplePatch))
	decoder.UseNumber()

	if err := decoder.Decode(&patch); err != nil {
		// Handle error.
	}

	txn, err := store.NewTransaction(ctx)
	if err != nil {
		// Handle error.
	}

	defer store.Close(ctx, txn)

	// Write values into storage and read result.
	err0 := store.Write(ctx, txn, storage.AddOp, storage.MustParsePath("/users/0/location"), patch)
	v1, err1 := store.Read(ctx, txn, storage.MustParsePath("/users/0/location/latitude"))
	err2 := store.Write(ctx, txn, storage.ReplaceOp, storage.MustParsePath("/users/1/color"), "red")

	// Inspect the return values.
	fmt.Println("err0:", err0)
	fmt.Println("v1:", v1)
	fmt.Println("err1:", err1)
	fmt.Println("err2:", err2)

	// Rollback transaction because write failed.

}

Output:

err0: <nil>
v1: -62.338889
err1: <nil>
err2: storage_not_found_error: /users/1/color: document does not exist

type Store added in v0.2.0 

type Store interface {
	Trigger

	// Returns a unique identifier for this store. The function should namespace
	// the identifier to avoid potential conflicts, e.g.,
	// com.example/foo-service.
	ID() string

	// Begin is called to indicate that a new transaction has started. The store
	// can use the call to initialize any resources that may be required for the
	// transaction.
	Begin(ctx context.Context, txn Transaction, params TransactionParams) error

	// Read is called to fetch a document referred to by path.
	Read(ctx context.Context, txn Transaction, path Path) (interface{}, error)

	// Write is called to modify a document referred to by path.
	Write(ctx context.Context, txn Transaction, op PatchOp, path Path, value interface{}) error

	// Close indicates a transaction has finished. The store can use the call to
	// release any resources temporarily allocated for the transaction.
	Close(ctx context.Context, txn Transaction)
}

Store defines the interface for the storage layer's backend. Users can implement their own stores and mount them into the storage layer to provide the policy engine access to external data sources.

type Transaction added in v0.2.0 

type Transaction interface {

	// ID returns a unique identifier for this transaction.
	ID() uint64
}

Transaction defines the interface that identifies a consistent snapshot over the policy engine's storage layer.

func NewTransactionOrDie added in v0.2.0 

func NewTransactionOrDie(ctx context.Context, store *Storage) Transaction

NewTransactionOrDie is a helper function to create a new transaction. If the storage layer cannot create a new transaction, this function will panic. This function should only be used for tests.

type TransactionParams added in v0.3.0 

type TransactionParams struct {

	// Paths represents a set of document paths that may be read during the
	// transaction. The paths may be provided by the caller to hint to the
	// storage layer that certain documents could be pre-loaded.
	Paths []Path
}

TransactionParams describes a new transaction.

func NewTransactionParams added in v0.3.0 

func NewTransactionParams() TransactionParams

NewTransactionParams returns a new TransactionParams object.

func (TransactionParams) WithPaths added in v0.3.0 

func (params TransactionParams) WithPaths(paths []Path) TransactionParams

WithPaths returns a new TransactionParams object with the paths set.

type Trigger added in v0.2.0 

type Trigger interface {
	Register(id string, config TriggerConfig) error

	// Unregister instructs the trigger to remove the registration.
	Unregister(id string)
}

Trigger defines the interface that stores implement to register for change notifications when data in the store changes.

type TriggerCallback added in v0.2.0 

type TriggerCallback func(ctx context.Context, txn Transaction, op PatchOp, path Path, value interface{}) error

TriggerCallback defines the interface that callers can implement to handle changes in the stores.

type TriggerConfig added in v0.2.0 

type TriggerConfig struct {

	// Before is called before the change is applied to the store.
	Before TriggerCallback

	// After is called after the change is applied to the store.
	After TriggerCallback
}

TriggerConfig contains the trigger registration configuration.

type TriggersNotSupported added in v0.2.0 

type TriggersNotSupported struct{}

TriggersNotSupported provides default implementations of the Trigger interface which may be used if the backend does not support triggers.

func (TriggersNotSupported) Register added in v0.2.0 

func (TriggersNotSupported) Register(string, TriggerConfig) error

Register always returns an error indicating triggers are not supported.

func (TriggersNotSupported) Unregister added in v0.2.0 

func (TriggersNotSupported) Unregister(string)

Unregister is a no-op.

type WritesNotSupported added in v0.2.0 

type WritesNotSupported struct{}

WritesNotSupported provides a default implementation of the write interface which may be used if the backend does not support writes.

func (WritesNotSupported) Write added in v0.2.0 

func (WritesNotSupported) Write(ctx context.Context, txn Transaction, op PatchOp, path Path, value interface{}) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.