dataloader

package module
v5.0.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 5, 2018 License: MIT Imports: 7 Imported by: 122

README

DataLoader

GoDoc Build Status

This is an implementation of Facebook's DataLoader in Golang.

Install

go get -u github.com/graph-gophers/dataloader

Usage

// setup batch function
batchFn := func(ctx context.Context, keys dataloader.Keys) []*dataloader.Result {
  var results []*dataloader.Result
  // do some aync work to get data for specified keys
  // append to this list resolved values
  return results
}

// create Loader with an in-memory cache
loader := dataloader.NewBatchedLoader(batchFn)

/**
 * Use loader
 *
 * A thunk is a function returned from a function that is a
 * closure over a value (in this case an interface value and error).
 * When called, it will block until the value is resolved.
 */
thunk := loader.Load(ctx.TODO(), dataloader.StringKey("key1")) // StringKey is a convenience method that make wraps string to implement `Key` interface
result, err := thunk()
if err != nil {
  // handle data error
}

log.Printf("value: %#v", result)
Don't need/want to use context?

You're welcome to install the v1 version of this library.

Cache

This implementation contains a very basic cache that is intended only to be used for short lived DataLoaders (i.e. DataLoaders that ony exsist for the life of an http request). You may use your own implementation if you want.

it also has a NoCache type that implements the cache interface but all methods are noop. If you do not wish to cache anyting.

Examples

There are a few basic examples in the example folder.

Documentation

Overview

Package dataloader is an implimentation of facebook's dataloader in go. See https://github.com/facebook/dataloader for more information

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BatchFunc 

type BatchFunc func(context.Context, Keys) []*Result

BatchFunc is a function, which when given a slice of keys (string), returns an slice of `results`. It's important that the length of the input keys matches the length of the output results.

The keys passed to this function are guaranteed to be unique

type Cache 

type Cache interface {
	Get(context.Context, Key) (Thunk, bool)
	Set(context.Context, Key, Thunk)
	Delete(context.Context, Key) bool
	Clear()
}

The Cache interface. If a custom cache is provided, it must implement this interface.

type InMemoryCache 

type InMemoryCache struct {
	// contains filtered or unexported fields
}

InMemoryCache is an in memory implementation of Cache interface. this simple implementation is well suited for a "per-request" dataloader (i.e. one that only lives for the life of an http request) but it not well suited for long lived cached items.

func NewCache 

func NewCache() *InMemoryCache

NewCache constructs a new InMemoryCache

func (*InMemoryCache) Clear 

func (c *InMemoryCache) Clear()

Clear clears the entire cache

func (*InMemoryCache) Delete 

func (c *InMemoryCache) Delete(_ context.Context, key Key) bool

Delete deletes item at `key` from cache

func (*InMemoryCache) Get 

func (c *InMemoryCache) Get(_ context.Context, key Key) (Thunk, bool)

Get gets the value at `key` if it exsits, returns value (or nil) and bool indicating of value was found

func (*InMemoryCache) Set 

func (c *InMemoryCache) Set(_ context.Context, key Key, value Thunk)

Set sets the `value` at `key` in the cache

type Interface 

type Interface interface {
	Load(context.Context, Key) Thunk
	LoadMany(context.Context, Keys) ThunkMany
	Clear(context.Context, Key) Interface
	ClearAll() Interface
	Prime(ctx context.Context, key Key, value interface{}) Interface
}

Interface is a `DataLoader` Interface which defines a public API for loading data from a particular data back-end with unique keys such as the `id` column of a SQL table or document name in a MongoDB database, given a batch loading function.

Each `DataLoader` instance should contain a unique memoized cache. Use caution when used in long-lived applications or those which serve many users with different access permissions and consider creating a new instance per web request.

type Key 

type Key interface {
	// String returns a guaranteed unique string that can be used to identify an object
	String() string
	// Raw returns the raw, underlaying value of the key
	Raw() interface{}
}

Key is the interface that all keys need to implement

type Keys 

type Keys []Key

Keys wraps a slice of Key types to provide some convenience methods.

func NewKeysFromStrings 

func NewKeysFromStrings(strings []string) Keys

NewKeysFromStrings converts a `[]strings` to a `Keys` ([]Key)

func (Keys) Keys 

func (l Keys) Keys() []string

Keys returns the list of strings. One for each "Key" in the list

type Loader 

type Loader struct {
	// contains filtered or unexported fields
}

Loader implements the dataloader.Interface.

func NewBatchedLoader 

func NewBatchedLoader(batchFn BatchFunc, opts ...Option) *Loader

NewBatchedLoader constructs a new Loader with given options.

func (*Loader) Clear 

func (l *Loader) Clear(ctx context.Context, key Key) Interface

Clear clears the value at `key` from the cache, it it exsits. Returs self for method chaining

func (*Loader) ClearAll 

func (l *Loader) ClearAll() Interface

ClearAll clears the entire cache. To be used when some event results in unknown invalidations. Returns self for method chaining.

func (*Loader) Load 

func (l *Loader) Load(originalContext context.Context, key Key) Thunk

Load load/resolves the given key, returning a channel that will contain the value and error

func (*Loader) LoadMany 

func (l *Loader) LoadMany(originalContext context.Context, keys Keys) ThunkMany

LoadMany loads mulitiple keys, returning a thunk (type: ThunkMany) that will resolve the keys passed in.

func (*Loader) Prime 

func (l *Loader) Prime(ctx context.Context, key Key, value interface{}) Interface

Prime adds the provided key and value to the cache. If the key already exists, no change is made. Returns self for method chaining

type NoCache 

type NoCache struct{}

NoCache implements Cache interface where all methods are noops. This is useful for when you don't want to cache items but still want to use a data loader

func (*NoCache) Clear 

func (c *NoCache) Clear()

Clear is a NOOP

func (*NoCache) Delete 

func (c *NoCache) Delete(context.Context, Key) bool

Delete is a NOOP

func (*NoCache) Get 

func (c *NoCache) Get(context.Context, Key) (Thunk, bool)

Get is a NOOP

func (*NoCache) Set 

func (c *NoCache) Set(context.Context, Key, Thunk)

Set is a NOOP

type NoopTracer 

type NoopTracer struct{}

NoopTracer is the default (noop) tracer

func (NoopTracer) TraceBatch 

func (NoopTracer) TraceBatch(ctx context.Context, keys Keys) (context.Context, TraceBatchFinishFunc)

TraceBatch is a noop function

func (NoopTracer) TraceLoad 

func (NoopTracer) TraceLoad(ctx context.Context, key Key) (context.Context, TraceLoadFinishFunc)

TraceLoad is a noop function

func (NoopTracer) TraceLoadMany 

func (NoopTracer) TraceLoadMany(ctx context.Context, keys Keys) (context.Context, TraceLoadManyFinishFunc)

TraceLoadMany is a noop function

type OpenTracingTracer 

type OpenTracingTracer struct{}

OpenTracing Tracer implements a tracer that can be used with the Open Tracing standard.

func (OpenTracingTracer) TraceBatch 

func (OpenTracingTracer) TraceBatch(ctx context.Context, keys Keys) (context.Context, TraceBatchFinishFunc)

TraceBatch will trace a call to dataloader.LoadMany with Open Tracing

func (OpenTracingTracer) TraceLoad 

func (OpenTracingTracer) TraceLoad(ctx context.Context, key Key) (context.Context, TraceLoadFinishFunc)

TraceLoad will trace a call to dataloader.LoadMany with Open Tracing

func (OpenTracingTracer) TraceLoadMany 

func (OpenTracingTracer) TraceLoadMany(ctx context.Context, keys Keys) (context.Context, TraceLoadManyFinishFunc)

TraceLoadMany will trace a call to dataloader.LoadMany with Open Tracing

type Option 

type Option func(*Loader)

Option allows for configuration of Loader fields.

func WithBatchCapacity 

func WithBatchCapacity(c int) Option

WithBatchCapacity sets the batch capacity. Default is 0 (unbounded).

func WithCache 

func WithCache(c Cache) Option

WithCache sets the BatchedLoader cache. Defaults to InMemoryCache if a Cache is not set.

func WithClearCacheOnBatch 

func WithClearCacheOnBatch() Option

WithClearCacheOnBatch allows batching of items but no long term caching. It accomplishes this by clearing the cache after each batch operation.

func WithInputCapacity 

func WithInputCapacity(c int) Option

WithInputCapacity sets the input capacity. Default is 1000.

func WithOpenTracingTracer 

func WithOpenTracingTracer() Option

WithOpenTracingTracer allows tracing of calls to Load and LoadMany

func WithTracer 

func WithTracer(tracer Tracer) Option

WithTracer allows tracing of calls to Load and LoadMany

func WithWait 

func WithWait(d time.Duration) Option

WithWait sets the amount of time to wait before triggering a batch. Default duration is 16 milliseconds.

type Result 

type Result struct {
	Data  interface{}
	Error error
}

Result is the data structure that a BatchFunc returns. It contains the resolved data, and any errors that may have occurred while fetching the data.

type ResultMany 

type ResultMany struct {
	Data  []interface{}
	Error []error
}

ResultMany is used by the LoadMany method. It contains a list of resolved data and a list of errors. The lengths of the data list and error list will match, and elements at each index correspond to each other.

type StringKey 

type StringKey string

StringKey implements the Key interface for a string

func (StringKey) Raw 

func (k StringKey) Raw() interface{}

String is an identity method. Used to implement Key Raw

func (StringKey) String 

func (k StringKey) String() string

String is an identity method. Used to implement String interface

type Thunk 

type Thunk func() (interface{}, error)

Thunk is a function that will block until the value (*Result) it contains is resolved. After the value it contains is resolved, this function will return the result. This function can be called many times, much like a Promise is other languages. The value will only need to be resolved once so subsequent calls will return immediately.

type ThunkMany 

type ThunkMany func() ([]interface{}, []error)

ThunkMany is much like the Thunk func type but it contains a list of results.

type TraceBatchFinishFunc 

type TraceBatchFinishFunc func([]*Result)

type TraceLoadFinishFunc 

type TraceLoadFinishFunc func(Thunk)

type TraceLoadManyFinishFunc 

type TraceLoadManyFinishFunc func(ThunkMany)

type Tracer 

type Tracer interface {
	// TraceLoad will trace the calls to Load
	TraceLoad(ctx context.Context, key Key) (context.Context, TraceLoadFinishFunc)
	// TraceLoadMany will trace the calls to LoadMany
	TraceLoadMany(ctx context.Context, keys Keys) (context.Context, TraceLoadManyFinishFunc)
	// TraceBatch will trace data loader batches
	TraceBatch(ctx context.Context, keys Keys) (context.Context, TraceBatchFinishFunc)
}

Tracer is an interface that may be used to implement tracing.

Source Files

View all Source files

Directories

Path Synopsis
example
lru-cache
This is an exmaple of using go-cache as a long term cache solution for dataloader.
 This is an exmaple of using go-cache as a long term cache solution for dataloader.
no-cache
ttl-cache
This is an exmaple of using go-cache as a long term cache solution for dataloader.
 This is an exmaple of using go-cache as a long term cache solution for dataloader.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.