dyndump

Documentation

Overview

Package dyndump provides a scan-based method for dumping an entire DynamoDB table using scan.

It supports parallel connections to the server for increased throughput along with rate limiting to a specific read capacity.

Items are written to an ItemWriter interface until the table is exhausted, or the Stop method is called.

Index

• type Fetcher
  • func (f *Fetcher) Run() error
  • func (f *Fetcher) Stats() Stats
  • func (f *Fetcher) Stop()
• type ItemWriter
• type JSONItemEncoder
  • func NewJSONItemEncoder(w io.Writer, encodeTypes, numbersAsStrings bool) *JSONItemEncoder
  • func (jie *JSONItemEncoder) WriteItem(item map[string]*dynamodb.AttributeValue) error
• type Stats

Types

type Fetcher

type Fetcher struct {
	Dyn            *dynamodb.DynamoDB
	TableName      string
	ConsistentRead bool       // Setting to true will use double the read capacity.
	MaxParallel    int        // Maximum number of parallel requests to make to Dynamo.
	MaxItems       int64      // Maximum (approximately) number of items to read from Dynamo.
	ReadCapacity   float64    // Average global read capacity to use for the scan.
	Writer         ItemWriter // Retrieved items are sent to this ItemWriter.
	// contains filtered or unexported fields
}

Fetcher fetches data from DynamoDB at a specified capacity and writes fetched items to a writer implementing the ItemWriter interface.

func (*Fetcher) Run

func (f *Fetcher) Run() error

Run executes the fetcher, starting as many parallel reads as specified by the MaxParallel option and returns when the read has finished, failed, or been stopped.

func (*Fetcher) Stats

func (f *Fetcher) Stats() Stats

Stats returns current aggregate statistics about an ongoing or completed run. It is safe to call from concurrent goroutines.

func (*Fetcher) Stop

func (f *Fetcher) Stop()

Stop requests a clean shutdown of active readers. Active readers will complete the current request and then exit.

type ItemWriter

type ItemWriter interface {
	WriteItem(item map[string]*dynamodb.AttributeValue) error
}

ItemWriter is the interface expected by a Fetcher when writing retrieved DynamoDB items. Implementors must take care to support writes from concurrent goroutines.

type JSONItemEncoder

type JSONItemEncoder struct {
	// contains filtered or unexported fields
}

JSONItemEncoder implements the ItemWriter interface, formatting incoming items as JSON and writing them to the supplied writer.

func NewJSONItemEncoder

func NewJSONItemEncoder(w io.Writer, encodeTypes, numbersAsStrings bool) *JSONItemEncoder

NewJSONItemEncoder returns a JSONItmeEncoder writing to the supplied io.Writer.

func (*JSONItemEncoder) WriteItem

func (jie *JSONItemEncoder) WriteItem(item map[string]*dynamodb.AttributeValue) error

WriteItem JSON encodes a single DynamoDB item.

type Stats

type Stats struct {
	ItemsRead    int64
	CapacityUsed float64
}

Stats is returned by Fetcher.Stats to return current global throughput statistics.