pull

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 19, 2021 License: GPL-3.0 Imports: 6 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IncFiltersCount added in v1.4.0 

func IncFiltersCount()

func IncLinesPerStepCount added in v1.4.0 

func IncLinesPerStepCount(step string)

func Reset added in v1.4.0 

func Reset()

Reset all statistics to zero

Types

type Cycle 

type Cycle interface {
	RelationList
}

Cycle is a list of relations.

func NewCycle 

func NewCycle(relations []Relation) Cycle

NewCycle initialize a new Cycle object

type CycleList 

type CycleList interface {
	Len() uint
	Cycle(idx uint) Cycle
}

CycleList is a list of cycles.

func NewCycleList 

func NewCycleList(cycles []Cycle) CycleList

NewCycleList initialize a new CycleList object

type DataSource 

type DataSource interface {
	Open() *Error
	RowReader(source Table, filter Filter) (RowReader, *Error)
	Close() *Error
}

DataSource to read in the pull process.

type DataSourceFactory 

type DataSourceFactory interface {
	New(url string, schema string) DataSource
}

DataSourceFactory exposes methods to create new datasources.

type Error 

type Error struct {
	Description string
}

Error is the error type returned by the domain

func Pull 

func Pull(plan Plan, filters RowReader, source DataSource, exporter RowExporter, diagnostic TraceListener) *Error

Pull data from source following the given puller plan.

func (*Error) Error 

func (e *Error) Error() string

type ExecutionStats added in v1.4.0 

type ExecutionStats interface {
	GetLinesPerStepCount() map[string]int
	GetFiltersCount() int

	ToJSON() []byte
}

func Compute added in v1.4.0 

func Compute() ExecutionStats

Compute current statistics and give a snapshot

type Filter 

type Filter interface {
	Limit() uint
	Values() Row
	Where() string
}

Filter applied to data tables.

func NewFilter 

func NewFilter(limit uint, values Row, where string) Filter

NewFilter initialize a new Filter object

type MockCycle 

type MockCycle struct {
	mock.Mock
}

MockCycle is an autogenerated mock type for the Cycle type

func (*MockCycle) Len 

func (_m *MockCycle) Len() uint

Len provides a mock function with given fields:

func (*MockCycle) Relation 

func (_m *MockCycle) Relation(idx uint) Relation

Relation provides a mock function with given fields: idx

type MockCycleList 

type MockCycleList struct {
	mock.Mock
}

MockCycleList is an autogenerated mock type for the CycleList type

func (*MockCycleList) Cycle 

func (_m *MockCycleList) Cycle(idx uint) Cycle

Cycle provides a mock function with given fields: idx

func (*MockCycleList) Len 

func (_m *MockCycleList) Len() uint

Len provides a mock function with given fields:

type MockDataSource 

type MockDataSource struct {
	mock.Mock
}

MockDataSource is an autogenerated mock type for the DataSource type

func (*MockDataSource) Close 

func (_m *MockDataSource) Close() *Error

Close provides a mock function with given fields:

func (*MockDataSource) Open 

func (_m *MockDataSource) Open() *Error

Open provides a mock function with given fields:

func (*MockDataSource) RowReader 

func (_m *MockDataSource) RowReader(source Table, filter Filter) (RowReader, *Error)

RowReader provides a mock function with given fields: source, filter

type MockDataSourceFactory 

type MockDataSourceFactory struct {
	mock.Mock
}

MockDataSourceFactory is an autogenerated mock type for the DataSourceFactory type

func (*MockDataSourceFactory) New 

func (_m *MockDataSourceFactory) New(url string, schema string) DataSource

New provides a mock function with given fields: url, schema

type MockFilter 

type MockFilter struct {
	mock.Mock
}

MockFilter is an autogenerated mock type for the Filter type

func (*MockFilter) Limit 

func (_m *MockFilter) Limit() uint

Limit provides a mock function with given fields:

func (*MockFilter) Values 

func (_m *MockFilter) Values() Row

Values provides a mock function with given fields:

type MockPlan 

type MockPlan struct {
	mock.Mock
}

MockPlan is an autogenerated mock type for the Plan type

func (*MockPlan) InitFilter 

func (_m *MockPlan) InitFilter() Filter

InitFilter provides a mock function with given fields:

func (*MockPlan) Steps 

func (_m *MockPlan) Steps() StepList

Steps provides a mock function with given fields:

type MockRelation 

type MockRelation struct {
	mock.Mock
}

MockRelation is an autogenerated mock type for the Relation type

func (*MockRelation) Child 

func (_m *MockRelation) Child() Table

Child provides a mock function with given fields:

func (*MockRelation) ChildKey 

func (_m *MockRelation) ChildKey() []string

ChildKey provides a mock function with given fields:

func (*MockRelation) Name 

func (_m *MockRelation) Name() string

Name provides a mock function with given fields:

func (*MockRelation) OppositeOf 

func (_m *MockRelation) OppositeOf(tablename string) Table

OppositeOf provides a mock function with given fields: tablename

func (*MockRelation) Parent 

func (_m *MockRelation) Parent() Table

Parent provides a mock function with given fields:

func (*MockRelation) ParentKey 

func (_m *MockRelation) ParentKey() []string

ParentKey provides a mock function with given fields:

type MockRelationList 

type MockRelationList struct {
	mock.Mock
}

MockRelationList is an autogenerated mock type for the RelationList type

func (*MockRelationList) Len 

func (_m *MockRelationList) Len() uint

Len provides a mock function with given fields:

func (*MockRelationList) Relation 

func (_m *MockRelationList) Relation(idx uint) Relation

Relation provides a mock function with given fields: idx

type MockRowExporter 

type MockRowExporter struct {
	mock.Mock
}

MockRowExporter is an autogenerated mock type for the RowExporter type

func (*MockRowExporter) Export 

func (_m *MockRowExporter) Export(_a0 Row) *Error

Export provides a mock function with given fields: _a0

type MockRowReader 

type MockRowReader struct {
	mock.Mock
}

MockRowReader is an autogenerated mock type for the RowReader type

func (*MockRowReader) Next 

func (_m *MockRowReader) Next() bool

Next provides a mock function with given fields:

func (*MockRowReader) Value 

func (_m *MockRowReader) Value() (Row, *Error)

Value provides a mock function with given fields:

type MockStep 

type MockStep struct {
	mock.Mock
}

MockStep is an autogenerated mock type for the Step type

func (*MockStep) Cycles 

func (_m *MockStep) Cycles() CycleList

Cycles provides a mock function with given fields:

func (*MockStep) Entry 

func (_m *MockStep) Entry() Table

Entry provides a mock function with given fields:

func (*MockStep) Follow 

func (_m *MockStep) Follow() Relation

Follow provides a mock function with given fields:

func (*MockStep) Index 

func (_m *MockStep) Index() uint

Index provides a mock function with given fields:

func (*MockStep) NextSteps 

func (_m *MockStep) NextSteps() StepList

NextSteps provides a mock function with given fields:

func (*MockStep) Relations 

func (_m *MockStep) Relations() RelationList

Relations provides a mock function with given fields:

type MockStepList 

type MockStepList struct {
	mock.Mock
}

MockStepList is an autogenerated mock type for the StepList type

func (*MockStepList) Len 

func (_m *MockStepList) Len() uint

Len provides a mock function with given fields:

func (*MockStepList) Step 

func (_m *MockStepList) Step(_a0 uint) Step

Step provides a mock function with given fields: _a0

type MockTable 

type MockTable struct {
	mock.Mock
}

MockTable is an autogenerated mock type for the Table type

func (*MockTable) Name 

func (_m *MockTable) Name() string

Name provides a mock function with given fields:

func (*MockTable) PrimaryKey 

func (_m *MockTable) PrimaryKey() []string

PrimaryKey provides a mock function with given fields:

type MockTraceListener 

type MockTraceListener struct {
	mock.Mock
}

MockTraceListener is an autogenerated mock type for the TraceListener type

func (*MockTraceListener) TraceStep 

func (_m *MockTraceListener) TraceStep(_a0 Step, _a1 Filter) TraceListener

TraceStep provides a mock function with given fields: _a0, _a1

type MockValue 

type MockValue struct {
	mock.Mock
}

MockValue is an autogenerated mock type for the Value type

type NoTraceListener 

type NoTraceListener struct{}

NoTraceListener default implementation do nothing.

func (NoTraceListener) TraceStep 

func (t NoTraceListener) TraceStep(s Step, filter Filter) TraceListener

TraceStep catch Step event.

type OneEmptyRowReader 

type OneEmptyRowReader struct {
	// contains filtered or unexported fields
}

OneOneEmptyRowReader return one empty row

func NewOneEmptyRowReader 

func NewOneEmptyRowReader() *OneEmptyRowReader

func (OneEmptyRowReader) Error 

func (r OneEmptyRowReader) Error() *Error

Error return always nil

func (*OneEmptyRowReader) Next 

func (r *OneEmptyRowReader) Next() bool

Next is always false except for the first one

func (OneEmptyRowReader) Value 

func (r OneEmptyRowReader) Value() Row

Value is always an empty row

type Plan 

type Plan interface {
	InitFilter() Filter
	Steps() StepList
}

Plan of the puller process.

func NewPlan 

func NewPlan(filter Filter, steps StepList) Plan

NewPlan initialize a new Plan object

type Relation 

type Relation interface {
	Name() string
	Parent() Table
	Child() Table
	ParentKey() []string
	ChildKey() []string
	OppositeOf(tablename string) Table
}

Relation between two tables.

func NewRelation 

func NewRelation(name string, parent Table, child Table, parentKey []string, childKey []string) Relation

NewRelation initialize a new Relation object

type RelationList 

type RelationList interface {
	Len() uint
	Relation(idx uint) Relation
}

RelationList is a list of relations.

func NewRelationList 

func NewRelationList(relations []Relation) RelationList

NewRelationList initialize a new RelationList object

type Row 

type Row map[string]Value

Row of data.

func (Row) Update 

func (r Row) Update(other Row) Row

Update Row with an other Row to generate a new one

type RowExporter 

type RowExporter interface {
	Export(Row) *Error
}

RowExporter receives pulled rows one by one.

type RowReader 

type RowReader interface {
	Next() bool
	Value() Row
	Error() *Error
}

RowReader over DataSource.

type Step 

type Step interface {
	Index() uint
	Entry() Table
	Follow() Relation
	Relations() RelationList
	Cycles() CycleList
	NextSteps() StepList
}

Step group of follows to perform.

func NewStep 

func NewStep(index uint, entry Table, follow Relation, relations RelationList, cycles CycleList, nextSteps StepList) Step

NewStep initialize a new Step object

type StepList 

type StepList interface {
	Len() uint
	Step(uint) Step
}

StepList list of steps to perform.

func NewStepList 

func NewStepList(steps []Step) StepList

NewStepList initialize a new StepList object

type Table 

type Table interface {
	Name() string
	PrimaryKey() []string
}

Table from which to pull data.

func NewTable 

func NewTable(name string, pk []string) Table

NewTable initialize a new Table object

type TraceListener 

type TraceListener interface {
	TraceStep(Step, Filter) TraceListener
}

TraceListener receives diagnostic trace

type Value 

type Value interface{}

Value is an untyped data.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.