datamovement

package
v0.3.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 25, 2022 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package datamovement provides way to read and write in bulk

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type QueryBatch 

type QueryBatch struct {
	URIs []string
	// contains filtered or unexported fields
}

QueryBatch batch of URIs matching a query and relevant meta information

func (*QueryBatch) Client 

func (qb *QueryBatch) Client() *clients.Client

Client used to with forest for the documents

func (*QueryBatch) Timestamp 

func (qb *QueryBatch) Timestamp() string

Timestamp for transaction

type QueryBatcher 

type QueryBatcher struct {
	// contains filtered or unexported fields
}

QueryBatcher reads documents in bulk

func (*QueryBatcher) BatchSize 

func (qbr *QueryBatcher) BatchSize() uint16

BatchSize is the number documents we'll retrieve in a single batch

func (*QueryBatcher) RemoveListener 

func (qbr *QueryBatcher) RemoveListener(listener chan *QueryBatch) *QueryBatcher

RemoveListener remove the a listener

func (*QueryBatcher) Run 

func (qbr *QueryBatcher) Run() *QueryBatcher

Run the QueryBatcher

func (*QueryBatcher) Timestamp 

func (qbr *QueryBatcher) Timestamp() string

Timestamp for the read operation

func (*QueryBatcher) Wait 

func (qbr *QueryBatcher) Wait() *QueryBatcher

Wait on the QueryBatcher to finish

func (*QueryBatcher) WithBatchSize 

func (qbr *QueryBatcher) WithBatchSize(batchSize uint16) *QueryBatcher

WithBatchSize set the batch size

func (*QueryBatcher) WithListener 

func (qbr *QueryBatcher) WithListener(listener chan *QueryBatch) *QueryBatcher

WithListener add a listener channel

func (*QueryBatcher) WithQuery 

func (qbr *QueryBatcher) WithQuery(query handle.Handle) *QueryBatcher

WithQuery add a query

func (*QueryBatcher) WithTransaction 

func (qbr *QueryBatcher) WithTransaction(transaction *util.Transaction) *QueryBatcher

WithTransaction add a transaction

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service is used for the datamovement service

func NewService 

func NewService(client *clients.Client) *Service

NewService returns a new datamovement.Service

func (*Service) QueryBatcher 

func (s *Service) QueryBatcher() *QueryBatcher

QueryBatcher for reading documents in bulk

func (*Service) WriteBatcher 

func (s *Service) WriteBatcher() *WriteBatcher

WriteBatcher for writing documents in bulk

type WriteBatch 

type WriteBatch struct {
	// contains filtered or unexported fields
}

WriteBatch batch of DocumentDescriptions to write to MarkLogic and relevant meta information

func (*WriteBatch) Client 

func (wb *WriteBatch) Client() *clients.Client

Client used to write the documents

func (*WriteBatch) DocumentDescriptions 

func (wb *WriteBatch) DocumentDescriptions() []*documents.DocumentDescription

DocumentDescriptions representations of the documents written

func (*WriteBatch) DocumentsService 

func (wb *WriteBatch) DocumentsService() *documents.Service

DocumentsService used to write the documents

func (*WriteBatch) Response 

func (wb *WriteBatch) Response() handle.ResponseHandle

Response return the write batch response

func (*WriteBatch) Timestamp 

func (wb *WriteBatch) Timestamp() string

Timestamp for the write operation

func (*WriteBatch) WithResponse 

func (wb *WriteBatch) WithResponse(response handle.ResponseHandle) *WriteBatch

WithResponse set write batch response

type WriteBatcher 

type WriteBatcher struct {
	// contains filtered or unexported fields
}

WriteBatcher writes documents in bulk

func (*WriteBatcher) BatchSize 

func (wbr *WriteBatcher) BatchSize() uint16

BatchSize is the number documents we'll create in a single batch

func (*WriteBatcher) RemoveListener 

func (wbr *WriteBatcher) RemoveListener(listener chan<- *WriteBatch) *WriteBatcher

RemoveListener remove the a listener

func (*WriteBatcher) Run 

func (wbr *WriteBatcher) Run() *WriteBatcher

Run the WriteBatcher

func (*WriteBatcher) ThreadCount 

func (wbr *WriteBatcher) ThreadCount() uint8

ThreadCount is the number threads we'll create

func (*WriteBatcher) Timestamp 

func (wbr *WriteBatcher) Timestamp() string

Timestamp for the write operation

func (*WriteBatcher) Wait 

func (wbr *WriteBatcher) Wait() *WriteBatcher

Wait on the WriteBatcher to finish

func (*WriteBatcher) WithBatchSize 

func (wbr *WriteBatcher) WithBatchSize(batchSize uint16) *WriteBatcher

WithBatchSize set the batch size

func (*WriteBatcher) WithListener 

func (wbr *WriteBatcher) WithListener(listener chan<- *WriteBatch) *WriteBatcher

WithListener add a listener channel

func (*WriteBatcher) WithThreadCount 

func (wbr *WriteBatcher) WithThreadCount(threadCount uint8) *WriteBatcher

WithThreadCount set the thread count

func (*WriteBatcher) WithTransaction 

func (wbr *WriteBatcher) WithTransaction(transaction *util.Transaction) *WriteBatcher

WithTransaction perform writes in given transaction

func (*WriteBatcher) WithTransform 

func (wbr *WriteBatcher) WithTransform(transform *util.Transform) *WriteBatcher

WithTransform transform to apply to the documents

func (*WriteBatcher) WithWriteChannel 

func (wbr *WriteBatcher) WithWriteChannel(writeChannel <-chan *documents.DocumentDescription) *WriteBatcher

WithWriteChannel add a channel od documents to write

func (*WriteBatcher) WriteChannel 

func (wbr *WriteBatcher) WriteChannel() <-chan *documents.DocumentDescription

WriteChannel is the channel that documents to write will be pulled from

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.