Documentation
¶
Index ¶
- Variables
- type ErrorHandler
- type Option
- type OptionFunc
- func WithDispatcherLoopWaitDuration(duration time.Duration) OptionFunc
- func WithErrorHandler(f ErrorHandler) OptionFunc
- func WithQueueSize(size int) OptionFunc
- func WithWorkerSize(size int) OptionFunc
- func WithWorkerStack(stack int) OptionFunc
- func WithWorkerWaitDuration(duration time.Duration) OptionFunc
- type Row
- type Streamer
- type TablePeriod
- type TableSchema
- type TableSchemaGroup
Constants ¶
This section is empty.
Variables ¶
Functions ¶
This section is empty.
Types ¶
type ErrorHandler ¶
type ErrorHandler func(error)
type Option ¶ added in v0.5.0
type Option interface {
// contains filtered or unexported methods
}
Option is an interface for dependency injection.
type OptionFunc ¶ added in v0.5.0
type OptionFunc func(cfg *config)
func WithDispatcherLoopWaitDuration ¶ added in v0.5.0
func WithDispatcherLoopWaitDuration(duration time.Duration) OptionFunc
WithDispatcherLoopWaitDuration returns a function which injects worker wait duration to streamer.
func WithErrorHandler ¶ added in v0.5.0
func WithErrorHandler(f ErrorHandler) OptionFunc
WithErrorHandler returns a function which injects error handler to streamer.
func WithQueueSize ¶ added in v0.5.0
func WithQueueSize(size int) OptionFunc
WithQueueSize returns a function which injects queue size to streamer.
func WithWorkerSize ¶ added in v0.5.0
func WithWorkerSize(size int) OptionFunc
WithWorkerSize returns a function which injects worker size to streamer.
func WithWorkerStack ¶ added in v0.5.0
func WithWorkerStack(stack int) OptionFunc
WithWorkerStack returns a function which injects worker stack to streamer.
func WithWorkerWaitDuration ¶ added in v0.5.0
func WithWorkerWaitDuration(duration time.Duration) OptionFunc
WithWorkerWaitDuration returns a function which injects worker wait duration to streamer.
type Row ¶
type Row interface { bigquery.ValueSaver ProjectId() string // GCP ProjectId Schema() *TableSchema // BigQuery Table Schema PublishedAt() time.Time // PublishedAt is time, such as sends row to Bigquery. InsertId() string // InsertId is unique id in BigQuery table. }
type Streamer ¶
type Streamer interface { AddRow(ctx context.Context, row Row) error // Add bigquery row to dispatcher AddRowSync(ctx context.Context, row Row) error // Add bigquery row to dispatcher sync. Start() error // Start Streamer Stop() error // Stop Streamer }
func NewStreamer ¶
func NewStreamer(tableSchemaGroup []*TableSchemaGroup, opts ...Option) (Streamer, error)
NewStreamer returns bigquery streamer which inserts data with bulk parallel.
type TableSchema ¶
func NewTableSchema ¶ added in v0.5.0
func NewTableSchema(projectId string, datasetId, prefix string, meta *bigquery.TableMetadata, period TablePeriod, checksumField bool) (*TableSchema, error)
NewTableSchema creates table schema using bigquery.
type TableSchemaGroup ¶ added in v0.5.0
type TableSchemaGroup struct {
// contains filtered or unexported fields
}
func NewTableSchemaGroup ¶ added in v0.6.0
func NewTableSchemaGroup(projectId, credential string, tableSchemaList []*TableSchema) (*TableSchemaGroup, error)
NewTableSchemaGroup creates table schema group using bigquery.