datasource

Documentation

Overview

Package datasource describes the interface that must be satisfied by any external data source accessible through the connector.

Index

• type DataSource
• type DataSourceFactory
• type DataSourceMock
  • func (m *DataSourceMock[T]) DescribeTable(ctx context.Context, logger log.Logger, ...) (*api_service_protos.TDescribeTableResponse, error)
  • func (m *DataSourceMock[T]) ReadSplit(ctx context.Context, logger log.Logger, split *api_service_protos.TSplit, ...)
  • func (m *DataSourceMock[T]) TypeMapper() utils.TypeMapper

Types

type DataSource

type DataSource[T utils.Acceptor] interface {
	// DescribeTable returns metadata about a table (or similair entity in non-relational data sources)
	// located within a particular database in a data source cluster.
	DescribeTable(
		ctx context.Context,
		logger log.Logger,
		request *api_service_protos.TDescribeTableRequest,
	) (*api_service_protos.TDescribeTableResponse, error)

	// ReadSplit is a main method for reading data from the table.
	ReadSplit(
		ctx context.Context,
		logger log.Logger,
		split *api_service_protos.TSplit,
		sink paging.Sink[T],
	)
}

DataSource is an abstraction over external data storage that is available for data and metadata extraction. All new data sources must implement this interface. The types of data extracted from the data source are parametrized via [T utils.Acceptor] interface.

type DataSourceFactory

type DataSourceFactory[T utils.Acceptor] interface {
	Make(
		logger log.Logger,
		dataSourceType api_common.EDataSourceKind,
	) (DataSource[T], error)
}

type DataSourceMock

type DataSourceMock[T utils.Acceptor] struct {
	mock.Mock
}

func (*DataSourceMock[T]) DescribeTable

func (m *DataSourceMock[T]) DescribeTable(
	ctx context.Context,
	logger log.Logger,
	request *api_service_protos.TDescribeTableRequest,
) (*api_service_protos.TDescribeTableResponse, error)

func (*DataSourceMock[T]) ReadSplit

func (m *DataSourceMock[T]) ReadSplit(
	ctx context.Context,
	logger log.Logger,
	split *api_service_protos.TSplit,
	pagingWriter paging.Sink[T],
)

func (*DataSourceMock[T]) TypeMapper

func (m *DataSourceMock[T]) TypeMapper() utils.TypeMapper