README
¶
Objectives
Service fq-connector-go is a part of YDB Federative Query system.
It's an extension point making YDB capable of interaction with various external data sources.
You can deploy fq-connector-go alongside with YDB in order to query and join YDB tables
with the data extracted from the external sources.
Currently supported data sources:
- ClickHouse
- PostgreSQL
Usage
Use this command to run Connector with default configuration:
docker run -d \
--name=connector \
-p 50051:50051 \
-p 6060:6060 \
ghcr.io/ydb-platform/fq-connector-go:latest
Or you can mount custom configuration into container:
docker run -d \
--name=connector \
-p 50051:50051 \
-p 6060:6060 \
-v /tmp/example.conf:/usr/local/etc/fq-connector.conf \
ghcr.io/ydb-platform/fq-connector-go:latest
Directories
¶
| Path | Synopsis |
|---|---|
|
api
|
|
|
client
Package client contains code of a simple CLI application that can be used for server debugging.
|
Package client contains code of a simple CLI application that can be used for server debugging. |
|
server
Package server contains the code base of Connector GRPC service - the component of YDB's Federated Query system providing the unified interface to the external data sources.
|
Package server contains the code base of Connector GRPC service - the component of YDB's Federated Query system providing the unified interface to the external data sources. |
|
server/datasource
Package datasource describes the interface that must be satisfied by any external data source accessible through the connector.
|
Package datasource describes the interface that must be satisfied by any external data source accessible through the connector. |
|
server/datasource/rdbms
Package rdbms contains generalized abstractions suitable for use with any relational database.
|
Package rdbms contains generalized abstractions suitable for use with any relational database. |
|
server/datasource/rdbms/clickhouse
Package clickhouse contains code specific for ClickHouse database.
|
Package clickhouse contains code specific for ClickHouse database. |
|
server/datasource/rdbms/postgresql
Package postgresql contains code specific for PostgreSQL database.
|
Package postgresql contains code specific for PostgreSQL database. |
|
server/datasource/rdbms/utils
Package utils contains helper types and functions that can be used by any relational data source.
|
Package utils contains helper types and functions that can be used by any relational data source. |
|
server/datasource/rdbms/ydb
Package ydb contains code specific for YDB database.
|
Package ydb contains code specific for YDB database. |
|
server/datasource/s3
Package s3 contains the implementation of S3 (Simple Storage Service) based data source
|
Package s3 contains the implementation of S3 (Simple Storage Service) based data source |
|
server/paging
Package paging contains logic of splitting incoming data stream into the separate pages or blocks.
|
Package paging contains logic of splitting incoming data stream into the separate pages or blocks. |
|
server/streaming
Package streaming contains the logic of converting the data stream coming from the data source into the data stream sent over the network to the client.
|
Package streaming contains the logic of converting the data stream coming from the data source into the data stream sent over the network to the client. |
|
server/utils
Package utils contains various helpers and utility functions
|
Package utils contains various helpers and utility functions |
|
library
|
|
|
go/core/metrics
Package metrics provides interface collecting performance metrics.
|
Package metrics provides interface collecting performance metrics. |
|
go/core/metrics/collect
dashboard generator for these metrics can be found at: a.yandex-team.ru/arcadia/library/go/yandex/monitoring-dashboards
|
dashboard generator for these metrics can be found at: a.yandex-team.ru/arcadia/library/go/yandex/monitoring-dashboards |
|
go/core/xerrors
package xerrors is a drop in replacement for errors and golang.org/x/xerrors packages and functionally for github.com/pkg/errors.
|
package xerrors is a drop in replacement for errors and golang.org/x/xerrors packages and functionally for github.com/pkg/errors. |
|
go/test/yatest
Package yatest provides access to testing context, when running under ya make -t.
|
Package yatest provides access to testing context, when running under ya make -t. |
|
tests
|
|
Click to show internal directories.
Click to hide internal directories.