geist-connector-gcp

module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 4, 2024 License: MIT

README

GEIST GCP Connectors

Go Report Card Quality Gate Status Maintainability Rating Reliability Rating Security Rating

Usage

See GEIST core repo for general information.

Note:

  • The BigQuery connector has been moved out to its own repo here.
  • The Bigtable connector has been moved out to its own repo here.

Limitations and improvement areas

Pubsub extractor DLQ

The Kafka Extractor supports automated DLQ handling of unretryable events (e.g. corrupt events that can't be transformed or processed by the Sink), if that option ("dlq") is chosen in the Stream Spec, but the Pubsub Extractor currently only supports the options "discard" and "fail".

Using Pubsub's built-in dead-letter topic option in the subscription as a work-around until this feature is added in the Extractor will currently not give the functionality as provided natively with Geist.

Pubsub extractor microbatching

Also in contrast to Geist Kafka connector, the Pubsub extractor do not yet support the microbatch option as enabled for a stream with the stream spec field ops.microBatch.

Contact

info @ zpiroux . com

License

Geist GCP connectors source code is available under the MIT License.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL