README
¶
GEIST GCP Connectors
Usage
See GEIST core repo for general information.
Note:
- The BigQuery connector has been moved out to its own repo here.
- The Bigtable connector has been moved out to its own repo here.
Limitations and improvement areas
Pubsub extractor DLQ
The Kafka Extractor supports automated DLQ handling of unretryable events (e.g. corrupt events that can't be transformed or processed by the Sink), if that option ("dlq") is chosen in the Stream Spec, but the Pubsub Extractor currently only supports the options "discard" and "fail".
Using Pubsub's built-in dead-letter topic option in the subscription as a work-around until this feature is added in the Extractor will currently not give the functionality as provided natively with Geist.
Pubsub extractor microbatching
Also in contrast to Geist Kafka connector, the Pubsub extractor do not yet support the microbatch option as enabled for a stream with the stream spec field ops.microBatch.
Contact
info @ zpiroux . com
License
Geist GCP connectors source code is available under the MIT License.
Directories