command
Version:
v0.1.0
Opens a new window with list of versions in this module.
Published: Oct 16, 2017
License: MIT
Opens a new window with license information.
Imports: 8
Opens a new window with list of imports.
Imported by: 0
Opens a new window with list of known importers.
README
¶
S3 Pipeline
The S3 Connector Pipeline performs the following steps:
- Pull records from Kinesis and buffer them untill the desired threshold is met.
- Upload the batch of records to an S3 bucket.
- Set the current Shard checkpoint in Redis.
The pipleline config vars are loaded done with gcfg.
Environment Variables
Export the required environment vars for connecting to the Kinesis stream:
export AWS_ACCESS_KEY=
export AWS_REGION_NAME=
export AWS_SECRET_KEY=
Running the code
$ go run main.go -a appName -s streamName
Documentation
¶
There is no documentation for this package.
Source Files
¶
Click to show internal directories.
Click to hide internal directories.