beam-go-example

README

Example beam + Go + Flink

Install system software

1. Follow official Docker instructions for to install docker.

2. Follow the "Portable (Java/Python/Go)" instructions on the beam website.

Run the example

In one terminal:

docker run --net=host apache/beam_flink1.10_job_server:latest

In another terminal:

go run cmd/beamgo_pipeline/beamgo_pipeline.go --runner flink --endpoint localhost:8099 --output /tmp/output-flink.txt  --environment_type LOOPBACK

Regenerate the pb.go file

protoc -I objectpb --go_out=objectpb objectpb/object.proto
mv objectpb/github.com/gonzojive/beam-go-example/objectpb/object.pb.go objectpb/
rm -R objectpb/github.com