gzipio

README

go-apachebeam-gzipio

Overview

This project is to transform for reading and writing gzip files in apache beam.
And it is using to official Go SDK on apache beam.

Getting Started

import (
   gzipio "github.com/gjbae1212/go-apachebeam-gzipio"
)

p, s := beam.NewPipelineWithRoot()
// Read files
gzipio.Read(s, "read-path")

// Write file
gzipio.Write(s, "write-path", pcollection)

License

This project is licensed under the MIT License

Documentation

Index

• func Read(s beam.Scope, glob string) beam.PCollection
• func Write(s beam.Scope, filename string, col beam.PCollection)

Functions

func Read

func Read(s beam.Scope, glob string) beam.PCollection

Read is reading from glob

func Write

func Write(s beam.Scope, filename string, col beam.PCollection)

Write writes data to filename with gzip compress