parquetinfo

Documentation

Index

• Constants
• func AddAnthaInfo(meta parquet.FileKeyValueMetadata, dataInfo schemas.DataInfo) error
• func GetAnthaInfo(meta parquet.FileKeyValueMetadata) (schemas.DataInfo, error)

Constants

const ParquetInfoKey = "antha:data-info"

ParquetInfoKey is used to embed JSON DataInfo in a Parquet file footer as `key_value_metadata`.

Functions

func AddAnthaInfo

func AddAnthaInfo(meta parquet.FileKeyValueMetadata, dataInfo schemas.DataInfo) error

AddAnthaInfo inserts the JSON DataInfo representation in the footer when writing a file. Use it like:

meta := parquet.FileKeyValueMetadata{}
_ = parquetinfo.AddAnthaInfo(meta, dataInfo)
parquet.TableToBytes(table, meta.Write())

func GetAnthaInfo

func GetAnthaInfo(meta parquet.FileKeyValueMetadata) (schemas.DataInfo, error)

GetAnthaInfo reads the JSON DataInfo representation from the footer. Use it together with data tables like:

     meta := parquet.FileKeyValueMetadata{}
		table, err = parquet.TableFromReader(reader, meta.Read())
		dataInfo, err := GetAnthaInfo(meta)