content

Documentation

Index

• type ContentReader
  • func NewContentReader(filePath string, arrayKey string) *ContentReader
  • func (rc *ContentReader) ArrayKey(arrayKey string) *ContentReader
  • func (rc *ContentReader) Close() error
  • func (rc *ContentReader) GetError() error
  • func (rc *ContentReader) GetFilePath() string
  • func (rc *ContentReader) IsEmpty() (bool, error)
  • func (rc *ContentReader) NextRecord(recordOutput interface{}) error
  • func (rc *ContentReader) Reset()
  • func (rc *ContentReader) SetFilePath(newPath string) error
• type ContentWriter
  • func NewContentWriter(arrayKey string, isCompleteFile, useStdout bool) (*ContentWriter, error)
  • func (rw *ContentWriter) Close() error
  • func (rw *ContentWriter) GetError() error
  • func (rw *ContentWriter) GetFilePath() string
  • func (rw *ContentWriter) RemoveOutputFilePath() error
  • func (rw *ContentWriter) SetArrayKey(arrKey string) *ContentWriter
  • func (rw *ContentWriter) Write(record interface{})

Types

type ContentReader

type ContentReader struct {
	// contains filtered or unexported fields
}

Open and read JSON file, find the array key inside it and load its value to the memory in small chunks. Only array objects are valides to be fetch. Each chunk can be fetch using 'GetRecord' (thread-safe).

This technick solve the limit of memory size which may be too small to fit. That way, we can handle big queries result from Artifactory that may not fit in size to the host total memory.

func NewContentReader

func NewContentReader(filePath string, arrayKey string) *ContentReader

func (*ContentReader) ArrayKey

func (rc *ContentReader) ArrayKey(arrayKey string) *ContentReader

func (*ContentReader) Close

func (rc *ContentReader) Close() error

Cleanup the reader data.

func (*ContentReader) GetError

func (rc *ContentReader) GetError() error

func (*ContentReader) GetFilePath

func (rc *ContentReader) GetFilePath() string

func (*ContentReader) IsEmpty

func (rc *ContentReader) IsEmpty() (bool, error)

Return true if the file has more than one element in array.

func (*ContentReader) NextRecord

func (rc *ContentReader) NextRecord(recordOutput interface{}) error

fetch the next chunk into 'recordOutput' param. 'io.EOF' will be returned if no data is left.

func (*ContentReader) Reset

func (rc *ContentReader) Reset()

Initialize the reader to read a file that has already been read (not thread-safe).

func (*ContentReader) SetFilePath

func (rc *ContentReader) SetFilePath(newPath string) error

type ContentWriter

type ContentWriter struct {
	// contains filtered or unexported fields
}

Write a JSON file in small chunks. Only a single JSON key can be written to the file, and array as its value, the array's values could be any JSON value types (number, string, etc...). Once the first 'Write" call is made, the file will stay open, waiting for the next struct to be written (thread-safe). Finally, 'Close' will fill the end of the JSON file and the operation will be completed.

func NewContentWriter

func NewContentWriter(arrayKey string, isCompleteFile, useStdout bool) (*ContentWriter, error)

func (*ContentWriter) Close

func (rw *ContentWriter) Close() error

Finish writing to the file.

func (*ContentWriter) GetError

func (rw *ContentWriter) GetError() error

func (*ContentWriter) GetFilePath

func (rw *ContentWriter) GetFilePath() string

func (*ContentWriter) RemoveOutputFilePath

func (rw *ContentWriter) RemoveOutputFilePath() error

func (*ContentWriter) SetArrayKey

func (rw *ContentWriter) SetArrayKey(arrKey string) *ContentWriter

func (*ContentWriter) Write

func (rw *ContentWriter) Write(record interface{})

Write a single item to the JSON array.