handler

Documentation

Index

• Constants
• func ApplyMainSheetHeader(excelInMemoryStructure *excelize.File, doLargeSheet bool, ...) error
• func ErrorStack(message string) error
• func ValidateEvent(kafkaEvent *event.CantabularCsvCreated) error
• type DatasetAPIClient
• type Error
  • func NewError(err error, logData map[string]interface{}) *Error
  • func (e *Error) Error() string
  • func (e *Error) LogData() map[string]interface{}
  • func (e *Error) Unwrap() error
• type Generator
• type S3Client
• type VaultClient
• type XlsxCreate
  • func NewXlsxCreate(cfg config.Config, d DatasetAPIClient, sPrivate S3Client, sPublic S3Client, ...) *XlsxCreate
  • func (h *XlsxCreate) AddMetaDataToExcelStructure(ctx context.Context, excelInMemoryStructure *excelize.File, ...) error
  • func (h *XlsxCreate) GetCSVtoExcelStructure(ctx context.Context, excelInMemoryStructure *excelize.File, ...) error
  • func (h *XlsxCreate) GetS3ContentLength(event *event.CantabularCsvCreated, isPublished bool) (int, error)
  • func (h *XlsxCreate) GetVaultKeyForCSVFile(event *event.CantabularCsvCreated) ([]byte, error)
  • func (h *XlsxCreate) Handle(ctx context.Context, workerID int, msg kafka.Message) error
  • func (h *XlsxCreate) IsInstancePublished(ctx context.Context, instanceID string) (bool, error)
  • func (h *XlsxCreate) SaveExcelStructureToExcelFile(ctx context.Context, excelInMemoryStructure *excelize.File, ...) (string, error)
  • func (h *XlsxCreate) StreamAndWrite(ctx context.Context, filenameCsv string, event *event.CantabularCsvCreated, ...) (length int64, err error)
  • func (h *XlsxCreate) UpdateInstance(ctx context.Context, event *event.CantabularCsvCreated, size int, ...) error
  • func (h *XlsxCreate) UploadXLSXFile(ctx context.Context, event *event.CantabularCsvCreated, file io.Reader, ...) (string, error)

Constants

const (
	MaxAllowedRowCount = 999900
)

Functions

func ApplyMainSheetHeader

func ApplyMainSheetHeader(excelInMemoryStructure *excelize.File, doLargeSheet bool, efficientExcelAPIWriter *excelize.StreamWriter, sheet1 string) error

ApplyMainSheetHeader puts relevant header information in first rows of sheet

func ErrorStack

func ErrorStack(message string) error

ErrorStack wraps the message with a stack trace

func ValidateEvent

func ValidateEvent(kafkaEvent *event.CantabularCsvCreated) error

Types

type DatasetAPIClient

type DatasetAPIClient interface {
	PutVersion(ctx context.Context, userAuthToken, serviceAuthToken, collectionID, datasetID, edition, version string, m dataset.Version) error
	GetInstance(ctx context.Context, userAuthToken, serviceAuthToken, collectionID, instanceID, ifMatch string) (i dataset.Instance, eTag string, err error)
	GetVersionMetadataSelection(context.Context, dataset.GetVersionMetadataSelectionInput) (*dataset.Metadata, error)
}

DatasetAPIClient contains the required method for the Dataset API Client

type Error

type Error struct {
	// contains filtered or unexported fields
}

Error is the handler package's error type. Is not meant to be compared as a type, but information should be extracted via the interfaces it implements with callback functions. Is not guaranteed to remain exported so shouldn't be treated as such.

func NewError

func NewError(err error, logData map[string]interface{}) *Error

NewError creates a new Error

func (*Error) Error

func (e *Error) Error() string

Error implements the Go standard error interface

func (*Error) LogData

func (e *Error) LogData() map[string]interface{}

LogData implements the DataLogger interface which allows you extract embedded log.Data from an error

func (*Error) Unwrap

func (e *Error) Unwrap() error

type Generator

type Generator interface {
	NewPSK() ([]byte, error)
}

Generator contains methods for dynamically required strings and tokens e.g. UUIDs, PSKs.

type S3Client

type S3Client interface {
	Get(key string) (io.ReadCloser, *int64, error)
	GetWithPSK(key string, psk []byte) (io.ReadCloser, *int64, error)
	Head(key string) (*s3.HeadObjectOutput, error)
	UploadWithContext(ctx context.Context, input *s3manager.UploadInput, options ...func(*s3manager.Uploader)) (*s3manager.UploadOutput, error)
	UploadWithPSKAndContext(ctx context.Context, input *s3manager.UploadInput, psk []byte, options ...func(*s3manager.Uploader)) (*s3manager.UploadOutput, error)
	BucketName() string
}

S3Client contains the required method for the S3 Client

type VaultClient

type VaultClient interface {
	ReadKey(path, key string) (string, error)
	WriteKey(path, key, value string) error
}

VaultClient contains the required methods for the Vault Client

type XlsxCreate

type XlsxCreate struct {
	// contains filtered or unexported fields
}

XlsxCreate is the handle for the CsvHandler event

func NewXlsxCreate

func NewXlsxCreate(cfg config.Config, d DatasetAPIClient, sPrivate S3Client, sPublic S3Client,
	v VaultClient, g Generator) *XlsxCreate

NewXlsxCreate a new CsvHandler

func (*XlsxCreate) AddMetaDataToExcelStructure

func (h *XlsxCreate) AddMetaDataToExcelStructure(ctx context.Context, excelInMemoryStructure *excelize.File, event *event.CantabularCsvCreated) error

AddMetaDataToExcelStructure reads in the metadata structure and extracts the desired items in the desired order and places them into the metadata sheet of the in-memory excelize library structure

func (*XlsxCreate) GetCSVtoExcelStructure

func (h *XlsxCreate) GetCSVtoExcelStructure(ctx context.Context, excelInMemoryStructure *excelize.File, event *event.CantabularCsvCreated, doLargeSheet bool, efficientExcelAPIWriter *excelize.StreamWriter, sheet1 string, isPublished bool) error

GetCSVtoExcelStructure streams in a line at a time from csv file from S3 bucket and inserts it into the Excel "in memory structure"

func (*XlsxCreate) GetS3ContentLength

func (h *XlsxCreate) GetS3ContentLength(event *event.CantabularCsvCreated, isPublished bool) (int, error)

GetS3ContentLength obtains an S3 file size (in number of bytes) by calling Head Object

func (*XlsxCreate) GetVaultKeyForCSVFile

func (h *XlsxCreate) GetVaultKeyForCSVFile(event *event.CantabularCsvCreated) ([]byte, error)

func (*XlsxCreate) Handle

func (h *XlsxCreate) Handle(ctx context.Context, workerID int, msg kafka.Message) error

Handle takes a single event.

func (*XlsxCreate) IsInstancePublished

func (h *XlsxCreate) IsInstancePublished(ctx context.Context, instanceID string) (bool, error)

func (*XlsxCreate) SaveExcelStructureToExcelFile

func (h *XlsxCreate) SaveExcelStructureToExcelFile(ctx context.Context, excelInMemoryStructure *excelize.File, event *event.CantabularCsvCreated, isPublished bool) (string, error)

SaveExcelStructureToExcelFile uses the excelize library Write function to effectively write out the Excel "in memory structure" to a stream that is then streamed directly into a file in S3 bucket. returns s3Location (path) or Error

func (*XlsxCreate) StreamAndWrite

func (h *XlsxCreate) StreamAndWrite(ctx context.Context, filenameCsv string, event *event.CantabularCsvCreated, w io.Writer, isPublished bool) (length int64, err error)

StreamAndWrite decrypt and stream the request file writing the content to the provided io.Writer.

func (*XlsxCreate) UpdateInstance

func (h *XlsxCreate) UpdateInstance(ctx context.Context, event *event.CantabularCsvCreated, size int, isPublished bool, s3Url string) error

UpdateInstance updates the instance download CSV link using dataset API PUT /instances/{id} endpoint if the instance is published, then the s3Url will be set as public link and the instance state will be set to published otherwise, a private url will be generated and the state will not be changed

func (*XlsxCreate) UploadXLSXFile

func (h *XlsxCreate) UploadXLSXFile(ctx context.Context, event *event.CantabularCsvCreated, file io.Reader, isPublished bool, bucketName string, filename string) (string, error)

UploadXLSXFile uploads the provided file content to AWS S3 returns s3Location (path) or Error