etlutil

Documentation

Overview

Package etlutil contains basic iter utils.

Index

• func GroupByFunc[T, V any](fn func(T) V) dagg.GroupFn[T]
• func GroupByFuncE[T, V any](fn func(T) (V, error)) dagg.GroupFn[T]
• func Reduce[Ta, T any](name string, fn func(Ta, T) Ta, ffn ...func(Ta) any) dagg.OptFn[T]
• type Field
• type Iter
  • func Group[T any](it Iter, gfn dagg.GroupFn[T], opts ...dagg.OptFn[T]) Iter
  • func InnerJoin[L, R any](it1, it2 Iter, fn joinOnFunc[L, R]) Iter
  • func LeftJoin[L, R any](it1, it2 Iter, fn joinOnFunc[L, R]) Iter
  • func OuterJoin[L, R any](it1, it2 Iter, fn joinOnFunc[L, R]) Iter
  • func RightJoin[L, R any](it1, it2 Iter, fn joinOnFunc[L, R]) Iter
  • func Tee(it Iter, fn func(it Iter)) Iter
  • func Unslice[T any](it Iter) Iter
• type JoinData
  • func (j JoinData[L, R]) String() string
• type Row

Functions

func GroupByFunc

func GroupByFunc[T, V any](fn func(T) V) dagg.GroupFn[T]

GroupByFunc accepts a func that will group iterations.

func GroupByFuncE

func GroupByFuncE[T, V any](fn func(T) (V, error)) dagg.GroupFn[T]

GroupByFuncE accepts a function that expects a V type to group iterations and allows an error to be returned.

func Reduce

func Reduce[Ta, T any](name string, fn func(Ta, T) Ta, ffn ...func(Ta) any) dagg.OptFn[T]

Reduce adds a reduce func to aggregator

Types

type Field

type Field = drow.Field

Field is a drow.Field

type Iter

type Iter = etl.Iter

Iter is a iter.Iter

func Group

func Group[T any](it Iter, gfn dagg.GroupFn[T], opts ...dagg.OptFn[T]) Iter

Group groups data and produces a Row with the data.

func InnerJoin

func InnerJoin[L, R any](it1, it2 Iter, fn joinOnFunc[L, R]) Iter

InnerJoin loads it2 into memory and calls fn on each element of it1 and it2.

func LeftJoin

func LeftJoin[L, R any](it1, it2 Iter, fn joinOnFunc[L, R]) Iter

LeftJoin loads it2 into memory and calls fn on each element of it1 and it2. If the fn returns true it will produce a JoinData[L,R] with the left value and optionaly right value

func OuterJoin

func OuterJoin[L, R any](it1, it2 Iter, fn joinOnFunc[L, R]) Iter

OuterJoin loads it2 into memory and check each element of it1 and it2 produces JoinData[L,R] with the left and right value.

func RightJoin

func RightJoin[L, R any](it1, it2 Iter, fn joinOnFunc[L, R]) Iter

RightJoin loads it1 into memory and calls fn on each element of it1 and it2, if the fn returns true it will produce a JoinData[L,R] with the right value and the left value.

func Tee

func Tee(it Iter, fn func(it Iter)) Iter

Tee returns a new iterator that will return all the values from it and also pass them to the iterator in the given function.

func Unslice

func Unslice[T any](it Iter) Iter

Unslice receives a slice of type T and yields each value.

type JoinData

type JoinData[L, R any] struct {
	Left  *L
	Right *R
}

JoinData is the data yielded by the join functions.

func (JoinData[L, R]) String

func (j JoinData[L, R]) String() string

type Row

type Row = drow.Row

Row is a drow.Row