util

package
v0.0.0-...-bc12874 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 21, 2024 License: BSD-3-Clause Imports: 23 Imported by: 18

Documentation

Index

Constants

View Source 
const (
	// time.RFC3339Nano only uses as many sub-second digits are required to
	// represent the time, which makes it unsuitable for sorting. This
	// format ensures that all 9 nanosecond digits are used, padding with
	// zeroes if necessary.
	RFC3339NanoZeroPad = "2006-01-02T15:04:05.000000000Z07:00"

	// SAFE_TIMESTAMP_FORMAT is time format which is similar to
	// RFC3339NanoZeroPad, but with most of the punctuation omitted. This
	// timestamp can only be used to format and parse times in UTC.
	SAFE_TIMESTAMP_FORMAT = "20060102T150405.000000000Z"
)

Variables

This section is empty.

Functions

func AbsInt 

func AbsInt(v int) int

AbsInt returns the absolute value of v.

func AddParams 

func AddParams(a map[string]string, b ...map[string]string) map[string]string

AddParams adds the second instance of map[string]string to the first and returns the first map.

func AskForConfirmation 

func AskForConfirmation(format string, args ...interface{}) (bool, error)

AskForConfirmation waits for the user to type "y" or "n".

func ChunkIter 

func ChunkIter(length, chunkSize int, fn func(startIdx int, endIdx int) error) error

ChunkIter iterates over a slice in chunks of smaller slices.

func ChunkIterParallel 

func ChunkIterParallel(ctx context.Context, length, chunkSize int, fn func(ctx context.Context, startIdx int, endIdx int) error) error

ChunkIterParallel is similar to ChunkIter but it uses an errgroup to run the chunks in parallel. To avoid costly execution from happening after the error context is cancelled, it is recommended to include a context short-circuit inside of the loop processing the subslice: var xs []string 

util.ChunkIterParallel(ctx, len(xs), 10, func(ctx context.Context, start, stop int) error {
  for _, tr := range xs[start:stop] {
    if err := ctx.Err(); err != nil {
      return err
    }
    // Do work here.
  }
}

func ChunkIterParallelPool 

func ChunkIterParallelPool(ctx context.Context, length, chunkSize, poolSize int, fn func(ctx context.Context, startIdx int, endIdx int) error) error

ChunkIterParallelPool is similar to ChunkIterParallel but it uses a poolSize group of workers to run the chunks in parallel.

func Close 

func Close(c io.Closer)

Close wraps an io.Closer and logs an error if one is returned. When manipulating the file, prefer util.WithReadFile over util.Close, as it handles closing automatically.

func ContainsAny 

func ContainsAny(s string, a []string) bool

ContainsAny returns true if |s| contains any element of |a|.

func CopyFile 

func CopyFile(src, dst string) (rvErr error)

CopyFile copies the given src file to dst.

func CopyString 

func CopyString(s string) string

CopyString returns a copy of the given string. This may seem unnecessary, but is very important at preventing leaks of strings. For example, subslicing a string can prevent the larger string from being cleaned up.

func CopyStringMap 

func CopyStringMap(m map[string]string) map[string]string

CopyStringMap returns a copy of the provided map[string]string such that reflect.DeepEqual returns true for the given map and the returned map. In particular, preserves nil input.

func CopyStringSlice 

func CopyStringSlice(s []string) []string

CopyStringSlice copies the given []string such that reflect.DeepEqual returns true for the given slice and the returned slice. In particular, preserves nil slice input.

func FirstNonEmpty 

func FirstNonEmpty(args ...string) string

FirstNonEmpty returns the first of its args that is not "". It is useful when a certain value would be preferred if present but others are available as fallbacks. If all of its args are "", returns "".

func FromDos 

func FromDos(s string) string

FromDos performs like the "fromdos" tool on Linux: it converts line endings in the given string from Dos to Unix format.

func In 

func In(s string, a []string) bool

In returns true if |s| is *in* |a| slice.

func Index 

func Index(s string, a []string) int

Index returns the index of |s| *in* |a| slice, and -1 if not found.

func InsertStringSorted 

func InsertStringSorted(strs []string, s string) []string

InsertStringSorted inserts the given string into the sorted slice of strings if it does not already exist. Maintains sorted order.

func IsDirEmpty 

func IsDirEmpty(dir string) (bool, error)

IsDirEmpty checks to see if the specified directory has any contents.

func IsLocal 

func IsLocal() bool

IsLocal attempts to determine whether or not we're running on a developer machine vs in Swarming or Kubernetes.

func IsNil 

func IsNil(i interface{}) bool

IsNil returns true if i is nil or is an interface containing a nil or invalid value.

func IterTimeChunks 

func IterTimeChunks(start, end time.Time, chunkSize time.Duration, fn func(time.Time, time.Time) error) error

IterTimeChunks calls the given function for each time chunk of the given duration within the given time range.

func LogErr 

func LogErr(err error)

LogErr logs err if it's not nil. This is intended to be used for calls where generally a returned error can be ignored.

func MD5SSlice 

func MD5SSlice(val []string) (string, error)

MD5SSlice returns the MD5 hash of the provided []string.

func MD5Sum 

func MD5Sum(val interface{}) (string, error)

MD5Sum returns the MD5 hash of the given value. It supports anything that can be encoded via bencode (https://en.wikipedia.org/wiki/Bencode).

func MaxInt 

func MaxInt(intList ...int) int

MaxInt returns the largest integer of the arguments provided.

func MaxInt32 

func MaxInt32(a, b int32) int32

MaxInt32 returns largest integer of a and b.

func MaxInt64 

func MaxInt64(a, b int64) int64

MaxInt64 returns largest integer of a and b.

func MaybeReadGobFile 

func MaybeReadGobFile(file string, data interface{}) error

MaybeReadGobFile reads data from the given file into the given data structure. If the file does not exist, no error is returned and no data is written.

func MinInt 

func MinInt(a, b int) int

MinInt returns the smaller integer of a and b.

func MinInt32 

func MinInt32(a, b int32) int32

MinInt32 returns the smaller integer of a and b.

func MinInt64 

func MinInt64(a, b int64) int64

MinInt64 returns the smaller integer of a and b.

func ParseIntSet 

func ParseIntSet(expr string) ([]int, error)

ParseIntSet parses a string expression like "5", "3-8", or "3,4,9" into a slice of integers: [5], [3, 4, 5, 6, 7, 8], [3, 4, 9].

func PowerSet 

func PowerSet(n int) [][]int

PowerSet returns a slice of slices representing the power set of the indices of a slice.

func ReadGobFile 

func ReadGobFile(file string, data interface{}) error

ReadGobFile reads data from the given file into the given data structure.

func Remove 

func Remove(name string)

Remove removes the specified file and logs an error if one is returned.

func RemoveAll 

func RemoveAll(path string)

RemoveAll removes the specified path and logs an error if one is returned.

func Repeat 

func Repeat(interval time.Duration, stopCh <-chan bool, fn func())

Repeat calls the provided function 'fn' immediately and then in intervals defined by 'interval'. If anything is sent on the provided stop channel, the iteration stops.

func RepeatCtx 

func RepeatCtx(ctx context.Context, interval time.Duration, fn func(ctx context.Context))

RepeatCtx calls the provided function 'fn' immediately and then in intervals defined by 'interval'. If the given context is canceled, the iteration stops.

func RepeatJoin 

func RepeatJoin(str, sep string, n int) string

RepeatJoin repeats a given string N times with the given separator between each instance.

func Reverse 

func Reverse(s []string) []string

Reverse returns the given slice of strings in reverse order.

func RoundUpToPowerOf2 

func RoundUpToPowerOf2(i int32) int32

RoundUpToPowerOf2 rounds the given int up to the nearest power of 2.

func SSliceDedup 

func SSliceDedup(slice []string) []string

SSliceDedup deduplicates a slice of strings, preserving their order.

func SSliceEqual 

func SSliceEqual(a, b []string) bool

SSliceEqual returns true if the given string slices are equal

func SplitLines 

func SplitLines(s string) []string

SplitLines returns a slice of the lines of s, split on newline characters. If the input string ends in a single newline, we strip it rather than returning a blank extra line.

Note that this currently works only with UNIX-style line breaks, not DOS \r\n ones, though support for those may be added later.

func TimeIsZero 

func TimeIsZero(t time.Time) bool

TimeIsZero returns true if the time.Time is a zero-value or corresponds to a zero Unix timestamp.

func TimeStamp 

func TimeStamp(targetUnit time.Duration) int64

TimeStamp returns the current time in the units defined by the given target unit. e.g. TimeStamp(time.Millisecond) will return the time in Milliseconds. The result is always rounded down to the lowest integer from the representation in nano seconds.

func TimeStampMs 

func TimeStampMs() int64

TimeStampMs returns the current time in milliseconds since the epoch.

func ToDos 

func ToDos(s string) string

ToDos performs like the "todos" tool on Linux: it converts line endings in the given string from Unix to Dos format.

func Truncate 

func Truncate(s string, length int) string

Truncate the given string to the given length. If the string was shortened, change the last three characters to ellipses, unless the specified length is 3 or less.

func TruncateNoEllipses 

func TruncateNoEllipses(s string, length int) string

TruncateNoEllipses truncates the given string to the given length, without the use of ellipses.

func ValidateCommit 

func ValidateCommit(hash string) bool

ValidateCommit returns true iff the given commit hash looks valid. Does not perform any check as to whether the commit means anything in a particular repository.

func WithGzipWriter 

func WithGzipWriter(w io.Writer, fn func(w io.Writer) error) (err error)

WithGzipWriter is a helper for wrapping an io.Writer with a gzip.Writer.

func WithReadFile 

func WithReadFile(file string, fn func(f io.Reader) error) (err error)

WithReadFile opens the given file for reading and runs the given function.

func WithWriteFile 

func WithWriteFile(file string, writeFn func(io.Writer) error) error

WithWriteFile provides an interface for writing to a backing file using a temporary intermediate file for more atomicity in case a long-running write gets interrupted.

func WordWrap 

func WordWrap(s string, lineLength int) string

WordWrap inserts newlines into the string so that no lone is longer than the given lineLength. Only breaks on word boundaries. Strips whitespace from the ends of lines but preserves whitespace at the beginnings of lines.

func WriteGobFile 

func WriteGobFile(file string, data interface{}) error

WriteGobFile writes the given data to the given file, using gob encoding.

Types

type CleanupFunc 

type CleanupFunc func()

CleanupFunc is a function return value that can be deferred by the caller to clean up any resources created/acquired by the function.

type Codec 

type Codec interface {
	// Encode serializes the given value to a byte array (inverse of Decode).
	Encode(interface{}) ([]byte, error)

	// Decode deserializes the byte array to an instance of the type that
	// was passed to Encode in a prior call.
	Decode([]byte) (interface{}, error)
}

Codec serializes/deserializes an instance of a type to/from byte arrays. Encode and Decode have to be the inverse of each other.

type GobDecoder 

type GobDecoder struct {
	// contains filtered or unexported fields
}

GobDecoder decodes bytes into structs via GOB decoding. Not safe for concurrent use.

Here's a template for writing a type-specific decoder:

FooDecoder decodes bytes into Foos via GOB decoding. Not safe for concurrent use. 

type FooDecoder struct {
	*util.GobDecoder
}

// NewFooDecoder returns a FooDecoder instance. 

func NewFooDecoder() *FooDecoder {
	return &FooDecoder{
		GobDecoder: util.NewGobDecoder(func() interface{} {
			return &Foo{}
		}, func(ch <-chan interface{}) interface{} {
			items := []*Foo{}
			for item := range ch {
				items = append(items, item.(*Foo))
			}
			return items
		}),
	}
}

// Result returns all decoded Foos provided to Process (in arbitrary order), or // any error encountered. 

func (d *FooDecoder) Result() ([]*Foo, error) {
	res, err := d.GobDecoder.Result()
	if err != nil {
		return nil, err
	}
	return res.([]*Foo), nil
}

func NewGobDecoder 

func NewGobDecoder(newItem func() interface{}, collect func(<-chan interface{}) interface{}) *GobDecoder

NewGobDecoder returns a GobDecoder instance. The first argument is a goroutine-safe function which returns a zero-valued instance of the type being decoded, eg. 

func() interface{} {
	return &MyType{}
}

The second argument is a function which collects decoded instances of that type from a channel and returns a slice, eg. 

func(ch <-chan interface{}) interface{} {
	items := []*MyType{}
	for item := range ch {
		items = append(items, item.(*MyType))
	}
	return items
}

func (*GobDecoder) Process 

func (d *GobDecoder) Process(b []byte) bool

Process decodes the byte slice and includes it in Result() (in arbitrary order). Returns false if Result is certain to return an error. Caller must ensure b does not change until after Result() returns.

func (*GobDecoder) Result 

func (d *GobDecoder) Result() (interface{}, error)

Result returns all decoded items provided to Process (in arbitrary order), or any error encountered.

type GobEncoder 

type GobEncoder struct {
	// contains filtered or unexported fields
}

GobEncoder encodes structs into bytes via GOB encoding. Not safe for concurrent use.

Here's a template for writing a type-specific encoder:

// FooEncoder encodes Foos into bytes via GOB encoding. Not safe for // concurrent use. 

type FooEncoder {
	util.GobEncoder
}

// Next returns one of the Foox provided to Process (in arbitrary order) and // its serialized bytes. If any items remain, returns the item, the // serialized bytes, nil. If all items have been returned, returns nil, nil, // nil. If an error is encountered, returns nil, nil, error. 

func (e *FooEncoder) Next() (*Foo, []byte, error) {
	item, serialized, err := e.GobEncoder.Next()
	if err != nil {
		return nil, nil, err
	} else if item == nil {
		return nil, nil, nil
	}
	return item.(*Foo), serialized, nil
}

func (*GobEncoder) Next 

func (e *GobEncoder) Next() (interface{}, []byte, error)

Next returns one of the items provided to Process (in arbitrary order) and its serialized bytes. If any items remain, returns the item, the serialized bytes, nil. If all items have been returned, returns nil, nil, nil. If an error is encountered, returns nil, nil, error.

func (*GobEncoder) Process 

func (e *GobEncoder) Process(item interface{}) bool

Process encodes the item into a byte slice that will be returned from Next() (in arbitrary order). Returns false if Next is certain to return an error. Caller must ensure item does not change until after the first call to Next(). May not be called after calling Next().

type Int64Slice 

type Int64Slice []int64

func (Int64Slice) Len 

func (p Int64Slice) Len() int

func (Int64Slice) Less 

func (p Int64Slice) Less(i, j int) bool

func (Int64Slice) Swap 

func (p Int64Slice) Swap(i, j int)

type JSONCodec 

type JSONCodec struct {
	// contains filtered or unexported fields
}

JSONCodec implements the Codec interface by serializing and deserializing instances of the underlying type of 'instance'. Generally it's assumed that 'instance' is a struct, a pointer to a struct, a slice or a map.

func NewJSONCodec 

func NewJSONCodec(instance interface{}) *JSONCodec

NewJSONCodec returns a new JSONCodec instance.

func (*JSONCodec) Decode 

func (j *JSONCodec) Decode(byteData []byte) (interface{}, error)

See Codec interface.

func (*JSONCodec) Encode 

func (j *JSONCodec) Encode(data interface{}) ([]byte, error)

See Codec interface.

type MultiWriter 

type MultiWriter []io.Writer

MultiWriter is like io.MultiWriter but attempts to write to all of the given io.Writers, even if writing to one fails.

func (MultiWriter) Write 

func (mw MultiWriter) Write(b []byte) (int, error)

See documentation for io.Writer. Uses a multierror.Error to summarize any and all errors returned by each of the io.Writers.

type NamedErrGroup 

type NamedErrGroup struct {
	// contains filtered or unexported fields
}

NamedErrGroup is like errgroup.Group, except each function in the group gets a name. It waits for all goroutines to finish and reports all errors by name.

func NewNamedErrGroup 

func NewNamedErrGroup() *NamedErrGroup

NewNamedErrGroup returns a NamedErrGroup instance.

func (*NamedErrGroup) Go 

func (g *NamedErrGroup) Go(name string, fn func() error)

Go runs the given function in a goroutine.

func (*NamedErrGroup) Wait 

func (g *NamedErrGroup) Wait() error

Wait waits for all of the goroutines to finish and reports any errors.

type StringSet 

type StringSet map[string]bool

StringSet is a set of strings, represented by the keys of a map.

func NewStringSet 

func NewStringSet(lists ...[]string) StringSet

NewStringSet returns the given list(s) of strings as a StringSet.

func (StringSet) AddLists 

func (s StringSet) AddLists(lists ...[]string) StringSet

AddLists adds lists of strings to the StringSet and returns the receiving StringSet.

func (StringSet) Complement 

func (s StringSet) Complement(other StringSet) StringSet

Complement returns a new StringSet containing all the original strings that are not in StringSet other

func (StringSet) Copy 

func (s StringSet) Copy() StringSet

Copy returns a copy of the StringSet such that reflect.DeepEqual returns true for the original and copy. In particular, preserves nil input.

func (StringSet) Equals 

func (s StringSet) Equals(other StringSet) bool

func (StringSet) Intersect 

func (s StringSet) Intersect(other StringSet) StringSet

Intersect returns a new StringSet containing all the original strings that are also in StringSet other

func (StringSet) Keys 

func (s StringSet) Keys() []string

Keys returns the keys of a StringSet

func (StringSet) String 

func (s StringSet) String() string

String returns a comma seperated list of the values of the set.

func (StringSet) Union 

func (s StringSet) Union(other StringSet) StringSet

Union returns a new StringSet containing all the original strings and all the strings in StringSet other

type Validator 

type Validator interface {
	Validate() error
}

Validator is an interface which has a Validate() method.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.