Documentation
¶
Index ¶
- Constants
- func AbsInt(v int) int
- func AddParams(a map[string]string, b ...map[string]string) map[string]string
- func AskForConfirmation(format string, args ...interface{}) (bool, error)
- func ChunkIter(length, chunkSize int, fn func(startIdx int, endIdx int) error) error
- func ChunkIterParallel(ctx context.Context, length, chunkSize int, ...) error
- func ChunkIterParallelPool(ctx context.Context, length, chunkSize, poolSize int, ...) error
- func Close(c io.Closer)
- func ContainsAny(s string, a []string) bool
- func CopyFile(src, dst string) (rvErr error)
- func CopyString(s string) string
- func CopyStringMap(m map[string]string) map[string]string
- func CopyStringSlice(s []string) []string
- func FirstNonEmpty(args ...string) string
- func FromDos(s string) string
- func In(s string, a []string) bool
- func Index(s string, a []string) int
- func InsertStringSorted(strs []string, s string) []string
- func IsDirEmpty(dir string) (bool, error)
- func IsLocal() bool
- func IsNil(i interface{}) bool
- func IterTimeChunks(start, end time.Time, chunkSize time.Duration, ...) error
- func LogErr(err error)
- func MD5SSlice(val []string) (string, error)
- func MD5Sum(val interface{}) (string, error)
- func MaxInt(intList ...int) int
- func MaxInt32(a, b int32) int32
- func MaxInt64(a, b int64) int64
- func MaybeReadGobFile(file string, data interface{}) error
- func MinInt(a, b int) int
- func MinInt32(a, b int32) int32
- func MinInt64(a, b int64) int64
- func ParseIntSet(expr string) ([]int, error)
- func PowerSet(n int) [][]int
- func ReadGobFile(file string, data interface{}) error
- func Remove(name string)
- func RemoveAll(path string)
- func Repeat(interval time.Duration, stopCh <-chan bool, fn func())
- func RepeatCtx(ctx context.Context, interval time.Duration, fn func(ctx context.Context))
- func RepeatJoin(str, sep string, n int) string
- func Reverse(s []string) []string
- func RoundUpToPowerOf2(i int32) int32
- func SSliceDedup(slice []string) []string
- func SSliceEqual(a, b []string) bool
- func SplitLines(s string) []string
- func TimeIsZero(t time.Time) bool
- func TimeStamp(targetUnit time.Duration) int64
- func TimeStampMs() int64
- func ToDos(s string) string
- func Truncate(s string, length int) string
- func TruncateNoEllipses(s string, length int) string
- func ValidateCommit(hash string) bool
- func WithGzipWriter(w io.Writer, fn func(w io.Writer) error) (err error)
- func WithReadFile(file string, fn func(f io.Reader) error) (err error)
- func WithWriteFile(file string, writeFn func(io.Writer) error) error
- func WordWrap(s string, lineLength int) string
- func WriteGobFile(file string, data interface{}) error
- type CleanupFunc
- type Codec
- type GobDecoder
- type GobEncoder
- type Int64Slice
- type JSONCodec
- type MultiWriter
- type NamedErrGroup
- type StringSet
- func (s StringSet) AddLists(lists ...[]string) StringSet
- func (s StringSet) Complement(other StringSet) StringSet
- func (s StringSet) Copy() StringSet
- func (s StringSet) Equals(other StringSet) bool
- func (s StringSet) Intersect(other StringSet) StringSet
- func (s StringSet) Keys() []string
- func (s StringSet) String() string
- func (s StringSet) Union(other StringSet) StringSet
- type Validator
Constants ¶
const ( // time.RFC3339Nano only uses as many sub-second digits are required to // represent the time, which makes it unsuitable for sorting. This // format ensures that all 9 nanosecond digits are used, padding with // zeroes if necessary. RFC3339NanoZeroPad = "2006-01-02T15:04:05.000000000Z07:00" // SAFE_TIMESTAMP_FORMAT is time format which is similar to // RFC3339NanoZeroPad, but with most of the punctuation omitted. This // timestamp can only be used to format and parse times in UTC. SAFE_TIMESTAMP_FORMAT = "20060102T150405.000000000Z" )
Variables ¶
This section is empty.
Functions ¶
func AddParams ¶
AddParams adds the second instance of map[string]string to the first and returns the first map.
func AskForConfirmation ¶
AskForConfirmation waits for the user to type "y" or "n".
func ChunkIterParallel ¶
func ChunkIterParallel(ctx context.Context, length, chunkSize int, fn func(ctx context.Context, startIdx int, endIdx int) error) error
ChunkIterParallel is similar to ChunkIter but it uses an errgroup to run the chunks in parallel. To avoid costly execution from happening after the error context is cancelled, it is recommended to include a context short-circuit inside of the loop processing the subslice: var xs []string
util.ChunkIterParallel(ctx, len(xs), 10, func(ctx context.Context, start, stop int) error {
for _, tr := range xs[start:stop] {
if err := ctx.Err(); err != nil {
return err
}
// Do work here.
}
}
func ChunkIterParallelPool ¶
func ChunkIterParallelPool(ctx context.Context, length, chunkSize, poolSize int, fn func(ctx context.Context, startIdx int, endIdx int) error) error
ChunkIterParallelPool is similar to ChunkIterParallel but it uses a poolSize group of workers to run the chunks in parallel.
func Close ¶
Close wraps an io.Closer and logs an error if one is returned. When manipulating the file, prefer util.WithReadFile over util.Close, as it handles closing automatically.
func ContainsAny ¶
ContainsAny returns true if |s| contains any element of |a|.
func CopyString ¶
CopyString returns a copy of the given string. This may seem unnecessary, but is very important at preventing leaks of strings. For example, subslicing a string can prevent the larger string from being cleaned up.
func CopyStringMap ¶
CopyStringMap returns a copy of the provided map[string]string such that reflect.DeepEqual returns true for the given map and the returned map. In particular, preserves nil input.
func CopyStringSlice ¶
CopyStringSlice copies the given []string such that reflect.DeepEqual returns true for the given slice and the returned slice. In particular, preserves nil slice input.
func FirstNonEmpty ¶
FirstNonEmpty returns the first of its args that is not "". It is useful when a certain value would be preferred if present but others are available as fallbacks. If all of its args are "", returns "".
func FromDos ¶
FromDos performs like the "fromdos" tool on Linux: it converts line endings in the given string from Dos to Unix format.
func InsertStringSorted ¶
InsertStringSorted inserts the given string into the sorted slice of strings if it does not already exist. Maintains sorted order.
func IsDirEmpty ¶
IsDirEmpty checks to see if the specified directory has any contents.
func IsLocal ¶
func IsLocal() bool
IsLocal attempts to determine whether or not we're running on a developer machine vs in Swarming or Kubernetes.
func IsNil ¶
func IsNil(i interface{}) bool
IsNil returns true if i is nil or is an interface containing a nil or invalid value.
func IterTimeChunks ¶
func IterTimeChunks(start, end time.Time, chunkSize time.Duration, fn func(time.Time, time.Time) error) error
IterTimeChunks calls the given function for each time chunk of the given duration within the given time range.
func LogErr ¶
func LogErr(err error)
LogErr logs err if it's not nil. This is intended to be used for calls where generally a returned error can be ignored.
func MD5Sum ¶
MD5Sum returns the MD5 hash of the given value. It supports anything that can be encoded via bencode (https://en.wikipedia.org/wiki/Bencode).
func MaybeReadGobFile ¶
MaybeReadGobFile reads data from the given file into the given data structure. If the file does not exist, no error is returned and no data is written.
func ParseIntSet ¶
ParseIntSet parses a string expression like "5", "3-8", or "3,4,9" into a slice of integers: [5], [3, 4, 5, 6, 7, 8], [3, 4, 9].
func PowerSet ¶
PowerSet returns a slice of slices representing the power set of the indices of a slice.
func ReadGobFile ¶
ReadGobFile reads data from the given file into the given data structure.
func Remove ¶
func Remove(name string)
Remove removes the specified file and logs an error if one is returned.
func RemoveAll ¶
func RemoveAll(path string)
RemoveAll removes the specified path and logs an error if one is returned.
func Repeat ¶
Repeat calls the provided function 'fn' immediately and then in intervals defined by 'interval'. If anything is sent on the provided stop channel, the iteration stops.
func RepeatCtx ¶
RepeatCtx calls the provided function 'fn' immediately and then in intervals defined by 'interval'. If the given context is canceled, the iteration stops.
func RepeatJoin ¶
RepeatJoin repeats a given string N times with the given separator between each instance.
func RoundUpToPowerOf2 ¶
RoundUpToPowerOf2 rounds the given int up to the nearest power of 2.
func SSliceDedup ¶
SSliceDedup deduplicates a slice of strings, preserving their order.
func SSliceEqual ¶
SSliceEqual returns true if the given string slices are equal
func SplitLines ¶
SplitLines returns a slice of the lines of s, split on newline characters. If the input string ends in a single newline, we strip it rather than returning a blank extra line.
Note that this currently works only with UNIX-style line breaks, not DOS \r\n ones, though support for those may be added later.
func TimeIsZero ¶
TimeIsZero returns true if the time.Time is a zero-value or corresponds to a zero Unix timestamp.
func TimeStamp ¶
TimeStamp returns the current time in the units defined by the given target unit. e.g. TimeStamp(time.Millisecond) will return the time in Milliseconds. The result is always rounded down to the lowest integer from the representation in nano seconds.
func TimeStampMs ¶
func TimeStampMs() int64
TimeStampMs returns the current time in milliseconds since the epoch.
func ToDos ¶
ToDos performs like the "todos" tool on Linux: it converts line endings in the given string from Unix to Dos format.
func Truncate ¶
Truncate the given string to the given length. If the string was shortened, change the last three characters to ellipses, unless the specified length is 3 or less.
func TruncateNoEllipses ¶
TruncateNoEllipses truncates the given string to the given length, without the use of ellipses.
func ValidateCommit ¶
ValidateCommit returns true iff the given commit hash looks valid. Does not perform any check as to whether the commit means anything in a particular repository.
func WithGzipWriter ¶
WithGzipWriter is a helper for wrapping an io.Writer with a gzip.Writer.
func WithReadFile ¶
WithReadFile opens the given file for reading and runs the given function.
func WithWriteFile ¶
WithWriteFile provides an interface for writing to a backing file using a temporary intermediate file for more atomicity in case a long-running write gets interrupted.
func WordWrap ¶
WordWrap inserts newlines into the string so that no lone is longer than the given lineLength. Only breaks on word boundaries. Strips whitespace from the ends of lines but preserves whitespace at the beginnings of lines.
func WriteGobFile ¶
WriteGobFile writes the given data to the given file, using gob encoding.
Types ¶
type CleanupFunc ¶
type CleanupFunc func()
CleanupFunc is a function return value that can be deferred by the caller to clean up any resources created/acquired by the function.
type Codec ¶
type Codec interface {
// Encode serializes the given value to a byte array (inverse of Decode).
Encode(interface{}) ([]byte, error)
// Decode deserializes the byte array to an instance of the type that
// was passed to Encode in a prior call.
Decode([]byte) (interface{}, error)
}
Codec serializes/deserializes an instance of a type to/from byte arrays. Encode and Decode have to be the inverse of each other.
type GobDecoder ¶
type GobDecoder struct {
// contains filtered or unexported fields
}
GobDecoder decodes bytes into structs via GOB decoding. Not safe for concurrent use.
Here's a template for writing a type-specific decoder:
FooDecoder decodes bytes into Foos via GOB decoding. Not safe for concurrent use.
type FooDecoder struct {
*util.GobDecoder
}
// NewFooDecoder returns a FooDecoder instance.
func NewFooDecoder() *FooDecoder {
return &FooDecoder{
GobDecoder: util.NewGobDecoder(func() interface{} {
return &Foo{}
}, func(ch <-chan interface{}) interface{} {
items := []*Foo{}
for item := range ch {
items = append(items, item.(*Foo))
}
return items
}),
}
}
// Result returns all decoded Foos provided to Process (in arbitrary order), or // any error encountered.
func (d *FooDecoder) Result() ([]*Foo, error) {
res, err := d.GobDecoder.Result()
if err != nil {
return nil, err
}
return res.([]*Foo), nil
}
func NewGobDecoder ¶
func NewGobDecoder(newItem func() interface{}, collect func(<-chan interface{}) interface{}) *GobDecoder
NewGobDecoder returns a GobDecoder instance. The first argument is a goroutine-safe function which returns a zero-valued instance of the type being decoded, eg.
func() interface{} {
return &MyType{}
}
The second argument is a function which collects decoded instances of that type from a channel and returns a slice, eg.
func(ch <-chan interface{}) interface{} {
items := []*MyType{}
for item := range ch {
items = append(items, item.(*MyType))
}
return items
}
func (*GobDecoder) Process ¶
func (d *GobDecoder) Process(b []byte) bool
Process decodes the byte slice and includes it in Result() (in arbitrary order). Returns false if Result is certain to return an error. Caller must ensure b does not change until after Result() returns.
func (*GobDecoder) Result ¶
func (d *GobDecoder) Result() (interface{}, error)
Result returns all decoded items provided to Process (in arbitrary order), or any error encountered.
type GobEncoder ¶
type GobEncoder struct {
// contains filtered or unexported fields
}
GobEncoder encodes structs into bytes via GOB encoding. Not safe for concurrent use.
Here's a template for writing a type-specific encoder:
// FooEncoder encodes Foos into bytes via GOB encoding. Not safe for // concurrent use.
type FooEncoder {
util.GobEncoder
}
// Next returns one of the Foox provided to Process (in arbitrary order) and // its serialized bytes. If any items remain, returns the item, the // serialized bytes, nil. If all items have been returned, returns nil, nil, // nil. If an error is encountered, returns nil, nil, error.
func (e *FooEncoder) Next() (*Foo, []byte, error) {
item, serialized, err := e.GobEncoder.Next()
if err != nil {
return nil, nil, err
} else if item == nil {
return nil, nil, nil
}
return item.(*Foo), serialized, nil
}
func (*GobEncoder) Next ¶
func (e *GobEncoder) Next() (interface{}, []byte, error)
Next returns one of the items provided to Process (in arbitrary order) and its serialized bytes. If any items remain, returns the item, the serialized bytes, nil. If all items have been returned, returns nil, nil, nil. If an error is encountered, returns nil, nil, error.
func (*GobEncoder) Process ¶
func (e *GobEncoder) Process(item interface{}) bool
Process encodes the item into a byte slice that will be returned from Next() (in arbitrary order). Returns false if Next is certain to return an error. Caller must ensure item does not change until after the first call to Next(). May not be called after calling Next().
type Int64Slice ¶
type Int64Slice []int64
func (Int64Slice) Len ¶
func (p Int64Slice) Len() int
func (Int64Slice) Less ¶
func (p Int64Slice) Less(i, j int) bool
func (Int64Slice) Swap ¶
func (p Int64Slice) Swap(i, j int)
type JSONCodec ¶
type JSONCodec struct {
// contains filtered or unexported fields
}
JSONCodec implements the Codec interface by serializing and deserializing instances of the underlying type of 'instance'. Generally it's assumed that 'instance' is a struct, a pointer to a struct, a slice or a map.
func NewJSONCodec ¶
func NewJSONCodec(instance interface{}) *JSONCodec
NewJSONCodec returns a new JSONCodec instance.
type MultiWriter ¶
MultiWriter is like io.MultiWriter but attempts to write to all of the given io.Writers, even if writing to one fails.
type NamedErrGroup ¶
type NamedErrGroup struct {
// contains filtered or unexported fields
}
NamedErrGroup is like errgroup.Group, except each function in the group gets a name. It waits for all goroutines to finish and reports all errors by name.
func NewNamedErrGroup ¶
func NewNamedErrGroup() *NamedErrGroup
NewNamedErrGroup returns a NamedErrGroup instance.
func (*NamedErrGroup) Go ¶
func (g *NamedErrGroup) Go(name string, fn func() error)
Go runs the given function in a goroutine.
func (*NamedErrGroup) Wait ¶
func (g *NamedErrGroup) Wait() error
Wait waits for all of the goroutines to finish and reports any errors.
type StringSet ¶
StringSet is a set of strings, represented by the keys of a map.
func NewStringSet ¶
NewStringSet returns the given list(s) of strings as a StringSet.
func (StringSet) AddLists ¶
AddLists adds lists of strings to the StringSet and returns the receiving StringSet.
func (StringSet) Complement ¶
Complement returns a new StringSet containing all the original strings that are not in StringSet other
func (StringSet) Copy ¶
Copy returns a copy of the StringSet such that reflect.DeepEqual returns true for the original and copy. In particular, preserves nil input.
func (StringSet) Intersect ¶
Intersect returns a new StringSet containing all the original strings that are also in StringSet other
Source Files
Directories