parl

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 17, 2022 License: ISC Imports: 15 Imported by: 14

README

Parl


A Go library for parallel programming of command-line utilities and system services

features: moderator, sqlite interface, package-selectable logging, debouncer, self-signed certificate authority, watchers, thread management, file system scan and operations, …


© 2022–present Harald Rudell (https://haraldrudell.github.io/haraldrudell/)
ISC License

“Great things are simple”

How to use

import "github.com/haraldrudell/parl"

Documentation

on github

On 3/16/2022 Parl was open-sourced under an ISC License
Parl is about 9,000 lines of Go code with first line written on November 21, 2018

© 2022–present Harald Rudell harald.rudell@gmail.com (https://haraldrudell.github.io/haraldrudell/)

Documentation

Overview

Package parl handles inter-thread communication and controls parallelism

Logging of extected output is via Out(string, ...interface{}). Parl logging uses comma separator for numbers and is thread safe

Log(string, ...interface{}) always outputs to stderr. Console is the same intended to be used for command-line interactivity. SetDebug(true) appends code location

Info is active by default and outputs to stderr. SetSilent(true) removes this output. SetDebug(true) appends code location IsSilent deteremines if Info printing applies

Debug only prints if SetDebug(true) or the code location matches SetInfoRegexp(). The string matched for regular expression looks like: “github.com/haraldrudell/parl.FuncName” IsThisDebug determines if debug is active for the executing function

parl.D is intended for temporary printouts to be removed before check-in

parl provides generic recovery for goroutines and functions: capturing panics, annotating and storing errors and invoking an error handling function on errors: 

  func f() (err error) {
		defer parl.Recover(parl.Annotation(), &err, onError func(e error) { … })
	…

Default error string: “Recover from panic in somePackage.someFunction: 'File not found'. For multiple errors, Recover uses error116 error lists, while Recover2 instead invokes onError multiple times

Parl is about 9,000 lines of Go code with first line written on November 21, 2018

On 3/16/2022 Parl was open-sourced under an ISC License

© 2020–present Harald Rudell <harald.rudell@gmail.com> (https://haraldrudell.github.io/haraldrudell/)

Index

Constants

View Source 
const (
	Rfc3339s   = "2006-01-02 15:04:05-07:00"
	Rfc3339ms  = "2006-01-02 15:04:05.999-07:00"
	Rfc3339us  = "2006-01-02 15:04:05.999999-07:00"
	Rfc3339ns  = "2006-01-02 15:04:05.999999999-07:00"
	Rfc3339sz  = "2006-01-02T15:04:05Z"
	Rfc3339msz = "2006-01-02T15:04:05.999Z"
	Rfc3339usz = "2006-01-02T15:04:05.999999Z"
	Rfc3339nsz = "2006-01-02T15:04:05.999999999Z"
)
View Source 
const (
	SerialDoReady         = 0 + iota
	SerialDoLaunch        // from idle, now time
	SerialDoPending       // queued up invocation, request time
	SerialDoPendingLaunch // launch of pending invocation, request time
	SerialDoIdle          // busy since
)
View Source 
const (
	// LogObject context key for log object in valuecontext.go
	LogObject = "parl.Log"
)
View Source 
const (
	// TimePointer is context key for a pointer to a shared timestamp
	TimePointer = "parl.Time"
)

Variables

View Source 
var ErrModeratorShutdown = errors.New("Moderator shut down")
View Source 
var Errorf = error116.Errorf
View Source 
var New = error116.New
View Source 
var NewCodeLocation = error116.NewCodeLocation
View Source 
var PackFunc = error116.PackFunc

Functions

func AddToPanic 

func AddToPanic(panicValue interface{}, additionalErr error) (err error)

func Annotation 

func Annotation() (annotation string)

func Console 

func Console(format string, a ...interface{})

Console always print intended for command-line interactivity if debug is enabled, code location is appended

func ContextTime 

func ContextTime(ctx context.Context) (t time.Time, found bool)

ContextTime obtains a shared timestamp

func D 

func D(format string, a ...interface{})

D prints to stderr with code location Thread safe. D is meant for temporary output intended to be removed before check-in

func Debug 

func Debug(format string, a ...interface{})

Debug outputs only if debug is configured or the code location package matches regexp

func DelFromContext 

func DelFromContext(ctx context.Context, key string) (ok bool)

DelFromContext removes a value from context. Thread-safe

func EnsureError 

func EnsureError(panicValue interface{}) (err error)

func HandlePanic 

func HandlePanic(fn func()) (err error)

HandlePanic executes a function wrapped in panic recovery. If the function panics, HandlePanic returns an error value

func Info 

func Info(format string, a ...interface{})

Info prints unless silence has been configured with SetSilence(true) IsSilent deteremines the state of silence if debug is enabled, code location is appended

func IsSilent 

func IsSilent() (isSilent bool)

IsSilent if true it means that Info does not print

func IsThisDebug 

func IsThisDebug() bool

IsThisDebug returns whether debug logging is configured for the executing function

func Log 

func Log(format string, a ...interface{})

Log invocations always print if debug is enabled, code location is appended

func NewDebouncer 

func NewDebouncer(d time.Duration, receiver ReceiverFunc, sender SenderFunc, ctx context.Context) (err error)

Debouncer debounces event streams of Value

func Out 

func Out(format string, a ...interface{})

Out prints extected output to stdout

func Recover 

func Recover(annotation string, errp *error, onError func(error))

Recover recovers from a panic invoking a function no more than once. If there is *errp does not hold an error and there is no panic, onError is not invoked. Otherwise, onError is invoked exactly once. *errp is updated with a possible panic.

func Recover2 

func Recover2(annotation string, errp *error, onError func(error))

Recover2 recovers from a panic and may invoke onError multiple times. onError is invoked if there is an error at *errp and on a possible panic. *errp is updated with a possible panic.

func SetDebug 

func SetDebug(debug bool)

if SetDebug is true, all Debug prints everywhere produce output. More selective debug printing can be achieved using SetInfoRegexp that matches package names.

func SetRegexp 

func SetRegexp(regExp string) (err error)

func SetSilent 

func SetSilent(silent bool)

SetSilent

func StoreInContext 

func StoreInContext(ctx context.Context, key string, value interface{}) (ok bool)

StoreInContext stores a value in context. Thread-safe

Types

type AtomicBool 

type AtomicBool struct {
	// contains filtered or unexported fields
}

func (*AtomicBool) Clear 

func (ab *AtomicBool) Clear() (wasSet bool)

func (*AtomicBool) IsTrue 

func (ab *AtomicBool) IsTrue() (isTrue bool)

func (*AtomicBool) Set 

func (ab *AtomicBool) Set() (wasNotSet bool)

type Context 

type Context struct {
	context.Context
	Map sync.Map
}

Context is a context.Context with a Value() implementation

func NewContext 

func NewContext(ctx ...context.Context) (c *Context)

NewContext provides a context.Context with a Value() implementation

func (*Context) Delete 

func (c *Context) Delete(key string)

StoreInContext stores a value in context. Thread-safe

func (*Context) Store 

func (c *Context) Store(key string, value interface{})

StoreInContext stores a value in context. Thread-safe

func (*Context) Value 

func (c *Context) Value(key interface{}) (result interface{})

Value retrieves data from context. Thread-safe

func (*Context) Value2 

func (c *Context) Value2(key interface{}) (result interface{}, ok bool)

Value2 retrieves data and was–present indicator from context. Thread-safe

type FSLocation 

type FSLocation interface {
	Directory() (directory string)
}

type FanOut 

type FanOut struct {
	ErrCh   chan error
	Results chan interface{}
	// contains filtered or unexported fields
}

func NewFanOut 

func NewFanOut() (fo *FanOut)

func (*FanOut) Do 

func (cr *FanOut) Do(name string, proc FanProc)

Do executes a procedure in a goroutine that has no result other than a possible non-nil error

func (*FanOut) Run 

func (cr *FanOut) Run(name string, thunk FanThunk)

Run executes a thunk in a goroutine with a possible non-nil result and a possible non-nil error

func (*FanOut) Wait 

func (cr *FanOut) Wait()

Wait waits for all Do and Run invocations to complete, then shuts down

type FanProc 

type FanProc func() (err error)

type FanThunk 

type FanThunk func() (result interface{}, err error)

type LogInstance 

type LogInstance struct {
	// contains filtered or unexported fields
}

LogInstance provide logging delegating to log.Output

func GetLogAndContext 

func GetLogAndContext(debug bool, verbose bool, silent bool) (ctx context.Context, log *LogInstance)

GetLogAndContext gets a context and a log

func GetLogFromContext 

func GetLogFromContext(ctx context.Context) (log *LogInstance)

GetLogFromContext obtains a log possibly from context

func NewLog 

func NewLog(writers ...io.Writer) (lg *LogInstance)

NewLog gets a logger for Fatal and Warning for specific output

func (*LogInstance) D 

func (lg *LogInstance) D(format string, a ...interface{})

D prints to stderr with code location Thread safe. D is meant for temporary output intended to be removed before check-in

func (*LogInstance) Debug 

func (lg *LogInstance) Debug(format string, a ...interface{})

Debug outputs only if debug is configured or the code location package matches regexp

func (*LogInstance) Info 

func (lg *LogInstance) Info(format string, a ...interface{})

Info prints unless silence has been configured with SetSilence(true) IsSilent deteremines the state of silence if debug is enabled, code location is appended

func (*LogInstance) IsSilent 

func (lg *LogInstance) IsSilent() (isSilent bool)

IsSilent if true it means that Info does not print

func (*LogInstance) IsThisDebug 

func (lg *LogInstance) IsThisDebug() bool

IsThisDebug returns whether debug logging is configured

func (*LogInstance) Log 

func (lg *LogInstance) Log(format string, a ...interface{})

Log invocations always print if debug is enabled, code location is appended

func (*LogInstance) SetDebug 

func (lg *LogInstance) SetDebug(debug bool)

SetDebug prints everything with code location: IsInfo

func (*LogInstance) SetRegexp 

func (lg *LogInstance) SetRegexp(regExp string) (err error)

func (*LogInstance) SetSilent 

func (lg *LogInstance) SetSilent(silent bool)

SetSilent only prints Log

type Moderator 

type Moderator struct {
	// contains filtered or unexported fields
}

Moderator invokes functions at a limited level of parallelism

func NewModerator 

func NewModerator(parallelism uint64) (mo *Moderator)

NewModerator creates a new Moderator used to limit parallelism

func (*Moderator) Do 

func (mo *Moderator) Do(fn func() error) (err error)

Do calls fn limited by the moderator’s parallelism. If the moderator is shut down, ErrModeratorShutdown is returned

func (*Moderator) Shutdown 

func (mo *Moderator) Shutdown()

func (*Moderator) Status 

func (mo *Moderator) Status() (parallelism uint64, available uint64, waiting uint64, isShutdown bool)

func (*Moderator) String 

func (mo *Moderator) String() (s string)

type Password 

type Password interface {
	HasPassword() (hasPassword bool)
	Password() (password string)
}

type ReceiverFunc 

type ReceiverFunc func(c <-chan time.Time, done <-chan struct{}) (which TriggeringChan, value Value)

ReceiverFunc takes two channels, listens to them and a typed channel, returns what channel triggered and a possible untyped value

type SenderFunc 

type SenderFunc func([]Value)

SenderFunc takes an untyped value, type asserts and sends on a typed channel

type SerialDo 

type SerialDo struct {
	ErrCh chan error
	ID    string
	Wg    sync.WaitGroup
	// contains filtered or unexported fields
}

serialdo invokes method in sequence

func NewSerialDo 

func NewSerialDo(thunk func(), eventReceiver SerialDoFunc, ctx context.Context) (sdo *SerialDo)

NewSerialDo SerialDo. errors on sdo.ErrCh

func (*SerialDo) Do 

func (sdo *SerialDo) Do(now time.Time) (nowPending bool)

Invoke thunk serially, maximum queue one invocation, drop additional invocation requests prior to idle. non-blocking Thread-safe

func (*SerialDo) Shutdown 

func (sdo *SerialDo) Shutdown()

type SerialDoEvent 

type SerialDoEvent uint8

type SerialDoFunc 

type SerialDoFunc func(SerialDoEvent, *SerialDo, *time.Time)

type Timer 

type Timer struct {
	Label string
	// contains filtered or unexported fields
}

Timer is a simple request timer

func NewTimer 

func NewTimer(label string) (t *Timer)

NewTimer gets a simple timer with duration or string output

func (*Timer) End 

func (t *Timer) End() (d time.Duration)

End gets duration

func (*Timer) Endms 

func (t *Timer) Endms() (ms string)

Endms gets tring with duration in ms

type TriggeringChan 

type TriggeringChan uint8
const (
	TimerCh TriggeringChan = iota
	DoneCh
	ValueCh
)

type Value 

type Value interface{}

Value is an event value that is being debounced

Source Files

View all Source files

Directories

Path Synopsis
cmd
parl
parl.go demonstrate usage of the parl package, a go library for command-line utilities and concurrency
 parl.go demonstrate usage of the parl package, a go library for command-line utilities and concurrency
Package error116 adds stack traces and maps of values to error values.
 Package error116 adds stack traces and maps of values to error values.
Package ev provides standardized goroutine management events contain thread completions, failures and any type of data items.
 Package ev provides standardized goroutine management events contain thread completions, failures and any type of data items.
Package evx contains declarations not essential to event handling
 Package evx contains declarations not essential to event handling
Package mains contains functions for executing a command-line utility
 Package mains contains functions for executing a command-line utility
omaps module
Package parlca provides a self-signed certificate authority
 Package parlca provides a self-signed certificate authority
Package parlfs provides file-system related functions Package parlfs provides file-system related functions © 2020–present Harald Rudell <harald.rudell@gmail.com> (https://haraldrudell.github.io/haraldrudell/)
 Package parlfs provides file-system related functions Package parlfs provides file-system related functions © 2020–present Harald Rudell <harald.rudell@gmail.com> (https://haraldrudell.github.io/haraldrudell/)
Package parlnet provides IP-related functions with few dependencies beyond the net package
 Package parlnet provides IP-related functions with few dependencies beyond the net package
Package parlos provides simplified functions related to the os package
 Package parlos provides simplified functions related to the os package
Package parltime provides time utility functions
 Package parltime provides time utility functions
pfs module
process module
Package progress provides printable progress reporting for multi-threaded operations
 Package progress provides printable progress reporting for multi-threaded operations
psql module
pterm module
Package sqldb interfaces database/sql
 Package sqldb interfaces database/sql
Package sqlite wraps modernc.org/sqlite
 Package sqlite wraps modernc.org/sqlite
sqliter module
watchfs module
yaml module
yamler module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.