utils

package module
v3.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 29, 2023 License: MIT Imports: 46 Imported by: 7

README

Go-Utils

Many useful golang tools

Version Support Go
1.x >= v1.16
2.x >= v1.18
3.x >= v1.19

Commitizen friendly Go Report Card GoDoc Build Status codecov

Use as command-line executable binary

Install
go install github.com/Laisky/go-utils/v3/cmd/gutils@latest

Add $HOME/.go/bin to your PATH.

Usage
# find and delete duplicate files/ similar images
gutils remove-dup examples/images --dry

# move files to hash-based hierach directories
gutils md5dir -i examples/md5dir/ --dry

# generate tls cert
gutils gentls --host 1.2.3.4

# encrypt by aes
gutils encrypt aes -i <file_path> -s <password>

# sign or verify by rsa
gutils rsa sign
gutils rsa verify

Use as SDK

Install
go get github.com/Laisky/go-utils/v3@latest
Usage
import (
    gutils "github.com/Laisky/go-utils/v3"
)
Modules

Contains some useful tools in different directories:

  • settings: move go github.com/Laisky/go-config
  • color.go: colorful code
  • compressor.go: compress and extract dir/files
  • email/: SMTP email sdk
  • encrypt/: some tools for encrypt and decrypt, support AES, RSA, ECDSA, MD5, SHA128, SHA256
    • configserver.go: load configs from file or config-server
  • fs.go: some tools to read, move, walk dir/files
  • http.go: some tools to send http request
  • jwt/: some tools to generate and parse JWT
  • log/: enhanched zap logger
  • math.go: some math tools to deal with int, round
  • net.go: some tools to deal with tcp/udp
  • random.go: generate random string, int
  • sort.go: easier to sort
  • sync.go: some locks depends on atomic
  • throttle.go: faster rate limiter
  • time.go: faster clock (if you do not enable vdso)
  • utils: some useful tools

Documentation

Overview

Package utils some useful tools fo Golang

Modules

Contains some useful tools in different directories:

  • `color.go`: colorful code
  • `compressor.go`: compress and extract dir/files
  • `configserver.go`: load configs from file or config-server
  • `email.go`: SMTP email sdk
  • `encrypt.go`: some tools for encrypt and decrypt, support AES, RSA, ECDSA, MD5, SHA128, SHA256
  • `fs.go`: some tools to read, move, walk dir/files
  • `http.go`: some tools to send http request
  • `jwt.go`: some tools to generate and parse JWT
  • `logger.go`: enhanched zap logger
  • `math.go`: some math tools to deal with int, round
  • `net.go`: some tools to deal with tcp/udp
  • `random.go`: generate random string, int
  • `settings.go`: read configs from file or config-server
  • `sort.go`: easier to sort
  • `sync.go`: some locks depends on atomic
  • `throttle.go`: faster rate limiter
  • `time.go`: faster clock (if you do not enable vdso)
  • `utils`: some useful tools

Index

Examples

Constants

View Source 
const (
	ANSIColorReset int = iota
	ANSIColorBold
	ANSIColorFaint
	ANSIColorItalic
	ANSIColorUnderline
	ANSIColorBlinkSlow
	ANSIColorBlinkRapid
	ANSIColorReverseVideo
	ANSIColorConcealed
	ANSIColorCrossedOut
)

Base attributes

View Source 
const (
	ANSIColorFgBlack int = iota + 30
	ANSIColorFgRed
	ANSIColorFgGreen
	ANSIColorFgYellow
	ANSIColorFgBlue
	ANSIColorFgMagenta
	ANSIColorFgCyan
	ANSIColorFgWhite
)

Foreground text colors

View Source 
const (
	ANSIColorFgHiBlack int = iota + 90
	ANSIColorFgHiRed
	ANSIColorFgHiGreen
	ANSIColorFgHiYellow
	ANSIColorFgHiBlue
	ANSIColorFgHiMagenta
	ANSIColorFgHiCyan
	ANSIColorFgHiWhite
)

Foreground Hi-Intensity text colors

View Source 
const (
	ANSIColorBgBlack int = iota + 40
	ANSIColorBgRed
	ANSIColorBgGreen
	ANSIColorBgYellow
	ANSIColorBgBlue
	ANSIColorBgMagenta
	ANSIColorBgCyan
	ANSIColorBgWhite
)

Background text colors

View Source 
const (
	ANSIColorBgHiBlack int = iota + 100
	ANSIColorBgHiRed
	ANSIColorBgHiGreen
	ANSIColorBgHiYellow
	ANSIColorBgHiBlue
	ANSIColorBgHiMagenta
	ANSIColorBgHiCyan
	ANSIColorBgHiWhite
)

Background Hi-Intensity text colors

View Source 
const (

	// HTTPHeaderHost HTTP header name
	HTTPHeaderHost = "Host"
	// HTTPHeaderReferer HTTP header name
	HTTPHeaderReferer = "Referer"
	// HTTPHeaderContentType HTTP header name
	HTTPHeaderContentType = "Content-Type"

	// HTTPHeaderContentTypeValJSON HTTP header value
	HTTPHeaderContentTypeValJSON = "application/json"
)
View Source 
const (
	// TimeFormatDate "2006-01-02"
	TimeFormatDate = "2006-01-02"
	// Nano2Sec 1e9
	Nano2Sec = 1e9
	// BitSize64 64
	BitSize64 = 64
	// BaseHex 16
	BaseHex = 16
)
View Source 
const ANSIColorEscape = "\x1b"

ANSIColorEscape escape string for ANSI color

Variables

View Source 
var (
	// ParseTs2UTC can parse unix timestamp(int64) to time.Time
	ParseTs2UTC = ParseUnix2UTC
	// ParseTs2String can parse unix timestamp(int64) to string
	ParseTs2String = ParseUnix2String
)
View Source 
var (
	// ParseTs2Time can parse unix timestamp(int64) to time.Time
	ParseTs2Time = ParseTs2UTC
	// UnixNano2UTC convert unixnano to UTC time
	UnixNano2UTC = ParseUnixNano2UTC
)
View Source 
var (
	// TimeZoneUTC timezone UTC
	TimeZoneUTC = time.UTC
	// TimeZoneShanghai timezone Shanghai
	// TimeZoneShanghai = time.FixedZone("Asia/Shanghai", 8*3600)
	TimeZoneShanghai *time.Location
)
View Source 
var (
	// ForceGC force to start gc blocking
	ForceGC = ForceGCBlocking
	// TriggerGC force to start gc unblocking
	TriggerGC = ForceGCUnBlocking
)
View Source 
var (
	// Clock high performance time utils, replace Clock1
	Clock = NewClock(context.Background(), defaultClockInterval)
)
View Source 
var (
	// ErrAsyncTask root error for async tasks
	ErrAsyncTask = errors.New("async task error")
)
View Source 
var (

	// JSON effective json
	JSON = json
)

Functions

func AbsInt32 

func AbsInt32(v int32) int32

AbsInt32 abs(v)

ignore int exceeds limit error, abs(MinInt32) == MaxInt32

func AbsInt64 

func AbsInt64(v int64) int64

AbsInt64 abs(v)

ignore int exceeds limit error, abs(MinInt64) == MaxInt64

func AutoGC 

func AutoGC(ctx context.Context, opts ...GcOptFunc) (err error)

AutoGC auto trigger GC when memory usage exceeds the custom ration

default to /sys/fs/cgroup/memory/memory.limit_in_bytes

Example 
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
if err := AutoGC(
	ctx,
	WithGCMemRatio(85), // default
	WithGCMemLimitFilePath("/sys/fs/cgroup/memory/memory.limit_in_bytes"), // default
); err != nil {
	log.Shared.Error("enable autogc", zap.Error(err))
}

Output:

func Base64Decode 

func Base64Decode(encoded string) ([]byte, error)

Base64Decode decode string to bytes use base64

func Base64Encode 

func Base64Encode(raw []byte) string

Base64Encode encode bytes to string use base64

func Bytes2Str 

func Bytes2Str(b []byte) string

Bytes2Str unsafe convert bytes to str

func CheckResp 

func CheckResp(resp *http.Response) error

CheckResp check HTTP response's status code and return the error with body message

func Color 

func Color(color int, s string) string

Color wrap with ANSI color

inspired by github.com/fatih/color

func Contains 

func Contains[V comparable](collection []V, ele V) bool

Contains if collection contains ele

func ConvertMap2StringKey 

func ConvertMap2StringKey(inputMap any) map[string]any

ConvertMap2StringKey convert any map to `map[string]any`

func CopyFile 

func CopyFile(src, dst string, optfs ...CopyFileOptionFunc) (err error)

CopyFile copy file content from src to dst

func CostSecs added in v3.2.0 

func CostSecs(cost time.Duration) string

CostSecs convert duration to string like `0.25s`

func Dedent 

func Dedent(v string, optfs ...DedentOptFunc) string

Dedent removes leading whitespace or tab from the beginning of each line

will replace all tab to 4 blanks.

func DeepClone 

func DeepClone(src any) (dst any)

DeepClone deep clone a struct

will ignore all unexported fields

func DirSize 

func DirSize(path string) (size int64, err error)

DirSize calculate directory size

inspired by https://stackoverflow.com/a/32482941/2368737

Example 
dirPath := "."
size, err := DirSize(dirPath)
if err != nil {
	log.Shared.Error("get dir size", zap.Error(err), zap.String("path", dirPath))
}
log.Shared.Info("got size", zap.Int64("size", size), zap.String("path", dirPath))

Output:

func EmptyAllChans 

func EmptyAllChans[T any](chans ...chan T)

EmptyAllChans receive all thins in all chans

func FallBack 

func FallBack(orig func() any, fallback any) (ret any)

FallBack return the fallback when orig got error utils.FallBack(func() any { return getIOStatMetric(fs) }, &IOStat{}).(*IOStat)

Example 
targetFunc := func() any {
	panic("someting wrong")
}

FallBack(targetFunc, 10) // got 10

Output:

func FileExists added in v3.1.1 

func FileExists(path string) (bool, error)

FileExists is path a valid file

func FileMD5 

func FileMD5(path string) (hashed string, err error)

FileMD5 read file and calculate MD5

func FileSHA1 

func FileSHA1(path string) (hashed string, err error)

FileSHA1 read file and calculate sha1

return hashed string in 40 bytes

func FlattenMap 

func FlattenMap(data map[string]any, delimiter string)

FlattenMap make embedded map into flatten map

Example 
data := map[string]any{
	"a": "1",
	"b": map[string]any{
		"c": 2,
		"d": map[string]any{
			"e": 3,
		},
	},
}
FlattenMap(data, "__")
fmt.Println(data)

Output:

map[a:1 b__c:2 b__d__e:3]

func ForceGCBlocking 

func ForceGCBlocking()

ForceGCBlocking force to run blocking manual gc.

Example 
ForceGCBlocking()

Output:

func ForceGCUnBlocking 

func ForceGCUnBlocking()

ForceGCUnBlocking trigger GC unblocking

Example 
ForceGCUnBlocking()

Output:

func GetFuncName 

func GetFuncName(f any) string

GetFuncName return the name of func

Example 
GetFuncName(testFoo) // "github.com/Laisky/go-utils.testFoo"

Output:

func GetStructFieldByName 

func GetStructFieldByName(st any, fieldName string) any

GetStructFieldByName get struct field by name

func GracefulCancel 

func GracefulCancel(cancel func())

GracefulCancel is a function that will be called when the process is about to be terminated.

func HTTPInvalidStatusError 

func HTTPInvalidStatusError(statusCode int) error

HTTPInvalidStatusError return error about status code

func HasField 

func HasField(st any, fieldName string) bool

HasField check is struct has field

inspired by https://mrwaggel.be/post/golang-reflect-if-initialized-struct-has-member-method-or-fields/

func HasMethod 

func HasMethod(st any, methodName string) bool

HasMethod check is struct has method

inspired by https://mrwaggel.be/post/golang-reflect-if-initialized-struct-has-member-method-or-fields/

func HumanReadableByteCount 

func HumanReadableByteCount(bytes int64, si bool) (ret string)

HumanReadableByteCount convert bytes to human readable string

Args:

  • bytes:
  • si: `si ? 1024 : 1000`

Example: 

`HumanReadableByteCount(1005, false) -> "1.01KB"`

func InArray deprecated 

func InArray(collection any, ele any) bool

InArray if collection contains ele

Deprecated: use Contains instead

func IntersectSortedChans 

func IntersectSortedChans[T Sortable](chans ...chan T) (result chan T, err error)

IntersectSortedChans return the intersection of multiple sorted chans

func IsDir 

func IsDir(path string) (bool, error)

IsDir is path exists as dir

func IsDirWritable 

func IsDirWritable(dir string) (err error)

IsDirWritable if dir is writable

func IsEmpty added in v3.1.2 

func IsEmpty(val any) bool

IsEmpty is empty

func IsFile 

func IsFile(path string) (bool, error)

IsFile is path exists as file

func IsFileATimeChanged 

func IsFileATimeChanged(path string, expectATime time.Time) (changed bool, newATime time.Time, err error)

IsFileATimeChanged check is file's atime equal to expectATime

func IsPanic 

func IsPanic(f func()) (isPanic bool)

IsPanic is `f()` throw panic

func IsPtr 

func IsPtr(t any) bool

IsPtr check if t is pointer

func IsRemoteUDPPortOpen 

func IsRemoteUDPPortOpen(addr string) error

IsRemoteUDPPortOpen check is remote udp port open. return err if remote port is not open.

Args: 

addr: "scanme.nmap.org:53"

func JumpHash 

func JumpHash(key uint64, numBuckets int) (int32, error)

JumpHash fatest consistent hashing created by google. inspired by https://medium.com/@dgryski/consistent-hashing-algorithmic-tradeoffs-ef6b8e2fcae8

func ListFilesInDir 

func ListFilesInDir(dir string, optfs ...ListFilesInDirOptionFunc) (files []string, err error)

ListFilesInDir list files in dir

func MD5JSON 

func MD5JSON(data any) (string, error)

MD5JSON calculate md5(jsonify(data))

func Max 

func Max[T Sortable](vals ...T) T

Max return the maximal value

func Min 

func Min[T Sortable](vals ...T) T

Min return the minimal value

func MoveFile 

func MoveFile(src, dst string) (err error)

MoveFile move file from src to dst by copy

sometimes move file by `rename` not work. for example, you can not move file between docker volumes by `rename`.

func NewHTTPClient 

func NewHTTPClient(opts ...HTTPClientOptFunc) (c *http.Client, err error)

NewHTTPClient create http client

func NewRand 

func NewRand() *rand.Rand

NewRand new individual random to aviod global mutex

func NewTmpFileForContent 

func NewTmpFileForContent(content []byte) (path string, err error)

NewTmpFileForContent write content to tmp file and return path

func NilInterface 

func NilInterface(data any) bool

NilInterface make sure data is nil interface or another type with nil value

Example: 

type foo struct{}
var f *foo
var v any
v = f
v == nil // false
NilInterface(v) // true

func NotEmpty added in v3.1.2 

func NotEmpty(val any, name string) error

NotEmpty val should not be empty, with pretty error msg

func OptionalVal added in v3.1.2 

func OptionalVal[T any](ptr *T, optionalVal T) T

OptionalVal return optionval if not empty

func PanicIfErr 

func PanicIfErr(err error)

PanicIfErr panic if err is not nil

func ParseHex2UTC 

func ParseHex2UTC(ts string) (t time.Time, err error)

ParseHex2UTC parse hex to UTC time

func ParseHexNano2UTC 

func ParseHexNano2UTC(ts string) (t time.Time, err error)

ParseHexNano2UTC parse hex contains nano to UTC time

func ParseUnix2String 

func ParseUnix2String(ts int64, layout string) string

ParseUnix2String can parse unix timestamp(int64) to string

func ParseUnix2UTC 

func ParseUnix2UTC(ts int64) time.Time

ParseUnix2UTC convert unix to UTC time

func ParseUnixNano2UTC 

func ParseUnixNano2UTC(ts int64) time.Time

ParseUnixNano2UTC convert unixnano to UTC time

func Pipeline added in v3.2.0 

func Pipeline[T any](funcs []func(T) error, v T) (T, error)

Pipeline run f(v) for all funcs

func PrettyBuildInfo 

func PrettyBuildInfo() string

PrettyBuildInfo get build info in formatted json

Print: 

{
  "Path": "github.com/Laisky/go-ramjet",
  "Version": "v0.0.0-20220718014224-2b10e57735f1",
  "Sum": "h1:08Ty2gR+Xxz0B3djHVuV71boW4lpNdQ9hFn4ZIGrhec=",
  "Replace": null
}

func Race deprecated 

func Race(gs ...func())

Race return when any goroutine returned

Deprecated: use RaceErr instead

Example 
startAt := time.Now()
Race(
	func() { time.Sleep(time.Millisecond) },
	func() { time.Sleep(time.Second) },
	func() { time.Sleep(time.Minute) },
)

fmt.Println(time.Since(startAt) < time.Second)

Output:

true

func RaceErr 

func RaceErr(gs ...func() error) (err error)

RaceErr return when any goroutine returned

func RaceErrWithCtx 

func RaceErrWithCtx(ctx context.Context, gs ...func(context.Context) error) error

RaceErrWithCtx return when any goroutine returned or ctx canceled

func RandomBytesWithLength 

func RandomBytesWithLength(n int) ([]byte, error)

RandomBytesWithLength generate random bytes

func RandomStringWithLength 

func RandomStringWithLength(n int) string

RandomStringWithLength generate random string with specific length

func RegexNamedSubMatch 

func RegexNamedSubMatch(r *regexp.Regexp, str string, subMatchMap map[string]string) error

RegexNamedSubMatch extract key:val map from string by group match

Example 
reg := regexp.MustCompile(`(?P<key>\d+.*)`)
str := "12345abcde"
groups := map[string]string{}
if err := RegexNamedSubMatch(reg, str, groups); err != nil {
	log.Shared.Error("try to group match got error", zap.Error(err))
}

fmt.Println(groups)

Output:

map[key:12345abcde]

func RemoveEmpty 

func RemoveEmpty(vs []string) (r []string)

RemoveEmpty remove duplicate string in slice

func RenderTemplate added in v3.3.0 

func RenderTemplate(tplContent string, args any) ([]byte, error)

RenderTemplate render template with args

func RenderTemplateFile added in v3.3.0 

func RenderTemplateFile(tplFile string, args any) ([]byte, error)

RenderTemplateFile render template file with args

func RequestJSON 

func RequestJSON(method, url string, request *RequestData, resp any) (err error)

RequestJSON request JSON and return JSON by default client

func RequestJSONWithClient 

func RequestJSONWithClient(httpClient *http.Client,
	method,
	url string,
	request *RequestData,
	resp any,
) (err error)

RequestJSONWithClient request JSON and return JSON with specific client

func Round 

func Round(val float64, d int) float64

Round round float64

Round(1.005, 2) -> 1.01

func RunCMD 

func RunCMD(ctx context.Context, app string, args ...string) (stdout []byte, err error)

RunCMD run command script

func RunCMDWithEnv added in v3.1.4 

func RunCMDWithEnv(ctx context.Context, app string,
	args []string, envs []string) (stdout []byte, err error)

RunCMDWithEnv run command with environments

Args

  • envs: []string{"FOO=BAR"}

func RunWithTimeout 

func RunWithTimeout(timeout time.Duration, f func())

RunWithTimeout run func with timeout

Example 
slow := func() { time.Sleep(10 * time.Second) }
startAt := time.Now()
RunWithTimeout(5*time.Millisecond, slow)

fmt.Println(time.Since(startAt) < 10*time.Second)

Output:

true

func SecRandInt 

func SecRandInt(n int) (int, error)

SecRandInt generate security int

func SecRandomBytesWithLength 

func SecRandomBytesWithLength(n int) ([]byte, error)

SecRandomBytesWithLength generate crypto random bytes

func SecRandomStringWithLength 

func SecRandomStringWithLength(n int) (string, error)

SecRandomStringWithLength generate random string with specific length

func SetInternalClock 

func SetInternalClock(interval time.Duration)

SetInternalClock set internal Clock with refresh interval

func SetStructFieldsBySlice 

func SetStructFieldsBySlice(structs, vals any) (err error)

SetStructFieldsBySlice set field value of structs slice by values slice

Example 
type ST struct{ A, B string }
var (
	err error
	ss  = []*ST{{}, {}}
	vs  = [][]string{
		{"x0", "y0"},
		{"x1", "y1"},
	}
)
if err = SetStructFieldsBySlice(ss, vs); err != nil {
	log.Shared.Error("set struct val", zap.Error(err))
	return
}

fmt.Printf("%+v\n", ss)
// ss = []*ST{{A: "x0", B: "y0"}, {A: "x1", B: "y1"}}

Output:

func SilentClose 

func SilentClose(v interface{ Close() error })

SilentClose close and ignore error

Example 

defer SilentClose(fp)

func SilentFlush 

func SilentFlush(v interface{ Flush() error })

SilentFlush flush and ignore error

func SleepWithContext 

func SleepWithContext(ctx context.Context, duration time.Duration)

SleepWithContext sleep duration with context, if context is done, return

func StopSignal 

func StopSignal(optfs ...StopSignalOptFunc) (stopCh <-chan struct{})

StopSignal registered for SIGTERM and SIGINT. A stop channel is returned which is closed on one of these signals. If a second signal is caught, the program is terminated with exit code 1.

Copied from https://github.com/kubernetes/sample-controller

func Str2Bytes 

func Str2Bytes(s string) []byte

Str2Bytes unsafe convert str to bytes

func TemplateWithMap 

func TemplateWithMap(tpl string, data map[string]any) string

TemplateWithMap replace `${var}` in template string

func TemplateWithMapAndRegexp 

func TemplateWithMapAndRegexp(tplReg *regexp.Regexp, tpl string, data map[string]any) string

TemplateWithMapAndRegexp replace `${var}` in template string

func TrimEleSpaceAndRemoveEmpty 

func TrimEleSpaceAndRemoveEmpty(vs []string) (r []string)

TrimEleSpaceAndRemoveEmpty remove duplicate string in slice

func URLMasking 

func URLMasking(url, mask string) string

URLMasking masking password in url

Example 
originURL := "http://12ijij:3j23irj@jfjlwef.ffe.com"
newURL := URLMasking(originURL, "*****")
fmt.Println(newURL)

Output:

http://12ijij:*****@jfjlwef.ffe.com

func UTCNow 

func UTCNow() time.Time

UTCNow get current time in utc

func UniqueStrings 

func UniqueStrings(vs []string) (r []string)

UniqueStrings remove duplicate string in slice

func ValidateFileHash 

func ValidateFileHash(filepath string, hashed string) error

ValidateFileHash validate file content with hashed string

Args:

  • filepath: file path to check
  • hashed: hashed string, like `sha256: xxxx`

func WatchFileChanging 

func WatchFileChanging(ctx context.Context, files []string, callback func(fsnotify.Event)) error

WatchFileChanging watch file changing

when file changed, callback will be called, callback will only received fsnotify.Write no matter what happened to changing a file.

TODO: only calculate hash when file's folder got fsnotiy

Types

type AsyncTask 

type AsyncTask interface {
	// ID get task id
	ID() string
	// Status get task status, pending/done/failed
	Status() AsyncTaskStatus
	// SetDone set task done with result data
	SetDone(ctx context.Context, data string) (err error)
	// SetError set task error with err message
	SetError(ctx context.Context, errMsg string) (err error)
}

asyncTask async task

func NewAsyncTask 

func NewAsyncTask(ctx context.Context, store AsyncTaskStore) (AsyncTask, error)

NewTask new async task

type AsyncTaskResult 

type AsyncTaskResult struct {
	TaskID string          `json:"task_id"`
	Status AsyncTaskStatus `json:"status"`
	Data   string          `json:"data"`
	Err    string          `json:"err"`
}

AsyncTaskResult result of async task

type AsyncTaskStatus 

type AsyncTaskStatus string

AsyncTaskStatus status of async task 

const (
	// AsyncTaskStatusPending task pending
	AsyncTaskStatusPending AsyncTaskStatus = "pending"
	// AsyncTaskStatusDone task done
	AsyncTaskStatusDone AsyncTaskStatus = "done"
	// AsyncTaskStatusFailed task failed
	AsyncTaskStatusFailed AsyncTaskStatus = "failed"
)

func (AsyncTaskStatus) String 

func (s AsyncTaskStatus) String() string

String convert status to string

type AsyncTaskStore 

type AsyncTaskStore interface {
	// New create new AsyncTaskResult with id
	New(ctx context.Context) (result *AsyncTaskResult, err error)
	// Set set AsyncTaskResult
	Set(ctx context.Context, taskID string, result *AsyncTaskResult) (err error)
	// Heartbeat refresh async task's updated time to mark this task is still alive
	Heartbeat(ctx context.Context, taskID string) (alived bool, err error)
}

AsyncTaskStore persistency storage for async task

type ClockItf 

type ClockItf interface {
	Close()

	GetUTCNow() time.Time
	GetDate() (time.Time, error)
	GetTimeInRFC3339Nano() string
	SetInterval(interval time.Duration)
	GetTimeInHex() string
	GetNanoTimeInHex() string
	Interval() time.Duration
	// contains filtered or unexported methods
}

ClockItf high performance lazy clock

type ClockT added in v3.1.0 

type ClockT struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

ClockT high performance ClockT with lazy refreshing

func NewClock 

func NewClock(ctx context.Context, refreshInterval time.Duration) *ClockT

NewClock create new Clock

func (*ClockT) Close added in v3.1.0 

func (c *ClockT) Close()

Close stop Clock update

func (*ClockT) GetDate added in v3.1.0 

func (c *ClockT) GetDate() (time.Time, error)

GetDate return "yyyy-mm-dd"

func (*ClockT) GetNanoTimeInHex added in v3.1.0 

func (c *ClockT) GetNanoTimeInHex() string

GetNanoTimeInHex return current time with nano in hex

func (*ClockT) GetTimeInHex added in v3.1.0 

func (c *ClockT) GetTimeInHex() string

GetTimeInHex return current time in hex

func (*ClockT) GetTimeInRFC3339Nano added in v3.1.0 

func (c *ClockT) GetTimeInRFC3339Nano() string

GetTimeInRFC3339Nano return Clock current time in string

func (*ClockT) GetUTCNow added in v3.1.0 

func (c *ClockT) GetUTCNow() time.Time

GetUTCNow return Clock current time.Time

func (*ClockT) Interval added in v3.1.0 

func (c *ClockT) Interval() time.Duration

Interval get current interval

func (*ClockT) SetInterval added in v3.1.0 

func (c *ClockT) SetInterval(interval time.Duration)

SetInterval setup update interval

type CopyFileOptionFunc added in v3.1.2 

type CopyFileOptionFunc func(o *copyFileOption) error

CopyFileOptionFunc set options for copy file

func Overwrite added in v3.1.2 

func Overwrite() CopyFileOptionFunc

Overwrite overwrite file if target existed

func WithFileFlag added in v3.1.1 

func WithFileFlag(flag int) CopyFileOptionFunc

WithFileFlag how to write dst file

func WithFileMode added in v3.1.1 

func WithFileMode(perm fs.FileMode) CopyFileOptionFunc

WithFileMode if create new dst file, set the file's mode

type DedentOptFunc 

type DedentOptFunc func(opt *dedentOpt)

DedentOptFunc dedent option

func WithReplaceTabBySpaces 

func WithReplaceTabBySpaces(spaces int) DedentOptFunc

WithReplaceTabBySpaces replace tab to spaces

type ExpCache 

type ExpCache[T any] struct {
	// contains filtered or unexported fields
}

ExpCache cache with expires

can Store/Load like map

Example 
cc := NewExpCache[string](context.Background(), 100*time.Millisecond)
cc.Store("key", "val")
cc.Load("key") // return "val"

// data expired
time.Sleep(200 * time.Millisecond)
data, ok := cc.Load("key")
fmt.Println(data)
fmt.Println(ok)

Output:

<nil>
false

func NewExpCache 

func NewExpCache[T any](ctx context.Context, ttl time.Duration) *ExpCache[T]

NewExpCache new cache manager

use with generic: 

cc := NewExpCache[string](context.Background(), 100*time.Millisecond)
cc.Store("key", "val")
val, ok := cc.Load("key")

func (*ExpCache[T]) Delete 

func (c *ExpCache[T]) Delete(key string)

Delete remove key

func (*ExpCache[T]) Load 

func (c *ExpCache[T]) Load(key string) (data T, ok bool)

Load load val from cache

func (*ExpCache[T]) LoadAndDelete added in v3.3.0 

func (c *ExpCache[T]) LoadAndDelete(key string) (data T, ok bool)

LoadAndDelete load and delete val from cache

func (*ExpCache[T]) Store 

func (c *ExpCache[T]) Store(key string, val T)

Store store new key and val into cache

type ExpCacheInterface added in v3.3.0 

type ExpCacheInterface[T any] interface {
	// Store store new key and val into cache
	Store(key string, val T)
	// Delete remove key
	Delete(key string)
	// LoadAndDelete load and delete val from cache
	LoadAndDelete(key string) (data T, ok bool)
	// Load load val from cache
	Load(key string) (data T, ok bool)
}

ExpCacheInterface cache with expire duration

type ExpiredRLock 

type ExpiredRLock struct {
	// contains filtered or unexported fields
}

ExpiredRLock Lock with expire time

func NewExpiredRLock 

func NewExpiredRLock(ctx context.Context, exp time.Duration) (el *ExpiredRLock, err error)

NewExpiredRLock new ExpiredRLock

func (*ExpiredRLock) GetLock 

func (e *ExpiredRLock) GetLock(key string) *sync.RWMutex

GetLock get lock

type FLock 

type FLock interface {
	Lock() error
	Unlock() error
}

FLock lock by file

func NewFlock 

func NewFlock(lockFilePath string) FLock

NewFlock new file lock

type GcOptFunc 

type GcOptFunc func(*gcOption) error

GcOptFunc option for GC utils

func WithGCMemLimitFilePath 

func WithGCMemLimitFilePath(path string) GcOptFunc

WithGCMemLimitFilePath set memory limit file

func WithGCMemRatio 

func WithGCMemRatio(ratio int) GcOptFunc

WithGCMemRatio set mem ratio trigger for GC

default to 85

type GoroutineTest added in v3.1.0 

type GoroutineTest struct {
	testing.TB
	// contains filtered or unexported fields
}

GoroutineTest testing.T support goroutine

func NewGoroutineTest added in v3.1.0 

func NewGoroutineTest(t testing.TB, cancel func()) *GoroutineTest

NewGoroutineTest new test for goroutine

any fail will call cancel()

func (*GoroutineTest) FailNow added in v3.1.0 

func (t *GoroutineTest) FailNow()

FailNow call cancal and exit current goroutine

type HTTPClientOptFunc 

type HTTPClientOptFunc func(*httpClientOption) error

HTTPClientOptFunc http client options

func WithHTTPClientInsecure 

func WithHTTPClientInsecure() HTTPClientOptFunc

WithHTTPClientInsecure set http client igonre ssl issue

default to false

func WithHTTPClientMaxConn 

func WithHTTPClientMaxConn(maxConn int) HTTPClientOptFunc

WithHTTPClientMaxConn set http client max connection

default to 20

func WithHTTPClientTimeout 

func WithHTTPClientTimeout(timeout time.Duration) HTTPClientOptFunc

WithHTTPClientTimeout set http client timeout

default to 30s

type LRUExpiredMap 

type LRUExpiredMap struct {
	// contains filtered or unexported fields
}

LRUExpiredMap map with expire time, auto delete expired item.

`Get` will auto refresh item's expires.

func NewLRUExpiredMap 

func NewLRUExpiredMap(ctx context.Context,
	ttl time.Duration,
	new func() any) (el *LRUExpiredMap, err error)

NewLRUExpiredMap new ExpiredMap

func (*LRUExpiredMap) Get 

func (e *LRUExpiredMap) Get(key string) any

Get get item

will auto refresh key's ttl

type ListFilesInDirOptionFunc added in v3.1.2 

type ListFilesInDirOptionFunc func(*listFilesInDirOption) error

ListFilesInDirOptionFunc options for ListFilesInDir

func Recursive added in v3.1.2 

func Recursive() ListFilesInDirOptionFunc

Recursive list files recursively

type Mutex 

type Mutex struct {
	// contains filtered or unexported fields
}

Mutex mutex that support unblocking lock

Example 
l := NewMutex()
if !l.TryLock() {
	log.Shared.Info("can not acquire lock")
	return
}
defer l.ForceRelease()

Output:

func NewMutex 

func NewMutex() *Mutex

NewMutex create new mutex

func (*Mutex) ForceRelease 

func (m *Mutex) ForceRelease()

ForceRelease force release lock

func (*Mutex) IsLocked 

func (m *Mutex) IsLocked() bool

IsLocked return true if is locked

func (*Mutex) SpinLock 

func (m *Mutex) SpinLock(step, timeout time.Duration)

SpinLock block until succee acquired lock

func (*Mutex) TryLock 

func (m *Mutex) TryLock() bool

TryLock return true if succeed locked

func (*Mutex) TryRelease 

func (m *Mutex) TryRelease() bool

TryRelease return true if succeed release

type Number 

type Number interface {
	int | int8 | int16 | int32 | int64 |
		uint | uint8 | uint16 | uint32 | uint64 |
		float32 | float64
}

Number is a number type

type PairList 

type PairList []SortItemItf

PairList array of sort items

func SortBiggest 

func SortBiggest(items PairList) PairList

SortBiggest sort from biggest to smallest

func SortSmallest 

func SortSmallest(items PairList) PairList

SortSmallest sort from smallest to biggest

func (PairList) Len 

func (p PairList) Len() int

Len return length of sort items

func (PairList) Less 

func (p PairList) Less(i, j int) bool

Less compare two items

func (PairList) Swap 

func (p PairList) Swap(i, j int)

Swap change two items

type RBACPermFullKey 

type RBACPermFullKey string

RBACPermFullKey key with ancesters

func (RBACPermFullKey) Append 

func (p RBACPermFullKey) Append(key RBACPermKey) RBACPermFullKey

Append new key to full key

func (RBACPermFullKey) Contains 

func (p RBACPermFullKey) Contains(acquire RBACPermFullKey) bool

Contains is contains acquire permission

func (RBACPermFullKey) Parent 

func (p RBACPermFullKey) Parent() RBACPermFullKey

Parent get element parent key

func (RBACPermFullKey) String 

func (p RBACPermFullKey) String() string

String to string

type RBACPermKey 

type RBACPermKey string

RBACPermKey permission identity keyword

format like `a`

func (RBACPermKey) String 

func (p RBACPermKey) String() string

String to string

type RBACPermissionElem 

type RBACPermissionElem struct {
	// Title display name of this element
	Title string `json:"title" binding:"min=1"`
	// Key element's identity
	Key RBACPermKey `json:"key,omitempty"`
	// FullKey within all ancester keys, demilite by rbacPermKeyDelimiter
	FullKey  RBACPermFullKey       `json:"full_key,omitempty"`
	Children []*RBACPermissionElem `json:"children,omitempty"`
}

RBACPermissionElem element node of permission tree

the whole permission tree can represented by the head node

func NewPermissionTree 

func NewPermissionTree() *RBACPermissionElem

NewPermissionTree new permission tree only contains root node

func (*RBACPermissionElem) Clone 

func (p *RBACPermissionElem) Clone() *RBACPermissionElem

Clone clone permission tree

func (*RBACPermissionElem) Cut 

func (p *RBACPermissionElem) Cut(key RBACPermFullKey)

Cut 剪除指定节点

Args: 

key: 形如 `root.sys.a.b`，或 `root.sys.a.*`

头节点不允许剪除。 可以使用 `*` 作为通配符，代表剪除所有子节点。

func (*RBACPermissionElem) FillDefault 

func (p *RBACPermissionElem) FillDefault(ancesterKey RBACPermFullKey) error

FillDefault auto filling some default valus

it is best to call this function immediately after initialization

func (*RBACPermissionElem) GetElemByKey 

func (p *RBACPermissionElem) GetElemByKey(key RBACPermFullKey) *RBACPermissionElem

GetElemByKey 通过 key 获取指定的权限树节点

Args:

  • key: 权限树路径，形如 `root.sys`

func (*RBACPermissionElem) HasPerm 

func (p *RBACPermissionElem) HasPerm(acquiredKey RBACPermFullKey) bool

HasPerm check whether has specified key 

| user prems   | acquired key | match  |
| :----------: | :----------: | :---:  |
|   `"root"`   |   `"root"`   |   ✅   |
|     `""`     |   `"root"`   |   ❌   |
|   `"root"`   |     `""`     |   ✅   |
| `"root.sys"` |   `"root"`   |   ✅   |
|   `"root"`   | `"root.sys"` |   ❌   |

func (*RBACPermissionElem) Intersection 

func (p *RBACPermissionElem) Intersection(other *RBACPermissionElem)

Intersection intersect with other permission tree

func (*RBACPermissionElem) OverwriteBy 

func (p *RBACPermissionElem) OverwriteBy(another *RBACPermissionElem, intersection bool)

OverwriteBy overwrite element's content by another tree, but do not append any element from another tree if not exists in current tree.

Args:

  • intersection: if set to true, will intersect by another tree

func (*RBACPermissionElem) Scan 

func (p *RBACPermissionElem) Scan(input any) error

Scan implement GORM interface

func (*RBACPermissionElem) UnionAndOverwriteBy 

func (p *RBACPermissionElem) UnionAndOverwriteBy(other *RBACPermissionElem)

UnionAndOverwriteBy merge(union) another tree into this tree by key comparison

func (*RBACPermissionElem) Valid 

func (p *RBACPermissionElem) Valid() error

Valid valid permission tree

func (RBACPermissionElem) Value 

func (p RBACPermissionElem) Value() (driver.Value, error)

Value implement GORM interface

type RequestData 

type RequestData struct {
	Headers map[string]string
	Data    any
}

RequestData http request

type SingleItemExpCache 

type SingleItemExpCache struct {
	// contains filtered or unexported fields
}

SingleItemExpCache single item with expires

func NewSingleItemExpCache 

func NewSingleItemExpCache(ttl time.Duration) *SingleItemExpCache

NewSingleItemExpCache new expcache contains single data

func (*SingleItemExpCache) Get 

func (c *SingleItemExpCache) Get() (data any, ok bool)

Get get data

if data is expired, ok=false

func (*SingleItemExpCache) GetString 

func (c *SingleItemExpCache) GetString() (data string, ok bool)

GetString same as Get, but return string

func (*SingleItemExpCache) GetUintSlice 

func (c *SingleItemExpCache) GetUintSlice() (data []uint, ok bool)

GetUintSlice same as Get, but return []uint

func (*SingleItemExpCache) Set 

func (c *SingleItemExpCache) Set(data any)

Set set data and refresh expires

type SortItemItf 

type SortItemItf interface {
	GetValue() int
	GetData() any
}

SortItemItf interface of sort item

type Sortable 

type Sortable interface {
	Number | string
}

Sortable Data types that can be compared by >, <, ==

type StopSignalOptFunc 

type StopSignalOptFunc func(*stopSignalOpt)

StopSignalOptFunc options for StopSignal

func WithStopSignalCloseSignals 

func WithStopSignalCloseSignals(signals ...os.Signal) StopSignalOptFunc

WithStopSignalCloseSignals set signals that will trigger close

type Throttle 

type Throttle struct {
	*ThrottleCfg
	// contains filtered or unexported fields
}

Throttle current limitor

Example 
ctx := context.Background()
throttle, err := NewThrottleWithCtx(ctx, &ThrottleCfg{
	NPerSec: 10,
	Max:     100,
})
if err != nil {
	panic("new throttle")
}
defer throttle.Close()

inChan := make(chan int)

for msg := range inChan {
	if !throttle.Allow() {
		continue
	}

	// do something with msg
	fmt.Println(msg)
}

Output:

func NewThrottleWithCtx 

func NewThrottleWithCtx(ctx context.Context, cfg *ThrottleCfg) (t *Throttle, err error)

NewThrottleWithCtx create new Throttle

90x faster than `rate.NewLimiter`

func (*Throttle) Allow 

func (t *Throttle) Allow() bool

Allow check whether is allowed

func (*Throttle) Close 

func (t *Throttle) Close()

Close stop throttle

type ThrottleCfg 

type ThrottleCfg struct {
	Max, NPerSec int
}

ThrottleCfg Throttle's configuration

Source Files

View all Source files

Directories

Path Synopsis
Package algorithm contains some useful algorithms
 Package algorithm contains some useful algorithms
cmd
Package cmd some useful tools for command argument
 Package cmd some useful tools for command argument
gutils
Package main gutils' command line tool
 Package main gutils' command line tool
Package compress contains some useful tools to compress/decompress data or files
 Package compress contains some useful tools to compress/decompress data or files
Package counter contains varias counter tools
 Package counter contains varias counter tools
Package email simple email sender
 Package email simple email sender
Package encrypt contains some useful tools to deal with encryption/decryption
 Package encrypt contains some useful tools to deal with encryption/decryption
Package gorm some useful tools for gorm
 Package gorm some useful tools for gorm
Package jwt all in one JWT sdk
 Package jwt all in one JWT sdk
Package log enhanced zap logger
 Package log enhanced zap logger

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.