x

package
v82.0.0-...-84d68f0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 22, 2017 License: AGPL-3.0, Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *
  • Licensed under the Apache License, Version 2.0 (the "License");
  • you may not use this file except in compliance with the License.
  • You may obtain a copy of the License at *
  • http://www.apache.org/licenses/LICENSE-2.0 *
  • Unless required by applicable law or agreed to in writing, software
  • distributed under the License is distributed on an "AS IS" BASIS,
  • WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  • See the License for the specific language governing permissions and
  • limitations under the License.

Package x contains some very common utilities used by Dgraph. These utilities are of "miscellaneous" nature, e.g., error checking.

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *

  • This program is free software: you can redistribute it and/or modify

  • it under the terms of the GNU Affero General Public License as published by

  • the Free Software Foundation, either version 3 of the License, or

  • (at your option) any later version. *

  • This program is distributed in the hope that it will be useful,

  • but WITHOUT ANY WARRANTY; without even the implied warranty of

  • MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the

  • GNU Affero General Public License for more details. *

  • You should have received a copy of the GNU Affero General Public License

  • along with this program. If not, see <http://www.gnu.org/licenses/>.

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *

  • Licensed under the Apache License, Version 2.0 (the "License");

  • you may not use this file except in compliance with the License.

  • You may obtain a copy of the License at *

  • http://www.apache.org/licenses/LICENSE-2.0 *

  • Unless required by applicable law or agreed to in writing, software

  • distributed under the License is distributed on an "AS IS" BASIS,

  • WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  • See the License for the specific language governing permissions and

  • limitations under the License.

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *

  • Licensed under the Apache License, Version 2.0 (the "License");

  • you may not use this file except in compliance with the License.

  • You may obtain a copy of the License at *

  • http://www.apache.org/licenses/LICENSE-2.0 *

  • Unless required by applicable law or agreed to in writing, software

  • distributed under the License is distributed on an "AS IS" BASIS,

  • WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  • See the License for the specific language governing permissions and

  • limitations under the License.

Index

Constants

View Source 
const (
	// TODO(pawan) - Make this 2 bytes long. Right now ParsedKey has byteType and
	// bytePrefix. Change it so that it just has one field which has all the information.
	ByteData = byte(0x00)

	ByteIndex    = byte(0x02)
	ByteReverse  = byte(0x04)
	ByteCount    = byte(0x08)
	ByteCountRev = ByteCount | ByteReverse
)
View Source 
const (
	TLSClientConfig tlsConfigType = iota
	TLSServerConfig
)
View Source 
const (
	Success                 = "Success"
	ErrorUnauthorized       = "ErrorUnauthorized"
	ErrorInvalidMethod      = "ErrorInvalidMethod"
	ErrorInvalidRequest     = "ErrorInvalidRequest"
	ErrorMissingRequired    = "ErrorMissingRequired"
	Error                   = "Error"
	ErrorNoData             = "ErrorNoData"
	ErrorUptodate           = "ErrorUptodate"
	ErrorNoPermission       = "ErrorNoPermission"
	ErrorInvalidMutation    = "ErrorInvalidMutation"
	ErrorServiceUnavailable = "ErrorServiceUnavailable"
	ValidHostnameRegex      = "^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\\-]*[A-Za-z0-9])$"
	Star                    = "_STAR_ALL"
	GrpcMaxSize             = 256 << 20
)

Error constants representing different types of errors.

Variables

View Source 
var (
	// These are cummulative
	PostingReads  *expvar.Int
	PostingWrites *expvar.Int
	BytesRead     *expvar.Int
	BytesWrite    *expvar.Int
	EvictedPls    *expvar.Int
	NumQueries    *expvar.Int
	CacheHit      *expvar.Int
	CacheMiss     *expvar.Int
	CacheRace     *expvar.Int

	// value at particular point of time
	PendingQueries   *expvar.Int
	PendingProposals *expvar.Int
	LcacheSize       *expvar.Int
	LcacheLen        *expvar.Int
	LcacheCapacity   *expvar.Int
	DirtyMapSize     *expvar.Int
	NumGoRoutines    *expvar.Int
	MemoryInUse      *expvar.Int
	HeapIdle         *expvar.Int
	TotalMemory      *expvar.Int
	TotalOSMemory    *expvar.Int
	ActiveMutations  *expvar.Int
	ServerHealth     *expvar.Int
	MaxPlSize        *expvar.Int
	MaxPlLength      *expvar.Int

	PredicateStats *expvar.Map
	Conf           *expvar.Map

	MaxPlSz int64
)
View Source 
var (
	Logger = log.New(os.Stderr, "", log.Lshortfile|log.Flags())
)
View Source 
var Nilbyte []byte
View Source 
var WhiteSpace = strings.NewReplacer(" ", "", "\t", "")

WhiteSpace Replacer removes spaces and tabs from a string.

Functions

func AddCorsHeaders 

func AddCorsHeaders(w http.ResponseWriter)

func AddInit 

func AddInit(f func())

AddInit adds a function to be run in x.Init, which should be called at the beginning of all mains.

func AssertTrue 

func AssertTrue(b bool)

AssertTrue asserts that b is true. Otherwise, it would log fatal.

func AssertTruef 

func AssertTruef(b bool, format string, args ...interface{})

AssertTruef is AssertTrue with extra info.

func AssertTruefNoTrace 

func AssertTruefNoTrace(b bool, format string, args ...interface{})

func Check 

func Check(err error)

Check logs fatal if err != nil.

func Check2 

func Check2(_ interface{}, err error)

Check2 acts as convenience wrapper around Check, using the 2nd argument as error.

func Checkf 

func Checkf(err error, format string, args ...interface{})

Checkf is Check with extra info.

func CountKey 

func CountKey(attr string, count uint32, reverse bool) []byte

func DataKey 

func DataKey(attr string, uid uint64) []byte

func Errorf 

func Errorf(format string, args ...interface{}) error

Errorf creates a new error with stack trace, etc.

func Fatalf 

func Fatalf(format string, args ...interface{})

Fatalf logs fatal.

func GenerateTLSConfig 

func GenerateTLSConfig(config TLSHelperConfig) (tlsCfg *tls.Config, reloadConfig func(), err error)

GenerateTLSConfig creates and returns a new *tls.Config with the configuration provided. If the ConfigType provided in TLSHelperConfig is TLSServerConfig, it's return a reload function. If any problem is found, an error is returned

func HealthCheck 

func HealthCheck() error

HealthCheck returns whether the server is ready to accept requests or not Load balancer would add the node to the endpoint once health check starts returning true

func IndexKey 

func IndexKey(attr, term string) []byte

func Init 

func Init()

Init initializes flags and run all functions in initFunc.

func IsTestRun 

func IsTestRun() bool

func LoadConfigFromYAML 

func LoadConfigFromYAML(file string)

loadConfigFromYAML reads configurations from specified YAML file.

func NewTrace 

func NewTrace(title string, ctx context.Context) (trace.Trace, context.Context)

func PageRange 

func PageRange(count, offset, n int) (int, int)

PageRange returns start and end indices given pagination params. Note that n is the size of the input list.

func ParseRequest 

func ParseRequest(w http.ResponseWriter, r *http.Request, data interface{}) bool

func PredicatePrefix 

func PredicatePrefix(predicate string) []byte

PredicatePrefix returns the prefix for all keys belonging to this predicate except schema key.

func PrintVersionOnly 

func PrintVersionOnly()

PrintVersionOnly prints version and other helpful information if --version.

func Printf 

func Printf(format string, args ...interface{})

Printf does a log.Printf. We often do printf for debugging but has to keep adding import "fmt" or "log" and removing them after we are done. Let's add Printf to "x" and include "x" almost everywhere. Caution: Do remember to call x.Init. For tests, you need a TestMain that calls x.Init.

func Println 

func Println(args ...interface{})

func ReadLine 

func ReadLine(r *bufio.Reader, buf *bytes.Buffer) error

Reads a single line from a buffered reader. The line is read into the passed in buffer to minimize allocations. This is the preferred method for loading long lines which could be longer than the buffer size of bufio.Scanner.

func RemoveDuplicates 

func RemoveDuplicates(s []string) (out []string)

sorts the slice of strings and removes duplicates. changes the input slice. this function should be called like: someSlice = x.RemoveDuplicates(someSlice)

func Reply 

func Reply(w http.ResponseWriter, rep interface{})

func ReverseKey 

func ReverseKey(attr string, uid uint64) []byte

func Round 

func Round(d time.Duration) time.Duration

Go doesn't have a round function for Duration and the duration value can be a bit ugly to look at sometimes. This is an attempt to round the value.

func SchemaKey 

func SchemaKey(attr string) []byte

SchemaKey returns schema key for given attribute, schema keys are stored separately with unique prefix, since we need to iterate over all schema keys

func SchemaPrefix 

func SchemaPrefix() []byte

SchemaPrefix returns the prefix for Schema keys.

func SetError 

func SetError(prev *error, n error)

SetError sets the error logged in this package.

func SetStatus 

func SetStatus(w http.ResponseWriter, code, msg string)

SetStatus sets the error code, message and the newly assigned uids in the http response.

func SetStatusWithData 

func SetStatusWithData(w http.ResponseWriter, code, msg string)

In case an error was encountered after the query execution started, we have to return data key with null value according to GraphQL spec.

func SetTestRun 

func SetTestRun()

func UpdateHealthStatus 

func UpdateHealthStatus(ok bool)

func UpdateMemoryStatus 

func UpdateMemoryStatus(ok bool)

func ValidateAddress 

func ValidateAddress(addr string) bool

ValidateAddress checks whether given address can be used with grpc dial function

func Version 

func Version() string

func Wrap 

func Wrap(err error) error

Wrap wraps errors from external lib.

func Wrapf 

func Wrapf(err error, format string, args ...interface{}) error

Wrapf is Wrap with extra info.

func WriteFileSync 

func WriteFileSync(filename string, data []byte, perm os.FileMode) error

WriteFileSync is the same as bufio.WriteFile, but syncs the data before closing.

Types

type BytesBuffer 

type BytesBuffer struct {
	// contains filtered or unexported fields
}

func (*BytesBuffer) CopyTo 

func (b *BytesBuffer) CopyTo(o []byte) int

Caller should ensure that o is of appropriate length

func (*BytesBuffer) Length 

func (b *BytesBuffer) Length() int

func (*BytesBuffer) Slice 

func (b *BytesBuffer) Slice(n int) []byte

returns a slice of lenght n to be used to writing

func (*BytesBuffer) TruncateBy 

func (b *BytesBuffer) TruncateBy(n int)

Always give back <= touched bytes

type EmbeddedUidAllocator 

type EmbeddedUidAllocator struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func (*EmbeddedUidAllocator) AssignUids 

func (e *EmbeddedUidAllocator) AssignUids(ctx context.Context,
	num *protos.Num) (*protos.AssignedIds, error)

func (*EmbeddedUidAllocator) Init 

func (e *EmbeddedUidAllocator) Init(kv *badger.KV)

Start lease from 2, 1 is used by _lease_

type Histogram 

type Histogram struct {
	sync.Mutex
	// contains filtered or unexported fields
}

A Histogram collects observed values by keeping bucketed counts. For convenience, internally two sets of buckets are kept: A cumulative set (i.e. data is never evicted) and a windowed set (which keeps only recently collected samples).

Top-level methods generally apply to the cumulative buckets; the windowed variant is exposed through the Windowed method.

func NewHistogram 

func NewHistogram(duration time.Duration, maxVal int64, sigFigs int) *Histogram

NewHistogram initializes a given Histogram. The contained windowed histogram rotates every 'duration'; both the windowed and the cumulative histogram track nonnegative values up to 'maxVal' with 'sigFigs' decimal points of precision.

func (*Histogram) RecordValue 

func (h *Histogram) RecordValue(v int64)

RecordValue adds the given value to the histogram. Recording a value in excess of the configured maximum value for that histogram results in recording the maximum value instead.

func (*Histogram) Stats 

func (h *Histogram) Stats()

type NonceCounter 

type NonceCounter struct {
	// contains filtered or unexported fields
}

NonceCounter can generate unique 128-bit numbers. Only usable on a single thread. (All it would take to be multithreaded is to increment hi atomically.)

func NewNonceCounter 

func NewNonceCounter() NonceCounter

NewNonceCounter reads from /dev/urandom (or such) and generates a new randomized NonceCounter.

func (*NonceCounter) Generate 

func (nc *NonceCounter) Generate() [16]byte

Generate returns a new nonce value.

func (*NonceCounter) GenerationNumber 

func (nc *NonceCounter) GenerationNumber(buf []byte) uint64

GenerationNumber returns N+C when buf was produced by the Nth call to Generate(), for some global constant C.

type Options 

type Options struct {
	ConfigFile string
	Version    bool
	DebugMode  bool
	PortOffset int
}
var Config Options

type ParsedKey 

type ParsedKey struct {
	Attr  string
	Uid   uint64
	Term  string
	Count uint32
	// contains filtered or unexported fields
}

func Parse 

func Parse(key []byte) *ParsedKey

func (ParsedKey) CountPrefix 

func (p ParsedKey) CountPrefix(reverse bool) []byte

CountPrefix returns the prefix for count keys.

func (ParsedKey) DataPrefix 

func (p ParsedKey) DataPrefix() []byte

DataPrefix returns the prefix for data keys.

func (ParsedKey) IndexPrefix 

func (p ParsedKey) IndexPrefix() []byte

IndexPrefix returns the prefix for index keys.

func (ParsedKey) IsCount 

func (p ParsedKey) IsCount() bool

func (ParsedKey) IsData 

func (p ParsedKey) IsData() bool

func (ParsedKey) IsIndex 

func (p ParsedKey) IsIndex() bool

func (ParsedKey) IsReverse 

func (p ParsedKey) IsReverse() bool

func (ParsedKey) IsSchema 

func (p ParsedKey) IsSchema() bool

func (ParsedKey) IsType 

func (p ParsedKey) IsType(typ byte) bool

func (ParsedKey) ReversePrefix 

func (p ParsedKey) ReversePrefix() []byte

ReversePrefix returns the prefix for index keys.

func (ParsedKey) SkipPredicate 

func (p ParsedKey) SkipPredicate() []byte

func (ParsedKey) SkipRangeOfSameType 

func (p ParsedKey) SkipRangeOfSameType() []byte

func (ParsedKey) SkipSchema 

func (p ParsedKey) SkipSchema() []byte

type QueryResWithData 

type QueryResWithData struct {
	Errors []errRes `json:"errors"`
	Data   *string  `json:"data"`
}

type RaftValue 

type RaftValue struct {
	Group uint32
	Index uint64
}

RaftValue contains the raft group and the raft proposal id. This is attached to the context, so the information could be passed down to the many posting lists, involved in mutations.

type SafeMutex 

type SafeMutex struct {
	// contains filtered or unexported fields
}

SafeLock can be used in place of sync.RWMutex

func (*SafeMutex) AssertLock 

func (s *SafeMutex) AssertLock()

func (*SafeMutex) AssertRLock 

func (s *SafeMutex) AssertRLock()

func (*SafeMutex) Lock 

func (s *SafeMutex) Lock()

func (*SafeMutex) RLock 

func (s *SafeMutex) RLock()

func (*SafeMutex) RUnlock 

func (s *SafeMutex) RUnlock()

func (*SafeMutex) StartWait 

func (s *SafeMutex) StartWait() *SafeWait

func (*SafeMutex) Unlock 

func (s *SafeMutex) Unlock()

func (*SafeMutex) Wait 

func (s *SafeMutex) Wait()

type SafeWait 

type SafeWait struct {
	// contains filtered or unexported fields
}

func (*SafeWait) Done 

func (s *SafeWait) Done()

type TLSHelperConfig 

type TLSHelperConfig struct {
	ConfigType             tlsConfigType
	CertRequired           bool
	Cert                   string
	Key                    string
	KeyPassphrase          string
	ServerName             string
	Insecure               bool
	RootCACerts            string
	UseSystemRootCACerts   bool
	ClientAuth             string
	ClientCACerts          string
	UseSystemClientCACerts bool
	MinVersion             string
	MaxVersion             string
}

TLSHelperConfig define params used to create a tls.Config

type Throttle 

type Throttle struct {
	// contains filtered or unexported fields
}

Throttle allows a limited number of workers to run at a time. It also provides a mechanism to wait for all workers to finish.

func NewThrottle 

func NewThrottle(max int) *Throttle

NewThrottle creates a new throttle with a max number of workers.

func (*Throttle) Done 

func (t *Throttle) Done()

Done should be called by workers when they finish working. It panics if there wasn't a corresponding Start call.

func (*Throttle) Start 

func (t *Throttle) Start()

Start should be called by workers before they start working. It blocks if there are already the maximum number of workers working.

func (*Throttle) Wait 

func (t *Throttle) Wait()

Wait waits until all workers have finished working.

type ValueTypeInfo 

type ValueTypeInfo int32
const (
	ValueUnknown ValueTypeInfo = iota // unknown type of value
	ValueEmpty                        // no UID and no value
	ValueUid                          // UID
	ValuePlain                        // plain old value without defined language tag
	// Value which is part of a multi-value posting list (like language).
	ValueMulti
)

Type of a data inside DirectedEdge, Posting or NQuad

func ValueType 

func ValueType(hasValue, hasLang, hasSpecialId bool) ValueTypeInfo

Helper function, to decide value type of DirectedEdge/Posting/NQuad

type WaterMark 

type WaterMark struct {
	Name string
	// contains filtered or unexported fields
}

WaterMark is used to keep track of the minimum un-finished index. Typically, an index k becomes finished or "done" according to a WaterMark once Done(k) has been called

  1. as many times as Begin(k) has, AND
  2. a positive number of times.

An index may also become "done" by calling SetDoneUntil at a time such that it is not inter-mingled with Begin/Done calls.

func (*WaterMark) Begin 

func (w *WaterMark) Begin(index uint64)

func (*WaterMark) BeginMany 

func (w *WaterMark) BeginMany(indices []uint64)

func (*WaterMark) Done 

func (w *WaterMark) Done(index uint64)

func (*WaterMark) DoneMany 

func (w *WaterMark) DoneMany(indices []uint64)

func (*WaterMark) DoneUntil 

func (w *WaterMark) DoneUntil() uint64

DoneUntil returns the maximum index until which all tasks are done.

func (*WaterMark) Init 

func (w *WaterMark) Init()

Init initializes a WaterMark struct. MUST be called before using it.

func (*WaterMark) SetDoneUntil 

func (w *WaterMark) SetDoneUntil(val uint64)

func (*WaterMark) WaitForMark 

func (w *WaterMark) WaitForMark(ctx context.Context, index uint64) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.