x

package
v0.8.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2017 License: AGPL-3.0, Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *
  • Licensed under the Apache License, Version 2.0 (the "License");
  • you may not use this file except in compliance with the License.
  • You may obtain a copy of the License at *
  • http://www.apache.org/licenses/LICENSE-2.0 *
  • Unless required by applicable law or agreed to in writing, software
  • distributed under the License is distributed on an "AS IS" BASIS,
  • WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  • See the License for the specific language governing permissions and
  • limitations under the License.

Package x contains some very common utilities used by Dgraph. These utilities are of "miscellaneous" nature, e.g., error checking.

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *

  • This program is free software: you can redistribute it and/or modify

  • it under the terms of the GNU Affero General Public License as published by

  • the Free Software Foundation, either version 3 of the License, or

  • (at your option) any later version. *

  • This program is distributed in the hope that it will be useful,

  • but WITHOUT ANY WARRANTY; without even the implied warranty of

  • MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the

  • GNU Affero General Public License for more details. *

  • You should have received a copy of the GNU Affero General Public License

  • along with this program. If not, see <http://www.gnu.org/licenses/>.

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *

  • Licensed under the Apache License, Version 2.0 (the "License");

  • you may not use this file except in compliance with the License.

  • You may obtain a copy of the License at *

  • http://www.apache.org/licenses/LICENSE-2.0 *

  • Unless required by applicable law or agreed to in writing, software

  • distributed under the License is distributed on an "AS IS" BASIS,

  • WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  • See the License for the specific language governing permissions and

  • limitations under the License.

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *

  • Licensed under the Apache License, Version 2.0 (the "License");

  • you may not use this file except in compliance with the License.

  • You may obtain a copy of the License at *

  • http://www.apache.org/licenses/LICENSE-2.0 *

  • Unless required by applicable law or agreed to in writing, software

  • distributed under the License is distributed on an "AS IS" BASIS,

  • WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  • See the License for the specific language governing permissions and

  • limitations under the License.

Index

Constants

View Source 
const (
	// TODO(pawan) - Make this 2 bytes long. Right now ParsedKey has byteType and
	// bytePrefix. Change it so that it just has one field which has all the information.
	ByteData = byte(0x00)

	ByteIndex    = byte(0x02)
	ByteReverse  = byte(0x04)
	ByteCount    = byte(0x08)
	ByteCountRev = ByteCount | ByteReverse
)
View Source 
const (
	TLSClientConfig tlsConfigType = iota
	TLSServerConfig
)
View Source 
const (
	Success                 = "Success"
	ErrorUnauthorized       = "ErrorUnauthorized"
	ErrorInvalidMethod      = "ErrorInvalidMethod"
	ErrorInvalidRequest     = "ErrorInvalidRequest"
	ErrorMissingRequired    = "ErrorMissingRequired"
	Error                   = "Error"
	ErrorNoData             = "ErrorNoData"
	ErrorUptodate           = "ErrorUptodate"
	ErrorNoPermission       = "ErrorNoPermission"
	ErrorInvalidMutation    = "ErrorInvalidMutation"
	ErrorServiceUnavailable = "ErrorServiceUnavailable"
	ValidHostnameRegex      = "^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\\-]*[A-Za-z0-9])$"
	Star                    = "_STAR_ALL"
	GrpcMaxSize             = 256 << 20
)

Error constants representing different types of errors.

Variables

View Source 
var (
	// These are cummulative
	PostingReads  *expvar.Int
	PostingWrites *expvar.Int
	BytesRead     *expvar.Int
	BytesWrite    *expvar.Int
	EvictedPls    *expvar.Int
	NumQueries    *expvar.Int
	CacheHit      *expvar.Int
	CacheMiss     *expvar.Int
	CacheRace     *expvar.Int

	// value at particular point of time
	PendingQueries   *expvar.Int
	PendingProposals *expvar.Int
	LcacheSize       *expvar.Int
	LcacheLen        *expvar.Int
	LcacheCapacity   *expvar.Int
	DirtyMapSize     *expvar.Int
	NumGoRoutines    *expvar.Int
	MemoryInUse      *expvar.Int
	HeapIdle         *expvar.Int
	TotalMemory      *expvar.Int
	TotalOSMemory    *expvar.Int
	ActiveMutations  *expvar.Int
	ServerHealth     *expvar.Int
	MaxPlSize        *expvar.Int
	MaxPlLength      *expvar.Int

	PredicateStats *expvar.Map
	Conf           *expvar.Map

	MaxPlSz int64
)
View Source 
var (
	Logger = log.New(os.Stderr, "", log.Lshortfile|log.Flags())
)
View Source 
var Nilbyte []byte
View Source 
var WhiteSpace = strings.NewReplacer(" ", "", "\t", "")

WhiteSpace Replacer removes spaces and tabs from a string.

Functions

func AddCorsHeaders added in v0.8.3 

func AddCorsHeaders(w http.ResponseWriter)

func AddInit added in v0.4.4 

func AddInit(f func())

AddInit adds a function to be run in x.Init, which should be called at the beginning of all mains.

func AppendProtoMsg added in v0.8.3 

func AppendProtoMsg(p []byte, msg ProtoMessage) ([]byte, error)

func AppendUvarint added in v0.8.3 

func AppendUvarint(p []byte, x uint64) []byte

func AssertTrue added in v0.7.0 

func AssertTrue(b bool)

AssertTrue asserts that b is true. Otherwise, it would log fatal.

func AssertTruef added in v0.7.0 

func AssertTruef(b bool, format string, args ...interface{})

AssertTruef is AssertTrue with extra info.

func AssertTruefNoTrace added in v0.8.2 

func AssertTruefNoTrace(b bool, format string, args ...interface{})

func Check added in v0.7.0 

func Check(err error)

Check logs fatal if err != nil.

func Check2 added in v0.7.0 

func Check2(_ interface{}, err error)

Check2 acts as convenience wrapper around Check, using the 2nd argument as error.

func Checkf added in v0.7.0 

func Checkf(err error, format string, args ...interface{})

Checkf is Check with extra info.

func CountKey added in v0.8.2 

func CountKey(attr string, count uint32, reverse bool) []byte

func DataKey added in v0.7.0 

func DataKey(attr string, uid uint64) []byte

func Errorf added in v0.7.0 

func Errorf(format string, args ...interface{}) error

Errorf creates a new error with stack trace, etc.

func Fatalf added in v0.7.0 

func Fatalf(format string, args ...interface{})

Fatalf logs fatal.

func GenerateTLSConfig added in v0.8.2 

func GenerateTLSConfig(config TLSHelperConfig) (tlsCfg *tls.Config, reloadConfig func(), err error)

GenerateTLSConfig creates and returns a new *tls.Config with the configuration provided. If the ConfigType provided in TLSHelperConfig is TLSServerConfig, it's return a reload function. If any problem is found, an error is returned

func HealthCheck added in v0.8.2 

func HealthCheck() error

HealthCheck returns whether the server is ready to accept requests or not Load balancer would add the node to the endpoint once health check starts returning true

func IndexKey added in v0.7.0 

func IndexKey(attr, term string) []byte

func Init added in v0.4.4 

func Init()

Init initializes flags and run all functions in initFunc.

func IsTestRun added in v0.7.2 

func IsTestRun() bool

func LoadConfigFromYAML added in v0.8.2 

func LoadConfigFromYAML(file string)

loadConfigFromYAML reads configurations from specified YAML file.

func NewTrace added in v0.8.2 

func NewTrace(title string, ctx context.Context) (trace.Trace, context.Context)

func PageRange added in v0.8.2 

func PageRange(count, offset, n int) (int, int)

PageRange returns start and end indices given pagination params. Note that n is the size of the input list.

func ParseRequest 

func ParseRequest(w http.ResponseWriter, r *http.Request, data interface{}) bool

func PredicatePrefix added in v0.8.3 

func PredicatePrefix(predicate string) []byte

PredicatePrefix returns the prefix for all keys belonging to this predicate except schema key.

func PrintVersionOnly added in v0.4.3 

func PrintVersionOnly()

PrintVersionOnly prints version and other helpful information if --version.

func Printf added in v0.7.0 

func Printf(format string, args ...interface{})

Printf does a log.Printf. We often do printf for debugging but has to keep adding import "fmt" or "log" and removing them after we are done. Let's add Printf to "x" and include "x" almost everywhere. Caution: Do remember to call x.Init. For tests, you need a TestMain that calls x.Init.

func Println added in v0.8.2 

func Println(args ...interface{})

func ReadLine added in v0.7.0 

func ReadLine(r *bufio.Reader, buf *bytes.Buffer) error

Reads a single line from a buffered reader. The line is read into the passed in buffer to minimize allocations. This is the preferred method for loading long lines which could be longer than the buffer size of bufio.Scanner.

func RemoveDuplicates added in v0.8.2 

func RemoveDuplicates(s []string) (out []string)

sorts the slice of strings and removes duplicates. changes the input slice. this function should be called like: someSlice = x.RemoveDuplicates(someSlice)

func Reply 

func Reply(w http.ResponseWriter, rep interface{})

func ReserveCap added in v0.8.3 

func ReserveCap(p []byte, atLeast int) []byte

func ReverseKey added in v0.7.1 

func ReverseKey(attr string, uid uint64) []byte

func Round added in v0.7.3 

func Round(d time.Duration) time.Duration

Go doesn't have a round function for Duration and the duration value can be a bit ugly to look at sometimes. This is an attempt to round the value.

func SchemaKey added in v0.7.3 

func SchemaKey(attr string) []byte

SchemaKey returns schema key for given attribute, schema keys are stored separately with unique prefix, since we need to iterate over all schema keys

func SchemaPrefix added in v0.7.3 

func SchemaPrefix() []byte

SchemaPrefix returns the prefix for Schema keys.

func SetError 

func SetError(prev *error, n error)

SetError sets the error logged in this package.

func SetStatus 

func SetStatus(w http.ResponseWriter, code, msg string)

SetStatus sets the error code, message and the newly assigned uids in the http response.

func SetStatusWithData added in v0.8.2 

func SetStatusWithData(w http.ResponseWriter, code, msg string)

In case an error was encountered after the query execution started, we have to return data key with null value according to GraphQL spec.

func SetTestRun added in v0.7.2 

func SetTestRun()

func UpdateHealthStatus added in v0.8.2 

func UpdateHealthStatus(ok bool)

func UpdateMemoryStatus added in v0.8.2 

func UpdateMemoryStatus(ok bool)

func ValidateAddress added in v0.8.2 

func ValidateAddress(addr string) bool

ValidateAddress checks whether given address can be used with grpc dial function

func Version added in v0.8.2 

func Version() string

func Wrap added in v0.7.0 

func Wrap(err error) error

Wrap wraps errors from external lib.

func Wrapf added in v0.7.0 

func Wrapf(err error, format string, args ...interface{}) error

Wrapf is Wrap with extra info.

func WriteFileSync added in v0.8.2 

func WriteFileSync(filename string, data []byte, perm os.FileMode) error

WriteFileSync is the same as bufio.WriteFile, but syncs the data before closing.

Types

type BytesBuffer added in v0.8.2 

type BytesBuffer struct {
	// contains filtered or unexported fields
}

func (*BytesBuffer) CopyTo added in v0.8.2 

func (b *BytesBuffer) CopyTo(o []byte) int

Caller should ensure that o is of appropriate length

func (*BytesBuffer) Length added in v0.8.2 

func (b *BytesBuffer) Length() int

func (*BytesBuffer) Slice added in v0.8.2 

func (b *BytesBuffer) Slice(n int) []byte

returns a slice of lenght n to be used to writing

func (*BytesBuffer) TruncateBy added in v0.8.2 

func (b *BytesBuffer) TruncateBy(n int)

Always give back <= touched bytes

type EmbeddedUidAllocator added in v0.8.3 

type EmbeddedUidAllocator struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func (*EmbeddedUidAllocator) AssignUids added in v0.8.3 

func (e *EmbeddedUidAllocator) AssignUids(ctx context.Context,
	num *protos.Num) (*protos.AssignedIds, error)

func (*EmbeddedUidAllocator) Init added in v0.8.3 

func (e *EmbeddedUidAllocator) Init(kv *badger.KV)

Start lease from 2, 1 is used by _lease_

type Histogram added in v0.8.2 

type Histogram struct {
	sync.Mutex
	// contains filtered or unexported fields
}

A Histogram collects observed values by keeping bucketed counts. For convenience, internally two sets of buckets are kept: A cumulative set (i.e. data is never evicted) and a windowed set (which keeps only recently collected samples).

Top-level methods generally apply to the cumulative buckets; the windowed variant is exposed through the Windowed method.

func NewHistogram added in v0.8.2 

func NewHistogram(duration time.Duration, maxVal int64, sigFigs int) *Histogram

NewHistogram initializes a given Histogram. The contained windowed histogram rotates every 'duration'; both the windowed and the cumulative histogram track nonnegative values up to 'maxVal' with 'sigFigs' decimal points of precision.

func (*Histogram) RecordValue added in v0.8.2 

func (h *Histogram) RecordValue(v int64)

RecordValue adds the given value to the histogram. Recording a value in excess of the configured maximum value for that histogram results in recording the maximum value instead.

func (*Histogram) Stats added in v0.8.2 

func (h *Histogram) Stats()

type NonceCounter added in v0.8.2 

type NonceCounter struct {
	// contains filtered or unexported fields
}

NonceCounter can generate unique 128-bit numbers. Only usable on a single thread. (All it would take to be multithreaded is to increment hi atomically.)

func NewNonceCounter added in v0.8.2 

func NewNonceCounter() NonceCounter

NewNonceCounter reads from /dev/urandom (or such) and generates a new randomized NonceCounter.

func (*NonceCounter) Generate added in v0.8.2 

func (nc *NonceCounter) Generate() [16]byte

Generate returns a new nonce value.

func (*NonceCounter) GenerationNumber added in v0.8.2 

func (nc *NonceCounter) GenerationNumber(buf []byte) uint64

GenerationNumber returns N+C when buf was produced by the Nth call to Generate(), for some global constant C.

type Options added in v0.8.2 

type Options struct {
	ConfigFile string
	Version    bool
	DebugMode  bool
	PortOffset int
}
var Config Options

type ParsedKey added in v0.7.0 

type ParsedKey struct {
	Attr  string
	Uid   uint64
	Term  string
	Count uint32
	// contains filtered or unexported fields
}

func Parse added in v0.7.0 

func Parse(key []byte) *ParsedKey

func (ParsedKey) CountPrefix added in v0.8.2 

func (p ParsedKey) CountPrefix(reverse bool) []byte

CountPrefix returns the prefix for count keys.

func (ParsedKey) DataPrefix added in v0.7.2 

func (p ParsedKey) DataPrefix() []byte

DataPrefix returns the prefix for data keys.

func (ParsedKey) IndexPrefix added in v0.7.0 

func (p ParsedKey) IndexPrefix() []byte

IndexPrefix returns the prefix for index keys.

func (ParsedKey) IsCount added in v0.8.2 

func (p ParsedKey) IsCount() bool

func (ParsedKey) IsData added in v0.7.0 

func (p ParsedKey) IsData() bool

func (ParsedKey) IsIndex added in v0.7.0 

func (p ParsedKey) IsIndex() bool

func (ParsedKey) IsReverse added in v0.7.1 

func (p ParsedKey) IsReverse() bool

func (ParsedKey) IsSchema added in v0.7.3 

func (p ParsedKey) IsSchema() bool

func (ParsedKey) IsType added in v0.8.2 

func (p ParsedKey) IsType(typ byte) bool

func (ParsedKey) ReversePrefix added in v0.8.2 

func (p ParsedKey) ReversePrefix() []byte

ReversePrefix returns the prefix for index keys.

func (ParsedKey) SkipPredicate added in v0.7.0 

func (p ParsedKey) SkipPredicate() []byte

func (ParsedKey) SkipRangeOfSameType added in v0.7.0 

func (p ParsedKey) SkipRangeOfSameType() []byte

func (ParsedKey) SkipSchema added in v0.7.3 

func (p ParsedKey) SkipSchema() []byte

type ProtoMessage added in v0.8.3 

type ProtoMessage interface {
	Size() int
	MarshalTo([]byte) (int, error)
}

type QueryResWithData added in v0.8.2 

type QueryResWithData struct {
	Errors []errRes `json:"errors"`
	Data   *string  `json:"data"`
}

type RaftValue added in v0.7.1 

type RaftValue struct {
	Group uint32
	Index uint64
}

RaftValue contains the raft group and the raft proposal id. This is attached to the context, so the information could be passed down to the many posting lists, involved in mutations.

type SafeMutex added in v0.7.0 

type SafeMutex struct {
	// contains filtered or unexported fields
}

SafeLock can be used in place of sync.RWMutex

func (*SafeMutex) AssertLock added in v0.7.0 

func (s *SafeMutex) AssertLock()

func (*SafeMutex) AssertRLock added in v0.7.0 

func (s *SafeMutex) AssertRLock()

func (*SafeMutex) Lock added in v0.7.0 

func (s *SafeMutex) Lock()

func (*SafeMutex) RLock added in v0.7.0 

func (s *SafeMutex) RLock()

func (*SafeMutex) RUnlock added in v0.7.0 

func (s *SafeMutex) RUnlock()

func (*SafeMutex) StartWait added in v0.7.0 

func (s *SafeMutex) StartWait() *SafeWait

func (*SafeMutex) Unlock added in v0.7.0 

func (s *SafeMutex) Unlock()

func (*SafeMutex) Wait added in v0.7.0 

func (s *SafeMutex) Wait()

type SafeWait added in v0.7.0 

type SafeWait struct {
	// contains filtered or unexported fields
}

func (*SafeWait) Done added in v0.7.0 

func (s *SafeWait) Done()

type TLSHelperConfig added in v0.8.2 

type TLSHelperConfig struct {
	ConfigType             tlsConfigType
	CertRequired           bool
	Cert                   string
	Key                    string
	KeyPassphrase          string
	ServerName             string
	Insecure               bool
	RootCACerts            string
	UseSystemRootCACerts   bool
	ClientAuth             string
	ClientCACerts          string
	UseSystemClientCACerts bool
	MinVersion             string
	MaxVersion             string
}

TLSHelperConfig define params used to create a tls.Config

type Throttle added in v0.8.2 

type Throttle struct {
	// contains filtered or unexported fields
}

Throttle allows a limited number of workers to run at a time. It also provides a mechanism to wait for all workers to finish.

func NewThrottle added in v0.8.2 

func NewThrottle(max int) *Throttle

NewThrottle creates a new throttle with a max number of workers.

func (*Throttle) Done added in v0.8.2 

func (t *Throttle) Done()

Done should be called by workers when they finish working. It panics if there wasn't a corresponding Start call.

func (*Throttle) Start added in v0.8.2 

func (t *Throttle) Start()

Start should be called by workers before they start working. It blocks if there are already the maximum number of workers working.

func (*Throttle) Wait added in v0.8.2 

func (t *Throttle) Wait()

Wait waits until all workers have finished working.

type ValueTypeInfo added in v0.8.2 

type ValueTypeInfo int32
const (
	ValueUnknown ValueTypeInfo = iota // unknown type of value
	ValueEmpty                        // no UID and no value
	ValueUid                          // UID
	ValuePlain                        // plain old value without defined language tag
	// Value which is part of a multi-value posting list (like language).
	ValueMulti
)

Type of a data inside DirectedEdge, Posting or NQuad

func ValueType added in v0.8.2 

func ValueType(hasValue, hasLang, hasSpecialId bool) ValueTypeInfo

Helper function, to decide value type of DirectedEdge/Posting/NQuad

type WaterMark added in v0.7.1 

type WaterMark struct {
	Name string
	// contains filtered or unexported fields
}

WaterMark is used to keep track of the minimum un-finished index. Typically, an index k becomes finished or "done" according to a WaterMark once Done(k) has been called

  1. as many times as Begin(k) has, AND
  2. a positive number of times.

An index may also become "done" by calling SetDoneUntil at a time such that it is not inter-mingled with Begin/Done calls.

func (*WaterMark) Begin added in v0.8.2 

func (w *WaterMark) Begin(index uint64)

func (*WaterMark) BeginMany added in v0.8.2 

func (w *WaterMark) BeginMany(indices []uint64)

func (*WaterMark) Done added in v0.8.2 

func (w *WaterMark) Done(index uint64)

func (*WaterMark) DoneMany added in v0.8.2 

func (w *WaterMark) DoneMany(indices []uint64)

func (*WaterMark) DoneUntil added in v0.7.1 

func (w *WaterMark) DoneUntil() uint64

DoneUntil returns the maximum index until which all tasks are done.

func (*WaterMark) Init added in v0.7.1 

func (w *WaterMark) Init()

Init initializes a WaterMark struct. MUST be called before using it.

func (*WaterMark) SetDoneUntil added in v0.8.2 

func (w *WaterMark) SetDoneUntil(val uint64)

func (*WaterMark) WaitForMark added in v0.8.2 

func (w *WaterMark) WaitForMark(ctx context.Context, index uint64) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.