pilosa

package module
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 20, 2018 License: Apache-2.0 Imports: 43 Imported by: 0

README

CircleCI GoDoc Go Report Card license CLA Assistant GitHub release

An open source, distributed bitmap index.

Want to contribute? One of the easiest ways is to tell us how you're using (or want to use) Pilosa. We learn from every discussion!

Docs

See our Documentation for information about installing and working with Pilosa.

Getting Started

  1. Install Pilosa.

  2. Start Pilosa with the default configuration:

    pilosa server

    and verify that it's running:

    curl localhost:10101/nodes

  3. Follow along with the Sample Project to get a better understanding of Pilosa's capabilities.

Data Model

Check out how the Pilosa Data Model works.

Query Language

You can interact with Pilosa directly in the console using the Pilosa Query Language (PQL).

Client Libraries

There are supported libraries for the following languages:

Licenses

The core Pilosa code base and all default builds (referred to as Pilosa Community Edition) are licensed completely under the Apache License, Version 2.0. If you build Pilosa with the enterprise build tag (Pilosa Enterprise Edition), then that build will include features licensed under the GNU Affero General Public License (AGPL). Enterprise code is located entirely in the github.com/pilosa/pilosa/enterprise directory. See github.com/pilosa/pilosa/NOTICE and github.com/pilosa/pilosa/LICENSE for more information about Pilosa licenses.

Get Support

There are several channels available for you to reach out to us for support.

Contributing

Pilosa is an open source project. Please see our Contributing Guide for information about how to get involved.

Documentation

Overview

Package pilosa implements the core of the Pilosa distributed bitmap index. It contains all the domain objects, interfaces, and logic that defines pilosa.

Index

Constants

View Source 
const (

	// ClusterState represents the state returned in the /status endpoint.
	ClusterStateStarting = "STARTING"
	ClusterStateDegraded = "DEGRADED" // cluster is running but we've lost some # of hosts >0 but < replicaN
	ClusterStateNormal   = "NORMAL"
	ClusterStateResizing = "RESIZING"
)
View Source 
const (
	DefaultFieldType = FieldTypeSet

	DefaultCacheType = CacheTypeRanked

	// Default ranked field cache
	DefaultCacheSize = 50000
)

Default field settings.

View Source 
const (
	FieldTypeSet   = "set"
	FieldTypeInt   = "int"
	FieldTypeTime  = "time"
	FieldTypeMutex = "mutex"
	FieldTypeBool  = "bool"
)

Field types.

View Source 
const (
	CacheTypeLRU    = "lru"
	CacheTypeRanked = "ranked"
	CacheTypeNone   = "none"
)

Cache types.

View Source 
const (
	ShardWidth = 1 << shardWidthExponent

	// HashBlockSize is the number of rows in a merkle hash block.
	HashBlockSize = 100
)
View Source 
const (
	LogEntryTypeInsertColumn = 1
	LogEntryTypeInsertRow    = 2
)
View Source 
const TimeFormat = "2006-01-02T15:04"

TimeFormat is the go-style time format used to parse string dates.

Variables

View Source 
var (
	ErrHostRequired = errors.New("host required")

	ErrIndexRequired = errors.New("index required")
	ErrIndexExists   = errors.New("index already exists")
	ErrIndexNotFound = errors.New("index not found")

	// ErrFieldRequired is returned when no field is specified.
	ErrFieldRequired = errors.New("field required")
	ErrFieldExists   = errors.New("field already exists")
	ErrFieldNotFound = errors.New("field not found")

	ErrBSIGroupNotFound         = errors.New("bsigroup not found")
	ErrBSIGroupExists           = errors.New("bsigroup already exists")
	ErrBSIGroupNameRequired     = errors.New("bsigroup name required")
	ErrInvalidBSIGroupType      = errors.New("invalid bsigroup type")
	ErrInvalidBSIGroupRange     = errors.New("invalid bsigroup range")
	ErrInvalidBSIGroupValueType = errors.New("invalid bsigroup value type")
	ErrBSIGroupValueTooLow      = errors.New("bsigroup value too low")
	ErrBSIGroupValueTooHigh     = errors.New("bsigroup value too high")
	ErrInvalidRangeOperation    = errors.New("invalid range operation")
	ErrInvalidBetweenValue      = errors.New("invalid value for between operation")

	ErrInvalidView      = errors.New("invalid view")
	ErrInvalidCacheType = errors.New("invalid cache type")

	ErrName  = errors.New("invalid index or field name, must match [a-z0-9_-]")
	ErrLabel = errors.New("invalid row or column label, must match [A-Za-z0-9_-]")

	// ErrFragmentNotFound is returned when a fragment does not exist.
	ErrFragmentNotFound = errors.New("fragment not found")
	ErrQueryRequired    = errors.New("query required")
	ErrQueryCancelled   = errors.New("query cancelled")
	ErrQueryTimeout     = errors.New("query timeout")
	ErrTooManyWrites    = errors.New("too many write commands")

	// TODO(2.0) poorly named - used when a *node* doesn't own a shard. Probably
	// we won't need this error at all by 2.0 though.
	ErrClusterDoesNotOwnShard = errors.New("node does not own shard")

	ErrNodeIDNotExists    = errors.New("node with provided ID does not exist")
	ErrNodeNotCoordinator = errors.New("node is not the coordinator")
	ErrResizeNotRunning   = errors.New("no resize job currently running")

	ErrNotImplemented            = errors.New("not implemented")
	ErrFieldsArgumentRequired    = errors.New("fields argument required")
	ErrExpectedFieldListArgument = errors.New("expected field list argument")
)

System errors.

View Source 
var (
	ErrTranslateStoreClosed          = errors.New("pilosa: translate store closed")
	ErrTranslateStoreReaderClosed    = errors.New("pilosa: translate store reader closed")
	ErrReplicationNotSupported       = errors.New("pilosa: replication not supported")
	ErrTranslateStoreReadOnly        = errors.New("pilosa: translate store could not find or create key, translate store read only")
	ErrTranslateReadTargetUndersized = errors.New("pilosa: translate read target is undersized")
)
View Source 
var BuildTime = "not recorded"
View Source 
var Enterprise = "0"
View Source 
var EnterpriseEnabled = false
View Source 
var ErrInvalidTimeQuantum = errors.New("invalid time quantum")

ErrInvalidTimeQuantum is returned when parsing a time quantum.

View Source 
var NopBroadcaster broadcaster = &nopBroadcaster{}

NopBroadcaster represents a Broadcaster that doesn't do anything.

View Source 
var Version = "v0.0.0"

Functions

func DecodeAttrs added in v0.9.0 

func DecodeAttrs(v []byte) (map[string]interface{}, error)

DecodeAttrs decodes a byte slice into an attribute map.

func EncodeAttrs added in v0.9.0 

func EncodeAttrs(attr map[string]interface{}) ([]byte, error)

EncodeAttrs encodes an attribute map into a byte slice.

func MarshalInternalMessage added in v1.0.0 

func MarshalInternalMessage(m Message, s Serializer) ([]byte, error)

MarshalInternalMessage serializes the pilosa message and adds pilosa internal type info which is used by the internal messaging stuff.

func NewRankCache 

func NewRankCache(maxEntries uint32) *rankCache

NewRankCache returns a new instance of RankCache.

func OptAPIServer added in v1.0.0 

func OptAPIServer(s *Server) apiOption

Types

type API added in v0.9.0 

type API struct {
	Serializer Serializer
	// contains filtered or unexported fields
}

API provides the top level programmatic interface to Pilosa. It is usually wrapped by a handler which provides an external interface (e.g. HTTP).

func NewAPI added in v0.9.0 

func NewAPI(opts ...apiOption) (*API, error)

NewAPI returns a new API instance.

func (*API) AvailableShardsByIndex added in v1.2.0 

func (api *API) AvailableShardsByIndex(ctx context.Context) map[string]*roaring.Bitmap

AvailableShardsByIndex returns bitmaps of shards with available by index name.

func (*API) ClusterMessage added in v0.9.0 

func (api *API) ClusterMessage(ctx context.Context, reqBody io.Reader) error

PostClusterMessage is for internal use. It decodes a protobuf message out of the body and forwards it to the BroadcastHandler.

func (*API) CreateField added in v0.9.0 

func (api *API) CreateField(ctx context.Context, indexName string, fieldName string, opts ...FieldOption) (*Field, error)

CreateField makes the named field in the named index with the given options. This method currently only takes a single functional option, but that may be changed in the future to support multiple options.

func (*API) CreateIndex added in v0.9.0 

func (api *API) CreateIndex(ctx context.Context, indexName string, options IndexOptions) (*Index, error)

CreateIndex makes a new Pilosa index.

func (*API) DeleteAvailableShard added in v1.2.0 

func (api *API) DeleteAvailableShard(_ context.Context, indexName, fieldName string, shardID uint64) error

DeleteAvailableShard a shard ID from the available shard set cache.

func (*API) DeleteField added in v0.9.0 

func (api *API) DeleteField(ctx context.Context, indexName string, fieldName string) error

DeleteField removes the named field from the named index. If the index is not found, an error is returned. If the field is not found, it is ignored and no action is taken.

func (*API) DeleteIndex added in v0.9.0 

func (api *API) DeleteIndex(ctx context.Context, indexName string) error

DeleteIndex removes the named index. If the index is not found it does nothing and returns no error.

func (*API) DeleteView added in v0.9.0 

func (api *API) DeleteView(ctx context.Context, indexName string, fieldName string, viewName string) error

DeleteView removes the given view.

func (*API) ExportCSV added in v0.9.0 

func (api *API) ExportCSV(ctx context.Context, indexName string, fieldName string, shard uint64, w io.Writer) error

ExportCSV encodes the fragment designated by the index,field,shard as CSV of the form <row>,<col>

func (*API) Field added in v1.0.0 

func (api *API) Field(ctx context.Context, indexName, fieldName string) (*Field, error)

Field retrieves the named field.

func (*API) FieldAttrDiff added in v1.0.0 

func (api *API) FieldAttrDiff(ctx context.Context, indexName string, fieldName string, blocks []AttrBlock) (map[uint64]map[string]interface{}, error)

func (*API) FragmentBlockData added in v0.9.0 

func (api *API) FragmentBlockData(ctx context.Context, body io.Reader) ([]byte, error)

FragmentBlockData is an endpoint for internal usage. It is not guaranteed to return anything useful. Currently it returns protobuf encoded row and column ids from a "block" which is a subdivision of a fragment.

func (*API) FragmentBlocks added in v0.9.0 

func (api *API) FragmentBlocks(ctx context.Context, indexName, fieldName, viewName string, shard uint64) ([]FragmentBlock, error)

FragmentBlocks returns the checksums and block ids for all blocks in the specified fragment.

func (*API) FragmentData added in v1.2.0 

func (api *API) FragmentData(ctx context.Context, indexName, fieldName, viewName string, shard uint64) (io.WriterTo, error)

FragmentData returns all data in the specified fragment.

func (*API) GetTranslateData added in v1.0.0 

func (api *API) GetTranslateData(ctx context.Context, offset int64) (io.ReadCloser, error)

GetTranslateData provides a reader for key translation logs starting at offset.

func (*API) Hosts added in v0.9.0 

func (api *API) Hosts(ctx context.Context) []*Node

Hosts returns a list of the hosts in the cluster including their ID, URL, and which is the coordinator.

func (*API) Import added in v0.9.0 

func (api *API) Import(ctx context.Context, req *ImportRequest, opts ...ImportOption) error

Import bulk imports data into a particular index,field,shard.

func (*API) ImportRoaring added in v1.2.0 

func (api *API) ImportRoaring(ctx context.Context, indexName, fieldName string, shard uint64, remote bool, req *ImportRoaringRequest) (err error)

ImportRoaring is a low level interface for importing data to Pilosa when extremely high throughput is desired. The data must be encoded in a particular way which may be unintuitive (discussed below). The data is merged with existing data.

It takes as input a roaring bitmap which it uses as the data for the indicated index, field, and shard. The bitmap may be encoded according to the official roaring spec (https://github.com/RoaringBitmap/RoaringFormatSpec), or to the pilosa roaring spec which supports 64 bit integers (https://www.pilosa.com/docs/latest/architecture/#roaring-bitmap-storage-format).

The data should be encoded the same way that Pilosa stores fragments internally. A bit "i" being set in the input bitmap indicates that the bit is set in Pilosa row "i/ShardWidth", and in column (shard*ShardWidth)+(i%ShardWidth). That is to say that "data" represents all of the rows in this shard of this field concatenated together in one long bitmap.

func (*API) ImportValue added in v0.9.0 

func (api *API) ImportValue(ctx context.Context, req *ImportValueRequest, opts ...ImportOption) error

ImportValue bulk imports values into a particular field.

func (*API) Index added in v0.9.0 

func (api *API) Index(ctx context.Context, indexName string) (*Index, error)

Index retrieves the named index.

func (*API) IndexAttrDiff added in v0.9.0 

func (api *API) IndexAttrDiff(ctx context.Context, indexName string, blocks []AttrBlock) (map[uint64]map[string]interface{}, error)

IndexAttrDiff

func (*API) Info added in v0.10.0 

func (api *API) Info() serverInfo

Info returns information about this server instance

func (*API) LongQueryTime added in v0.9.0 

func (api *API) LongQueryTime() time.Duration

LongQueryTime returns the configured threshold for logging/statting long running queries.

func (*API) MaxShards added in v1.0.0 

func (api *API) MaxShards(ctx context.Context) map[string]uint64

MaxShards returns the maximum shard number for each index in a map. TODO (2.0): This method has been deprecated. Instead, use AvailableShardsByIndex.

func (*API) Node added in v1.0.0 

func (api *API) Node() *Node

Node gets the ID, URI and coordinator status for this particular node.

func (*API) Query added in v0.9.0 

func (api *API) Query(ctx context.Context, req *QueryRequest) (QueryResponse, error)

Query parses a PQL query out of the request and executes it.

func (*API) RecalculateCaches added in v0.9.0 

func (api *API) RecalculateCaches(ctx context.Context) error

RecalculateCaches forces all TopN caches to be updated. Used mainly for integration tests.

func (*API) RemoveNode added in v0.9.0 

func (api *API) RemoveNode(id string) (*Node, error)

RemoveNode puts the cluster into the "RESIZING" state and begins the job of removing the given node.

func (*API) ResizeAbort added in v0.9.0 

func (api *API) ResizeAbort() error

ResizeAbort stops the current resize job.

func (*API) Schema added in v0.9.0 

func (api *API) Schema(ctx context.Context) []*IndexInfo

Schema returns information about each index in Pilosa including which fields they contain.

func (*API) SetCoordinator added in v0.9.0 

func (api *API) SetCoordinator(ctx context.Context, id string) (oldNode, newNode *Node, err error)

SetCoordinator makes a new Node the cluster coordinator.

func (*API) ShardNodes added in v1.0.0 

func (api *API) ShardNodes(ctx context.Context, indexName string, shard uint64) ([]*Node, error)

ShardNodes returns the node and all replicas which should contain a shard's data.

func (*API) State added in v0.9.0 

func (api *API) State() string

State returns the cluster state which is usually "NORMAL", but could be "STARTING", "RESIZING", or potentially others. See cluster.go for more details.

func (*API) StatsWithTags added in v0.9.0 

func (api *API) StatsWithTags(tags []string) stats.StatsClient

StatsWithTags returns an instance of whatever implementation of StatsClient pilosa is using with the given tags.

func (*API) TranslateKeys added in v1.2.0 

func (api *API) TranslateKeys(body io.Reader) ([]byte, error)

func (*API) Version added in v0.9.0 

func (api *API) Version() string

Version returns the Pilosa version.

func (*API) Views added in v0.9.0 

func (api *API) Views(ctx context.Context, indexName string, fieldName string) ([]*view, error)

Views returns the views in the given field.

type AttrBlock 

type AttrBlock struct {
	ID       uint64 `json:"id"`
	Checksum []byte `json:"checksum"`
}

AttrBlock represents a checksummed block of the attribute store.

type AttrStore 

type AttrStore interface {
	Path() string
	Open() error
	Close() error
	Attrs(id uint64) (m map[string]interface{}, err error)
	SetAttrs(id uint64, m map[string]interface{}) error
	SetBulkAttrs(m map[uint64]map[string]interface{}) error
	Blocks() ([]AttrBlock, error)
	BlockData(i uint64) (map[uint64]map[string]interface{}, error)
}

AttrStore represents an interface for handling row/column attributes.

type BadRequestError added in v0.9.0 

type BadRequestError struct {
	// contains filtered or unexported fields
}

BadRequestError wraps an error value to signify that a request could not be read, decoded, or parsed such that in an HTTP scenario, http.StatusBadRequest would be returned.

func NewBadRequestError added in v1.0.0 

func NewBadRequestError(err error) BadRequestError

NewBadRequestError returns err wrapped in a BadRequestError.

type Bit 

type Bit struct {
	RowID     uint64
	ColumnID  uint64
	RowKey    string
	ColumnKey string
	Timestamp int64
}

Bit represents the intersection of a row and a column. It can be specified by integer ids or string keys.

type BlockDataRequest added in v1.0.0 

type BlockDataRequest struct {
	Index string
	Field string
	View  string
	Shard uint64
	Block uint64
}

type BlockDataResponse added in v1.0.0 

type BlockDataResponse struct {
	RowIDs    []uint64
	ColumnIDs []uint64
}

type ClusterStatus added in v1.0.0 

type ClusterStatus struct {
	ClusterID string
	State     string
	Nodes     []*Node
}

type CmdIO 

type CmdIO struct {
	Stdin  io.Reader
	Stdout io.Writer
	Stderr io.Writer
}

CmdIO holds standard unix inputs and outputs.

func NewCmdIO 

func NewCmdIO(stdin io.Reader, stdout, stderr io.Writer) *CmdIO

NewCmdIO returns a new instance of CmdIO with inputs and outputs set to the arguments.

type ColumnAttrSet 

type ColumnAttrSet struct {
	ID    uint64                 `json:"id,omitempty"`
	Key   string                 `json:"key,omitempty"`
	Attrs map[string]interface{} `json:"attrs,omitempty"`
}

ColumnAttrSet represents a set of attributes for a vertical column in an index. Can have a set of attributes attached to it.

type ConflictError added in v1.0.0 

type ConflictError struct {
	// contains filtered or unexported fields
}

ConflictError wraps an error value to signify that a conflict with an existing resource occurred such that in an HTTP scenario, http.StatusConflict would be returned.

type CreateFieldMessage added in v1.0.0 

type CreateFieldMessage struct {
	Index string
	Field string
	Meta  *FieldOptions
}

type CreateIndexMessage added in v1.0.0 

type CreateIndexMessage struct {
	Index string
	Meta  *IndexOptions
}

type CreateShardMessage added in v1.0.0 

type CreateShardMessage struct {
	Index string
	Field string
	Shard uint64
}

type CreateViewMessage added in v1.0.0 

type CreateViewMessage struct {
	Index string
	Field string
	View  string
}

type DeleteAvailableShardMessage added in v1.2.0 

type DeleteAvailableShardMessage struct {
	Index   string
	Field   string
	ShardID uint64
}

type DeleteFieldMessage added in v1.0.0 

type DeleteFieldMessage struct {
	Index string
	Field string
}

type DeleteIndexMessage added in v1.0.0 

type DeleteIndexMessage struct {
	Index string
}

type DeleteViewMessage added in v1.0.0 

type DeleteViewMessage struct {
	Index string
	Field string
	View  string
}

type Field added in v0.5.0 

type Field struct {
	Stats stats.StatsClient
	// contains filtered or unexported fields
}

Field represents a container for views.

func NewField added in v1.0.0 

func NewField(path, index, name string, opts FieldOption) (*Field, error)

NewField returns a new instance of field.

func (*Field) AddRemoteAvailableShards added in v1.2.0 

func (f *Field) AddRemoteAvailableShards(b *roaring.Bitmap) error

AddRemoteAvailableShards merges the set of available shards into the current known set and saves the set to a file.

func (*Field) AvailableShards added in v1.2.0 

func (f *Field) AvailableShards() *roaring.Bitmap

AvailableShards returns a bitmap of shards that contain data.

func (*Field) CacheSize added in v1.0.0 

func (f *Field) CacheSize() uint32

CacheSize returns the ranked field cache size.

func (*Field) ClearBit added in v1.0.0 

func (f *Field) ClearBit(rowID, colID uint64) (changed bool, err error)

ClearBit clears a bit within the field.

func (*Field) Close added in v1.0.0 

func (f *Field) Close() error

Close closes the field and its views.

func (*Field) Import added in v1.0.0 

func (f *Field) Import(rowIDs, columnIDs []uint64, timestamps []*time.Time, opts ...ImportOption) error

Import bulk imports data.

func (*Field) Index added in v1.0.0 

func (f *Field) Index() string

Index returns the index name the field was initialized with.

func (*Field) Max added in v0.5.0 

func (f *Field) Max(filter *Row, name string) (max, count int64, err error)

Max returns the max for a field. An optional filtering row can be provided.

func (*Field) Min added in v0.5.0 

func (f *Field) Min(filter *Row, name string) (min, count int64, err error)

Min returns the min for a field. An optional filtering row can be provided.

func (*Field) Name added in v0.5.0 

func (f *Field) Name() string

Name returns the name the field was initialized with.

func (*Field) Open added in v1.0.0 

func (f *Field) Open() error

Open opens and initializes the field.

func (*Field) Options added in v1.0.0 

func (f *Field) Options() FieldOptions

Options returns all options for this field.

func (*Field) Path added in v1.0.0 

func (f *Field) Path() string

Path returns the path the field was initialized with.

func (*Field) Range added in v1.0.0 

func (f *Field) Range(name string, op pql.Token, predicate int64) (*Row, error)

func (*Field) RemoveAvailableShard added in v1.2.0 

func (f *Field) RemoveAvailableShard(v uint64) error

RemoveAvailableShard removes a shard from the bitmap cache.

NOTE: This can be overridden on the next sync so all nodes should be updated.

func (*Field) Row added in v1.0.0 

func (f *Field) Row(rowID uint64) (*Row, error)

Row returns a row of the standard view. It seems this method is only being used by the test package, and the fact that it's only allowed on `set` fields is odd. This may be considered for deprecation in a future version.

func (*Field) RowAttrStore added in v1.0.0 

func (f *Field) RowAttrStore() AttrStore

RowAttrStore returns the attribute storage.

func (*Field) RowTime added in v1.0.0 

func (f *Field) RowTime(rowID uint64, time time.Time, quantum string) (*Row, error)

RowTime gets the row at the particular time with the granularity specified by the quantum.

func (*Field) SetBit added in v1.0.0 

func (f *Field) SetBit(rowID, colID uint64, t *time.Time) (changed bool, err error)

SetBit sets a bit on a view within the field.

func (*Field) SetCacheSize added in v1.0.0 

func (f *Field) SetCacheSize(v uint32) error

SetCacheSize sets the cache size for ranked fames. Persists to meta file on update. defaults to DefaultCacheSize 50000

func (*Field) SetValue added in v1.0.0 

func (f *Field) SetValue(columnID uint64, value int64) (changed bool, err error)

SetValue sets a field value for a column.

func (*Field) Sum added in v1.0.0 

func (f *Field) Sum(filter *Row, name string) (sum, count int64, err error)

Sum returns the sum and count for a field. An optional filtering row can be provided.

func (*Field) TimeQuantum added in v1.0.0 

func (f *Field) TimeQuantum() TimeQuantum

TimeQuantum returns the time quantum for the field.

func (*Field) Type added in v0.5.0 

func (f *Field) Type() string

Type returns the field type.

func (*Field) Value added in v1.0.0 

func (f *Field) Value(columnID uint64) (value int64, exists bool, err error)

Value reads a field value for a column.

type FieldInfo added in v1.0.0 

type FieldInfo struct {
	Name    string       `json:"name"`
	Options FieldOptions `json:"options"`
	Views   []*ViewInfo  `json:"views,omitempty"`
}

FieldInfo represents schema information for a field.

type FieldOption added in v1.0.0 

type FieldOption func(fo *FieldOptions) error

FieldOption is a functional option type for pilosa.fieldOptions.

func OptFieldKeys added in v1.0.0 

func OptFieldKeys() FieldOption

func OptFieldTypeBool added in v1.2.0 

func OptFieldTypeBool() FieldOption

func OptFieldTypeDefault added in v1.0.0 

func OptFieldTypeDefault() FieldOption

func OptFieldTypeInt added in v1.0.0 

func OptFieldTypeInt(min, max int64) FieldOption

func OptFieldTypeMutex added in v1.1.0 

func OptFieldTypeMutex(cacheType string, cacheSize uint32) FieldOption

func OptFieldTypeSet added in v1.0.0 

func OptFieldTypeSet(cacheType string, cacheSize uint32) FieldOption

func OptFieldTypeTime added in v1.0.0 

func OptFieldTypeTime(timeQuantum TimeQuantum, opt ...bool) FieldOption

OptFieldTypeTime sets the field type to time. Pass true to skip creation of the standard view.

type FieldOptions added in v1.0.0 

type FieldOptions struct {
	Min            int64       `json:"min,omitempty"`
	Max            int64       `json:"max,omitempty"`
	Keys           bool        `json:"keys"`
	NoStandardView bool        `json:"noStandardView,omitempty"`
	CacheSize      uint32      `json:"cacheSize,omitempty"`
	CacheType      string      `json:"cacheType,omitempty"`
	Type           string      `json:"type,omitempty"`
	TimeQuantum    TimeQuantum `json:"timeQuantum,omitempty"`
}

FieldOptions represents options to set when initializing a field.

func (*FieldOptions) MarshalJSON added in v1.0.0 

func (o *FieldOptions) MarshalJSON() ([]byte, error)

type FieldRow added in v1.2.0 

type FieldRow struct {
	Field  string `json:"field"`
	RowID  uint64 `json:"rowID"`
	RowKey string `json:"rowKey,omitempty"`
}

FieldRow is used to distinguish rows in a group by result.

func (FieldRow) MarshalJSON added in v1.2.0 

func (fr FieldRow) MarshalJSON() ([]byte, error)

func (FieldRow) String added in v1.2.0 

func (fr FieldRow) String() string

type FieldStatus added in v1.2.0 

type FieldStatus struct {
	Name            string
	AvailableShards *roaring.Bitmap
}

type FieldValue added in v0.7.0 

type FieldValue struct {
	ColumnID  uint64
	ColumnKey string
	Value     int64
}

FieldValue represents the value for a column within a range-encoded field.

type FragmentBlock 

type FragmentBlock struct {
	ID       int    `json:"id"`
	Checksum []byte `json:"checksum"`
}

FragmentBlock represents info about a subsection of the rows in a block. This is used for comparing data in remote blocks for active anti-entropy.

type GCNotifier added in v0.9.0 

type GCNotifier interface {
	Close()
	AfterGC() <-chan struct{}
}

GCNotifier represents an interface for garbage collection notificationss. 

var NopGCNotifier GCNotifier = &nopGCNotifier{}

NopGCNotifier represents a GCNotifier that doesn't do anything.

type GroupCount added in v1.2.0 

type GroupCount struct {
	Group []FieldRow `json:"group"`
	Count uint64     `json:"count"`
}

func (GroupCount) Compare added in v1.2.0 

func (g GroupCount) Compare(o GroupCount) int

type Handler 

type Handler interface {
	Serve() error
	Close() error
}
var NopHandler Handler = nopHandler{}

type Hasher 

type Hasher interface {
	// Hashes the key into a number between [0,N).
	Hash(key uint64, n int) int
}

Hasher represents an interface to hash integers into buckets.

type Holder 

type Holder struct {
	NewPrimaryTranslateStore func(interface{}) TranslateStore

	NewAttrStore func(string) AttrStore

	// Stats
	Stats stats.StatsClient

	// Data directory path.
	Path string

	Logger logger.Logger
	// contains filtered or unexported fields
}

Holder represents a container for indexes.

func NewHolder 

func NewHolder() *Holder

NewHolder returns a new instance of Holder.

func (*Holder) Close 

func (h *Holder) Close() error

Close closes all open fragments.

func (*Holder) CreateIndex 

func (h *Holder) CreateIndex(name string, opt IndexOptions) (*Index, error)

CreateIndex creates an index. An error is returned if the index already exists.

func (*Holder) CreateIndexIfNotExists 

func (h *Holder) CreateIndexIfNotExists(name string, opt IndexOptions) (*Index, error)

CreateIndexIfNotExists returns an index by name. The index is created if it does not already exist.

func (*Holder) DeleteIndex 

func (h *Holder) DeleteIndex(name string) error

DeleteIndex removes an index from the holder.

func (*Holder) Field added in v1.0.0 

func (h *Holder) Field(index, name string) *Field

Field returns the field for an index and name.

func (*Holder) HasData added in v0.9.0 

func (h *Holder) HasData() (bool, error)

HasData returns true if Holder contains at least one index. This is used to determine if the rebalancing of data is necessary when a node joins the cluster.

func (*Holder) Index 

func (h *Holder) Index(name string) *Index

Index returns the index by name.

func (*Holder) IndexPath 

func (h *Holder) IndexPath(name string) string

IndexPath returns the path where a given index is stored.

func (*Holder) Indexes 

func (h *Holder) Indexes() []*Index

Indexes returns a list of all indexes in the holder.

func (*Holder) Open 

func (h *Holder) Open() error

Open initializes the root data directory for the holder.

func (*Holder) Schema 

func (h *Holder) Schema() []*IndexInfo

Schema returns schema information for all indexes, fields, and views.

type ImportOption added in v1.2.0 

type ImportOption func(*ImportOptions) error

ImportOption is a functional option type for API.Import.

func OptImportOptionsClear added in v1.2.0 

func OptImportOptionsClear(c bool) ImportOption

func OptImportOptionsIgnoreKeyCheck added in v1.2.0 

func OptImportOptionsIgnoreKeyCheck(b bool) ImportOption

type ImportOptions added in v1.2.0 

type ImportOptions struct {
	Clear          bool
	IgnoreKeyCheck bool
}

ImportOptions holds the options for the API.Import method.

type ImportRequest added in v1.0.0 

type ImportRequest struct {
	Index      string
	Field      string
	Shard      uint64
	RowIDs     []uint64
	ColumnIDs  []uint64
	RowKeys    []string
	ColumnKeys []string
	Timestamps []int64
}

type ImportResponse added in v1.0.0 

type ImportResponse struct {
	Err string
}

type ImportRoaringRequest added in v1.2.0 

type ImportRoaringRequest struct {
	Clear bool
	Views map[string][]byte
}

type ImportValueRequest added in v1.0.0 

type ImportValueRequest struct {
	Index      string
	Field      string
	Shard      uint64
	ColumnIDs  []uint64
	ColumnKeys []string
	Values     []int64
}

type Index 

type Index struct {
	Stats stats.StatsClient
	// contains filtered or unexported fields
}

Index represents a container for fields.

func NewIndex 

func NewIndex(path, name string) (*Index, error)

NewIndex returns a new instance of Index.

func (*Index) AvailableShards added in v1.2.0 

func (i *Index) AvailableShards() *roaring.Bitmap

AvailableShards returns a bitmap of all shards with data in the index.

func (*Index) Close 

func (i *Index) Close() error

Close closes the index and its fields.

func (*Index) ColumnAttrStore 

func (i *Index) ColumnAttrStore() AttrStore

ColumnAttrStore returns the storage for column attributes.

func (*Index) CreateField added in v1.0.0 

func (i *Index) CreateField(name string, opts ...FieldOption) (*Field, error)

CreateField creates a field.

func (*Index) CreateFieldIfNotExists added in v1.0.0 

func (i *Index) CreateFieldIfNotExists(name string, opts ...FieldOption) (*Field, error)

CreateFieldIfNotExists creates a field with the given options if it doesn't exist.

func (*Index) DeleteField added in v1.0.0 

func (i *Index) DeleteField(name string) error

DeleteField removes a field from the index.

func (*Index) Field added in v1.0.0 

func (i *Index) Field(name string) *Field

Field returns a field in the index by name.

func (*Index) Fields added in v1.0.0 

func (i *Index) Fields() []*Field

Fields returns a list of all fields in the index.

func (*Index) Keys added in v1.0.0 

func (i *Index) Keys() bool

Keys returns true if the index uses string keys.

func (*Index) Name 

func (i *Index) Name() string

Name returns name of the index.

func (*Index) Open 

func (i *Index) Open() error

Open opens and initializes the index.

func (*Index) Options added in v0.7.0 

func (i *Index) Options() IndexOptions

Options returns all options for this index.

func (*Index) Path 

func (i *Index) Path() string

Path returns the path the index was initialized with.

type IndexInfo 

type IndexInfo struct {
	Name    string       `json:"name"`
	Options IndexOptions `json:"options"`
	Fields  []*FieldInfo `json:"fields"`
}

IndexInfo represents schema information for an index.

type IndexOptions 

type IndexOptions struct {
	Keys           bool `json:"keys"`
	TrackExistence bool `json:"trackExistence"`
}

IndexOptions represents options to set when initializing an index.

type IndexStatus added in v1.2.0 

type IndexStatus struct {
	Name   string
	Fields []*FieldStatus
}

type InternalClient added in v0.8.0 

type InternalClient interface {
	MaxShardByIndex(ctx context.Context) (map[string]uint64, error)
	Schema(ctx context.Context) ([]*IndexInfo, error)
	CreateIndex(ctx context.Context, index string, opt IndexOptions) error
	FragmentNodes(ctx context.Context, index string, shard uint64) ([]*Node, error)
	Nodes(ctx context.Context) ([]*Node, error)
	Query(ctx context.Context, index string, queryRequest *QueryRequest) (*QueryResponse, error)
	QueryNode(ctx context.Context, uri *URI, index string, queryRequest *QueryRequest) (*QueryResponse, error)
	Import(ctx context.Context, index, field string, shard uint64, bits []Bit, opts ...ImportOption) error
	ImportK(ctx context.Context, index, field string, bits []Bit, opts ...ImportOption) error
	EnsureIndex(ctx context.Context, name string, options IndexOptions) error
	EnsureField(ctx context.Context, indexName string, fieldName string) error
	EnsureFieldWithOptions(ctx context.Context, index, field string, opt FieldOptions) error
	ImportValue(ctx context.Context, index, field string, shard uint64, vals []FieldValue, opts ...ImportOption) error
	ImportValueK(ctx context.Context, index, field string, vals []FieldValue, opts ...ImportOption) error
	ExportCSV(ctx context.Context, index, field string, shard uint64, w io.Writer) error
	CreateField(ctx context.Context, index, field string) error
	CreateFieldWithOptions(ctx context.Context, index, field string, opt FieldOptions) error
	FragmentBlocks(ctx context.Context, uri *URI, index, field, view string, shard uint64) ([]FragmentBlock, error)
	BlockData(ctx context.Context, uri *URI, index, field, view string, shard uint64, block int) ([]uint64, []uint64, error)
	ColumnAttrDiff(ctx context.Context, uri *URI, index string, blks []AttrBlock) (map[uint64]map[string]interface{}, error)
	RowAttrDiff(ctx context.Context, uri *URI, index, field string, blks []AttrBlock) (map[uint64]map[string]interface{}, error)
	SendMessage(ctx context.Context, uri *URI, msg []byte) error
	RetrieveShardFromURI(ctx context.Context, index, field, view string, shard uint64, uri URI) (io.ReadCloser, error)
	ImportRoaring(ctx context.Context, uri *URI, index, field string, shard uint64, remote bool, req *ImportRoaringRequest) error
}

InternalClient should be implemented by any struct that enables any transport between nodes TODO: Refactor Note from Travis: Typically an interface containing more than two or three methods is an indication that something hasn't been architected correctly. While I understand that putting the entire Client behind an interface might require this many methods, I don't want to let it go unquestioned.

type InternalQueryClient added in v1.0.0 

type InternalQueryClient interface {
	QueryNode(ctx context.Context, uri *URI, index string, queryRequest *QueryRequest) (*QueryResponse, error)
}

type LogEntry added in v1.0.0 

type LogEntry struct {
	Type  uint8
	Index []byte
	Field []byte

	IDs  []uint64
	Keys [][]byte

	// Length of the entry, in bytes.
	// This is only populated after ReadFrom() or WriteTo().
	Length uint64
}

func (*LogEntry) ReadFrom added in v1.0.0 

func (e *LogEntry) ReadFrom(r io.Reader) (_ int64, err error)

ReadFrom deserializes a LogEntry from r. r must be a ByteReader.

func (*LogEntry) WriteTo added in v1.0.0 

func (e *LogEntry) WriteTo(w io.Writer) (_ int64, err error)

WriteTo serializes a LogEntry to w.

type Message added in v1.0.0 

type Message interface{}

Message is the interface implemented by all core pilosa types which can be serialized to messages. TODO add at least a single "isMessage()" method.

type Node 

type Node struct {
	ID            string `json:"id"`
	URI           URI    `json:"uri"`
	IsCoordinator bool   `json:"isCoordinator"`
	State         string `json:"state"`
}

Node represents a node in the cluster.

func (Node) String added in v0.9.0 

func (n Node) String() string

type NodeEvent added in v0.9.0 

type NodeEvent struct {
	Event NodeEventType
	Node  *Node
}

NodeEvent is a single event related to node activity in the cluster.

type NodeEventType added in v0.9.0 

type NodeEventType int

NodeEventType are the types of node events. 

const (
	NodeJoin NodeEventType = iota
	NodeLeave
	NodeUpdate
)

type NodeStateMessage added in v1.0.0 

type NodeStateMessage struct {
	NodeID string `protobuf:"bytes,1,opt,name=NodeID,proto3" json:"NodeID,omitempty"`
	State  string `protobuf:"bytes,2,opt,name=State,proto3" json:"State,omitempty"`
}

type NodeStatus added in v1.0.0 

type NodeStatus struct {
	Node    *Node
	Indexes []*IndexStatus
	Schema  *Schema
}

type Nodes 

type Nodes []*Node

Nodes represents a list of nodes.

func (Nodes) Clone 

func (a Nodes) Clone() []*Node

Clone returns a shallow copy of nodes.

func (Nodes) Contains 

func (a Nodes) Contains(n *Node) bool

Contains returns true if a node exists in the list.

func (Nodes) ContainsID added in v0.9.0 

func (a Nodes) ContainsID(id string) bool

ContainsID returns true if host matches one of the node's id.

func (Nodes) Filter 

func (a Nodes) Filter(n *Node) []*Node

Filter returns a new list of nodes with node removed.

func (Nodes) FilterID added in v0.9.0 

func (a Nodes) FilterID(id string) []*Node

FilterID returns a new list of nodes with ID removed.

func (Nodes) FilterURI added in v0.9.0 

func (a Nodes) FilterURI(uri URI) []*Node

FilterURI returns a new list of nodes with URI removed.

func (Nodes) IDs added in v0.9.0 

func (a Nodes) IDs() []string

IDs returns a list of all node IDs.

func (Nodes) URIs added in v0.9.0 

func (a Nodes) URIs() []URI

URIs returns a list of all uris.

type NotFoundError added in v1.0.0 

type NotFoundError struct {
	// contains filtered or unexported fields
}

NotFoundError wraps an error value to signify that a resource was not found such that in an HTTP scenario, http.StatusNotFound would be returned.

type Pair 

type Pair struct {
	ID    uint64 `json:"id"`
	Key   string `json:"key,omitempty"`
	Count uint64 `json:"count"`
}

Pair holds an id/count pair.

type Pairs 

type Pairs []Pair

Pairs is a sortable slice of Pair objects.

func (Pairs) Add 

func (p Pairs) Add(other []Pair) []Pair

Add merges other into p and returns a new slice.

func (Pairs) Keys 

func (p Pairs) Keys() []uint64

Keys returns a slice of all keys in p.

func (Pairs) Len 

func (p Pairs) Len() int

func (Pairs) Less 

func (p Pairs) Less(i, j int) bool

func (*Pairs) Pop 

func (p *Pairs) Pop() interface{}

Pop removes the minimum element from the Pair slice.

func (*Pairs) Push 

func (p *Pairs) Push(x interface{})

Push appends the element onto the Pair slice.

func (Pairs) String 

func (p Pairs) String() string

func (Pairs) Swap 

func (p Pairs) Swap(i, j int)

type QueryRequest 

type QueryRequest struct {
	// Index to execute query against.
	Index string

	// The query string to parse and execute.
	Query string

	// The shards to include in the query execution.
	// If empty, all shards are included.
	Shards []uint64

	// Return column attributes, if true.
	ColumnAttrs bool

	// Do not return row attributes, if true.
	ExcludeRowAttrs bool

	// Do not return columns, if true.
	ExcludeColumns bool

	// If true, indicates that query is part of a larger distributed query.
	// If false, this request is on the originating node.
	Remote bool
}

QueryRequest represent a request to process a query.

type QueryResponse 

type QueryResponse struct {
	// Result for each top-level query call.
	// Can be a Bitmap, Pairs, or uint64.
	Results []interface{}

	// Set of column attribute objects matching IDs returned in Result.
	ColumnAttrSets []*ColumnAttrSet

	// Error during parsing or execution.
	Err error
}

QueryResponse represent a response from a processed query.

func (*QueryResponse) MarshalJSON 

func (resp *QueryResponse) MarshalJSON() ([]byte, error)

MarshalJSON marshals QueryResponse into a JSON-encoded byte slice

type RecalculateCaches added in v1.0.0 

type RecalculateCaches struct{}

type ResizeInstruction added in v1.0.0 

type ResizeInstruction struct {
	JobID         int64
	Node          *Node
	Coordinator   *Node
	Sources       []*ResizeSource
	NodeStatus    *NodeStatus
	ClusterStatus *ClusterStatus
}

type ResizeInstructionComplete added in v1.0.0 

type ResizeInstructionComplete struct {
	JobID int64
	Node  *Node
	Error string
}

type ResizeSource added in v1.0.0 

type ResizeSource struct {
	Node  *Node  `protobuf:"bytes,1,opt,name=Node" json:"Node,omitempty"`
	Index string `protobuf:"bytes,2,opt,name=Index,proto3" json:"Index,omitempty"`
	Field string `protobuf:"bytes,3,opt,name=Field,proto3" json:"Field,omitempty"`
	View  string `protobuf:"bytes,4,opt,name=View,proto3" json:"View,omitempty"`
	Shard uint64 `protobuf:"varint,5,opt,name=Shard,proto3" json:"Shard,omitempty"`
}

type Row added in v1.0.0 

type Row struct {

	// String keys translated to/from segment columns.
	Keys []string

	// Attributes associated with the row.
	Attrs map[string]interface{}
	// contains filtered or unexported fields
}

Row is a set of integers (the associated columns), and attributes which are arbitrary key/value pairs storing metadata about what the row represents.

func NewRow added in v1.0.0 

func NewRow(columns ...uint64) *Row

NewRow returns a new instance of Row.

func (*Row) Columns added in v1.0.0 

func (r *Row) Columns() []uint64

Columns returns the columns in r as a slice of ints.

func (*Row) Count added in v1.0.0 

func (r *Row) Count() uint64

Count returns the number of columns in the row.

func (*Row) Difference added in v1.0.0 

func (r *Row) Difference(other *Row) *Row

Difference returns the diff of r and other.

func (*Row) Intersect added in v1.0.2 

func (r *Row) Intersect(other *Row) *Row

Intersect returns the itersection of r and other.

func (*Row) MarshalJSON added in v1.0.0 

func (r *Row) MarshalJSON() ([]byte, error)

MarshalJSON returns a JSON-encoded byte slice of r.

func (*Row) Merge added in v1.0.0 

func (r *Row) Merge(other *Row)

Merge merges data from other into r.

func (*Row) Segments added in v1.0.0 

func (r *Row) Segments() []rowSegment

Segments returns a list of all segments in the row.

func (*Row) SetBit added in v1.0.0 

func (r *Row) SetBit(i uint64) (changed bool)

SetBit sets the i-th column of the row.

func (*Row) Union added in v1.0.0 

func (r *Row) Union(other *Row) *Row

Union returns the bitwise union of r and other.

func (*Row) Xor added in v1.0.0 

func (r *Row) Xor(other *Row) *Row

Xor returns the xor of r and other.

type RowIDs added in v1.2.0 

type RowIDs []uint64

RowIDs is a query return type for just uint64 row ids. It should only be used internally (since RowIdentifiers is the external return type), but it is exported because the proto package needs access to it.

type RowIdentifiers added in v1.2.0 

type RowIdentifiers struct {
	Rows []uint64 `json:"rows"`
	Keys []string `json:"keys,omitempty"`
}

RowIdentifiers is a return type for a list of row ids or row keys. The names `Rows` and `Keys` are meant to follow the same convention as the Row query which returns `Columns` and `Keys`. TODO: Rename this to something better. Anything.

type Schema added in v1.0.0 

type Schema struct {
	Indexes []*IndexInfo
}

Schema contains information about indexes and their configuration.

type Serializer added in v1.0.0 

type Serializer interface {
	Marshal(Message) ([]byte, error)
	Unmarshal([]byte, Message) error
}

Serializer is an interface for serializing pilosa types to bytes and back.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server represents a holder wrapped by a running HTTP server.

func NewServer 

func NewServer(opts ...ServerOption) (*Server, error)

NewServer returns a new instance of Server.

func (*Server) Close 

func (s *Server) Close() error

Close closes the server and waits for it to shutdown.

func (*Server) Holder 

func (s *Server) Holder() *Holder

TODO: have this return an interface for Holder instead of concrete object?

func (*Server) NodeID added in v0.9.0 

func (s *Server) NodeID() string

NodeID returns the server's node id.

func (*Server) Open 

func (s *Server) Open() error

Open opens and initializes the server.

func (*Server) SendAsync added in v0.9.0 

func (s *Server) SendAsync(m Message) error

SendAsync represents an implementation of Broadcaster.

func (*Server) SendSync added in v0.9.0 

func (s *Server) SendSync(m Message) error

SendSync represents an implementation of Broadcaster.

func (*Server) SendTo added in v0.9.0 

func (s *Server) SendTo(to *Node, m Message) error

SendTo represents an implementation of Broadcaster.

func (*Server) SyncData added in v1.0.0 

func (s *Server) SyncData() error

SyncData manually invokes the anti entropy process which makes sure that this node has the data from all replicas across the cluster.

type ServerOption added in v0.9.0 

type ServerOption func(s *Server) error

ServerOption is a functional option type for pilosa.Server

func OptServerAntiEntropyInterval added in v0.9.0 

func OptServerAntiEntropyInterval(interval time.Duration) ServerOption

func OptServerAttrStoreFunc added in v0.9.0 

func OptServerAttrStoreFunc(af func(string) AttrStore) ServerOption

func OptServerClusterDisabled added in v1.0.0 

func OptServerClusterDisabled(disabled bool, hosts []string) ServerOption

OptClusterDisabled tells the server whether to use a static cluster with the defined hosts. Mostly used for testing.

func OptServerClusterHasher added in v1.0.0 

func OptServerClusterHasher(h Hasher) ServerOption

func OptServerDataDir added in v0.9.0 

func OptServerDataDir(dir string) ServerOption

func OptServerDiagnosticsInterval added in v0.9.0 

func OptServerDiagnosticsInterval(dur time.Duration) ServerOption

func OptServerGCNotifier added in v0.9.0 

func OptServerGCNotifier(gcn GCNotifier) ServerOption

func OptServerInternalClient added in v1.0.0 

func OptServerInternalClient(c InternalClient) ServerOption

func OptServerIsCoordinator added in v1.0.0 

func OptServerIsCoordinator(is bool) ServerOption

func OptServerLogger added in v0.9.0 

func OptServerLogger(l logger.Logger) ServerOption

func OptServerLongQueryTime added in v0.9.0 

func OptServerLongQueryTime(dur time.Duration) ServerOption

func OptServerMaxWritesPerRequest added in v0.9.0 

func OptServerMaxWritesPerRequest(n int) ServerOption

func OptServerMetricInterval added in v0.9.0 

func OptServerMetricInterval(dur time.Duration) ServerOption

func OptServerNodeID added in v1.0.0 

func OptServerNodeID(nodeID string) ServerOption

func OptServerPrimaryTranslateStore added in v1.0.0 

func OptServerPrimaryTranslateStore(store TranslateStore) ServerOption

DEPRECATED

func OptServerPrimaryTranslateStoreFunc added in v1.1.0 

func OptServerPrimaryTranslateStoreFunc(tf func(interface{}) TranslateStore) ServerOption

func OptServerReplicaN added in v0.9.0 

func OptServerReplicaN(n int) ServerOption

func OptServerSerializer added in v1.0.0 

func OptServerSerializer(ser Serializer) ServerOption

func OptServerStatsClient added in v0.9.0 

func OptServerStatsClient(sc stats.StatsClient) ServerOption

func OptServerSystemInfo added in v0.9.0 

func OptServerSystemInfo(si SystemInfo) ServerOption

func OptServerTranslateFileMapSize added in v1.2.0 

func OptServerTranslateFileMapSize(mapSize int) ServerOption

func OptServerURI added in v0.9.0 

func OptServerURI(uri *URI) ServerOption

type SetCoordinatorMessage added in v1.0.0 

type SetCoordinatorMessage struct {
	New *Node
}

type SystemInfo added in v0.9.0 

type SystemInfo interface {
	Uptime() (uint64, error)
	Platform() (string, error)
	Family() (string, error)
	OSVersion() (string, error)
	KernelVersion() (string, error)
	MemFree() (uint64, error)
	MemTotal() (uint64, error)
	MemUsed() (uint64, error)
	CPUArch() string
}

SystemInfo collects information about the host OS.

type TimeQuantum 

type TimeQuantum string

TimeQuantum represents a time granularity for time-based bitmaps.

func (TimeQuantum) HasDay 

func (q TimeQuantum) HasDay() bool

HasDay returns true if the quantum contains a 'D' unit.

func (TimeQuantum) HasHour 

func (q TimeQuantum) HasHour() bool

HasHour returns true if the quantum contains a 'H' unit.

func (TimeQuantum) HasMonth 

func (q TimeQuantum) HasMonth() bool

HasMonth returns true if the quantum contains a 'M' unit.

func (TimeQuantum) HasYear 

func (q TimeQuantum) HasYear() bool

HasYear returns true if the quantum contains a 'Y' unit.

func (*TimeQuantum) Set added in v0.7.0 

func (q *TimeQuantum) Set(value string) error

Set sets the time quantum value.

func (TimeQuantum) String added in v0.7.0 

func (q TimeQuantum) String() string

func (TimeQuantum) Type added in v0.7.0 

func (q TimeQuantum) Type() string

Type returns the type of a time quantum value.

func (TimeQuantum) Valid 

func (q TimeQuantum) Valid() bool

Valid returns true if q is a valid time quantum value.

type Topology added in v0.9.0 

type Topology struct {
	// contains filtered or unexported fields
}

Topology represents the list of hosts in the cluster.

func (*Topology) ContainsID added in v0.9.0 

func (t *Topology) ContainsID(id string) bool

ContainsID returns true if id matches one of the topology's IDs.

type TranslateFile added in v1.0.0 

type TranslateFile struct {
	Path string

	// If non-nil, data is streamed from a primary and this is a read-only store.
	PrimaryTranslateStore TranslateStore
	// contains filtered or unexported fields
}

TranslateFile is an on-disk storage engine for translating string-to-uint64 values.

func NewTranslateFile added in v1.0.0 

func NewTranslateFile(opts ...TranslateFileOption) *TranslateFile

NewTranslateFile returns a new instance of TranslateFile.

func (*TranslateFile) Close added in v1.0.0 

func (s *TranslateFile) Close() (err error)

func (*TranslateFile) Closing added in v1.0.0 

func (s *TranslateFile) Closing() <-chan struct{}

Closing returns a channel that is closed when the store is closed.

func (*TranslateFile) Open added in v1.0.0 

func (s *TranslateFile) Open() (err error)

func (*TranslateFile) Reader added in v1.0.0 

func (s *TranslateFile) Reader(ctx context.Context, offset int64) (io.ReadCloser, error)

Reader returns a reader that streams the underlying data file.

func (*TranslateFile) SetPrimaryStore added in v1.1.0 

func (s *TranslateFile) SetPrimaryStore(id string, ts TranslateStore)

SetPrimaryStore sets the translate files's primary translate store. The id value is used to determine whether the primary needs to be changed from the current value (i.e. calling this multiple times with the same input values will no-op on all subsequent calls).

func (*TranslateFile) TranslateColumnToString added in v1.0.0 

func (s *TranslateFile) TranslateColumnToString(index string, value uint64) (string, error)

TranslateColumnToString converts a uint64 id to its associated string value. If the id is not associated with a string value then a blank string is returned.

func (*TranslateFile) TranslateColumnsToUint64 added in v1.0.0 

func (s *TranslateFile) TranslateColumnsToUint64(index string, values []string) ([]uint64, error)

TranslateColumnsToUint64 converts values to a uint64 id. If value does not have an associated id then one is created.

func (*TranslateFile) TranslateRowToString added in v1.0.0 

func (s *TranslateFile) TranslateRowToString(index, field string, id uint64) (string, error)

func (*TranslateFile) TranslateRowsToUint64 added in v1.0.0 

func (s *TranslateFile) TranslateRowsToUint64(index, field string, values []string) ([]uint64, error)

func (*TranslateFile) WriteNotify added in v1.0.0 

func (s *TranslateFile) WriteNotify() <-chan struct{}

WriteNotify returns a channel that is closed when a new entry is written.

type TranslateFileOption added in v1.2.0 

type TranslateFileOption func(f *TranslateFile) error

TranslateFileOption is a functional option type for pilosa.TranslateFile

func OptTranslateFileLogger added in v1.2.0 

func OptTranslateFileLogger(l logger.Logger) TranslateFileOption

func OptTranslateFileMapSize added in v1.2.0 

func OptTranslateFileMapSize(mapSize int) TranslateFileOption

type TranslateKeysRequest added in v1.2.0 

type TranslateKeysRequest struct {
	Index string
	Field string
	Keys  []string
}

type TranslateKeysResponse added in v1.2.0 

type TranslateKeysResponse struct {
	IDs []uint64
}

type TranslateStore added in v1.0.0 

type TranslateStore interface {
	TranslateColumnsToUint64(index string, values []string) ([]uint64, error)
	TranslateColumnToString(index string, values uint64) (string, error)

	TranslateRowsToUint64(index, field string, values []string) ([]uint64, error)
	TranslateRowToString(index, field string, values uint64) (string, error)

	// Returns a reader from the given offset of the raw data file.
	// The returned reader must be closed by the caller when done.
	Reader(ctx context.Context, off int64) (io.ReadCloser, error)
}

TranslateStore is the storage for translation string-to-uint64 values.

type URI added in v0.8.0 

type URI struct {
	Scheme string `json:"scheme"`
	Host   string `json:"host"`
	Port   uint16 `json:"port"`
}

URI represents a Pilosa URI. A Pilosa URI consists of three parts: 1) Scheme: Protocol of the URI. Default: http. 2) Host: Hostname or IP URI. Default: localhost. IPv6 addresses should be written in brackets, e.g., `[fd42:4201:f86b:7e09:216:3eff:fefa:ed80]`. 3) Port: Port of the URI. Default: 10101.

All parts of the URI are optional. The following are equivalent: 

http://localhost:10101
http://localhost
http://:10101
localhost:10101
localhost
:10101

func AddressWithDefaults added in v0.6.0 

func AddressWithDefaults(addr string) (*URI, error)

AddressWithDefaults converts addr into a valid address, using defaults when necessary.

func NewURIFromAddress added in v0.8.0 

func NewURIFromAddress(address string) (*URI, error)

NewURIFromAddress parses the passed address and returns a URI.

func NewURIFromHostPort added in v0.8.0 

func NewURIFromHostPort(host string, port uint16) (*URI, error)

NewURIFromHostPort returns a URI with specified host and port.

func (*URI) HostPort added in v0.8.0 

func (u *URI) HostPort() string

HostPort returns `Host:Port`

func (*URI) MarshalJSON added in v0.9.0 

func (u *URI) MarshalJSON() ([]byte, error)

MarshalJSON marshals URI into a JSON-encoded byte slice.

func (*URI) Path added in v0.8.0 

func (u *URI) Path(path string) string

Path returns URI with path

func (*URI) Set added in v0.8.0 

func (u *URI) Set(value string) error

Set sets the uri value.

func (*URI) SetPort added in v0.8.0 

func (u *URI) SetPort(port uint16)

SetPort sets the port of this URI.

func (URI) String added in v0.8.0 

func (u URI) String() string

String returns the address as a string.

func (URI) Type added in v0.8.0 

func (u URI) Type() string

Type returns the type of a uri.

func (*URI) UnmarshalJSON added in v0.9.0 

func (u *URI) UnmarshalJSON(b []byte) error

type URIs added in v0.9.0 

type URIs []URI

func (URIs) HostPortStrings added in v0.9.0 

func (u URIs) HostPortStrings() []string

type UpdateCoordinatorMessage added in v1.0.0 

type UpdateCoordinatorMessage struct {
	New *Node
}

type ValCount added in v0.9.0 

type ValCount struct {
	Val   int64 `json:"value"`
	Count int64 `json:"count"`
}

ValCount represents a grouping of sum & count for Sum() and Average() calls.

type ViewInfo 

type ViewInfo struct {
	Name string `json:"name"`
}

ViewInfo represents schema information for a view.

Source Files

View all Source files

Directories

Path Synopsis
cmd
Package cmd contains all the pilosa subcommand definitions (1 per file).
 Package cmd contains all the pilosa subcommand definitions (1 per file).
pilosa
This is the entrypoint for the Pilosa binary.
 This is the entrypoint for the Pilosa binary.
package ctl contains all pilosa subcommands other than 'server'.
 package ctl contains all pilosa subcommands other than 'server'.
encoding
proto
Package enterprise injects enterprise implementations of various Pilosa features when Pilosa is built with "ENTERPRISE=1 make install".
 Package enterprise injects enterprise implementations of various Pilosa features when Pilosa is built with "ENTERPRISE=1 make install".
b
test
Package lru implements an LRU cache.
 Package lru implements an LRU cache.
Package pql defines the Pilosa Query Language.
 Package pql defines the Pilosa Query Language.
Package roaring implements roaring bitmaps with support for incremental changes.
 Package roaring implements roaring bitmaps with support for incremental changes.
opentracing

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.