datasets

package
v0.0.0-...-8cdd8b5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 9, 2025 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package datasets provides a client for interacting with Tilebox Datasets.

Documentation: https://docs.tilebox.com/datasets

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func As 

func As[T proto.Message](seq iter.Seq2[*RawDatapoint, error]) iter.Seq2[*Datapoint[T], error]

As converts a sequence of RawDatapoint into a sequence of Datapoint with the given type.

func Collect 

func Collect[K any](seq iter.Seq2[K, error]) ([]K, error)

Collect converts any sequence into a slice.

It returns an error if any of the elements in the sequence has a non-nil error.

Types

type Client 

type Client interface {
	Datasets(ctx context.Context) ([]*Dataset, error)
	Dataset(ctx context.Context, slug string) (*Dataset, error)
}

Client is a Tilebox Datasets client.

func NewClient 

func NewClient(options ...ClientOption) Client

NewClient creates a new Tilebox Datasets client.

By default, the returned Client is configured with:

The passed options are used to override these default values and configure the returned Client appropriately.

type ClientOption 

type ClientOption func(*clientConfig)

ClientOption is an interface for configuring a client. Using such options helpers is a quite common pattern in Go, as it allows for optional parameters in constructors. This concrete implementation here is inspired by how libraries such as axiom-go and connect do their configuration.

func WithAPIKey 

func WithAPIKey(apiKey string) ClientOption

WithAPIKey sets the API key to use for the client.

Defaults to no API key.

func WithConnectClientOptions 

func WithConnectClientOptions(options ...connect.ClientOption) ClientOption

WithConnectClientOptions sets additional options for the connect.HTTPClient.

func WithDisableTracing 

func WithDisableTracing() ClientOption

WithDisableTracing disables OpenTelemetry tracing for the client.

func WithHTTPClient 

func WithHTTPClient(httpClient connect.HTTPClient) ClientOption

WithHTTPClient sets the connect.HTTPClient to use for the client.

Defaults to grpc.RetryHTTPClient.

func WithURL 

func WithURL(url string) ClientOption

WithURL sets the URL of the Tilebox Datasets service.

Defaults to "https://api.tilebox.com".

type Collection 

type Collection struct {
	// ID is the unique identifier of the collection.
	ID uuid.UUID
	// Name is the name of the collection.
	Name string
	// Availability is the time interval for which data is available.
	Availability TimeInterval
	// Count is the number of datapoints in the collection.
	Count uint64
	// contains filtered or unexported fields
}

Collection represents a Tilebox Time Series Dataset collection.

Documentation: https://docs.tilebox.com/datasets/collections

func NewCollection 

func NewCollection(id uuid.UUID, name string, availability TimeInterval, count uint64, service Service) *Collection

NewCollection creates a new Collection.

func (*Collection) Delete 

func (c *Collection) Delete(ctx context.Context, data []*RawDatapoint) (*DeleteResponse, error)

Delete datapoints from a collection.

The datapoints are identified by their IDs.

func (*Collection) DeleteIDs 

func (c *Collection) DeleteIDs(ctx context.Context, datapointIDs []uuid.UUID) (*DeleteResponse, error)

DeleteIDs deletes datapoints from a collection by their IDs.

func (*Collection) Ingest 

func (c *Collection) Ingest(ctx context.Context, data []*RawDatapoint, allowExisting bool) (*IngestResponse, error)

Ingest datapoints into a collection.

data is a list of datapoints to ingest that should be created using Datapoints.

allowExisting specifies whether to allow existing datapoints as part of the request. If true, datapoints that already exist will be ignored, and the number of such existing datapoints will be returned in the response. If false, any datapoints that already exist will result in an error. Setting this to true is useful for achieving idempotency (e.g. allowing re-ingestion of datapoints that have already been ingested in the past).

func (*Collection) Load 

func (c *Collection) Load(ctx context.Context, interval LoadInterval, options ...LoadOption) iter.Seq2[*RawDatapoint, error]

Load loads datapoints from a collection.

interval specifies the time or data point interval for which data should be loaded.

WithSkipData and WithSkipMeta can be used to skip the data or metadata when loading datapoints. If both WithSkipData and WithSkipMeta are specified, the response will only consist of a list of datapoint IDs without any additional data or metadata.

The datapoints are loaded in a lazy manner, and returned as a sequence of RawDatapoint. The output sequence can be transformed into typed Datapoint using CollectAs or As functions.

Documentation: https://docs.tilebox.com/datasets/loading-data

type Datapoint 

type Datapoint[T proto.Message] struct {
	// Meta contains the metadata of the datapoint.
	Meta *DatapointMetadata
	// Data contains the data of the datapoint.
	Data T
}

Datapoint represents a datapoint in a collection. It contains the metadata and the data itself.

func CollectAs 

func CollectAs[T proto.Message](seq iter.Seq2[*RawDatapoint, error]) ([]*Datapoint[T], error)

CollectAs converts a sequence of RawDatapoint into a slice of Datapoint with the given type.

func NewDatapoint 

func NewDatapoint[T proto.Message](time time.Time, message T) *Datapoint[T]

NewDatapoint creates a new datapoint with the given time and message.

type DatapointMetadata 

type DatapointMetadata struct {
	// ID is the unique identifier of the datapoint.
	ID uuid.UUID
	// EventTime is the time when the datapoint was created.
	EventTime time.Time
	// IngestionTime is the time when the datapoint was ingested into Tilebox.
	IngestionTime time.Time
}

DatapointMetadata contains metadata for a datapoint.

type Dataset 

type Dataset struct {
	// ID is the unique identifier of the dataset.
	ID uuid.UUID

	// Name is the name of the dataset.
	Name string
	// Summary is a summary of the purpose of the dataset.
	Summary string
	// contains filtered or unexported fields
}

Dataset represents a Tilebox Time Series Dataset.

Documentation: https://docs.tilebox.com/datasets/timeseries

func NewDataset 

func NewDataset(id uuid.UUID, name string, summary string, service Service) *Dataset

NewDataset creates a new Dataset.

func (*Dataset) Collection 

func (d *Dataset) Collection(ctx context.Context, name string) (*Collection, error)

Collection returns a collection by its name.

func (*Dataset) Collections 

func (d *Dataset) Collections(ctx context.Context) ([]*Collection, error)

Collections returns a list of all available collections in the dataset.

func (*Dataset) CreateCollection 

func (d *Dataset) CreateCollection(ctx context.Context, collectionName string) (*Collection, error)

CreateCollection creates a new collection in the dataset with the given name.

type DeleteResponse 

type DeleteResponse struct {
	// NumDeleted is the number of datapoints that were deleted.
	NumDeleted int64
}

DeleteResponse contains the response from the Delete method.

type IngestResponse 

type IngestResponse struct {
	// NumCreated is the number of datapoints that were created.
	NumCreated int64
	// NumExisting is the number of datapoints that were ignored because they already existed.
	NumExisting int64
	// DatapointIDs is the list of all the datapoints IDs in the same order as the datapoints in the request.
	DatapointIDs []uuid.UUID
}

IngestResponse contains the response from the Ingest method.

type LoadInterval 

type LoadInterval interface {
	ToProtoTimeInterval() *datasetsv1.TimeInterval
	ToProtoDatapointInterval() *datasetsv1.DatapointInterval
}

LoadInterval is an interface for loading data from a collection.

type LoadOption 

type LoadOption func(*loadConfig)

LoadOption is an interface for configuring a Load request.

func WithSkipData 

func WithSkipData() LoadOption

WithSkipData skips the data when loading datapoints. It is an optional flag for omitting the actual datapoint data from the response. If set, no datapoint data will be returned.

Defaults to false.

func WithSkipMeta 

func WithSkipMeta() LoadOption

WithSkipMeta skips the metadata when loading datapoints. It is an optional flag for omitting the metadata from the response. If set, no metadata will be returned.

Defaults to false.

type RawDatapoint 

type RawDatapoint struct {
	// Meta contains the metadata of the datapoint.
	Meta *DatapointMetadata
	// Data contains the data of the datapoint in an internal raw format.
	Data []byte
}

RawDatapoint is an internal representation of a datapoint.

It can be transformed into a Datapoint using CollectAs or As functions.

func Datapoints 

func Datapoints[T proto.Message](data ...*Datapoint[T]) ([]*RawDatapoint, error)

Datapoints converts a list of Datapoint to RawDatapoint.

It is used to convert the data before ingesting it into a collection.

type Service 

type Service interface {
	GetDataset(ctx context.Context, slug string) (*datasetsv1.Dataset, error)
	ListDatasets(ctx context.Context) (*datasetsv1.ListDatasetsResponse, error)
	CreateCollection(ctx context.Context, datasetID uuid.UUID, collectionName string) (*datasetsv1.CollectionInfo, error)
	GetCollections(ctx context.Context, datasetID uuid.UUID) (*datasetsv1.Collections, error)
	GetCollectionByName(ctx context.Context, datasetID uuid.UUID, collectionName string) (*datasetsv1.CollectionInfo, error)
	GetDatasetForInterval(ctx context.Context, collectionID uuid.UUID, timeInterval *datasetsv1.TimeInterval, datapointInterval *datasetsv1.DatapointInterval, page *datasetsv1.Pagination, skipData bool, skipMeta bool) (*datasetsv1.Datapoints, error)
	GetDatapointByID(ctx context.Context, collectionID uuid.UUID, datapointID uuid.UUID, skipData bool) (*datasetsv1.Datapoint, error)
	IngestDatapoints(ctx context.Context, collectionID uuid.UUID, datapoints *datasetsv1.Datapoints, allowExisting bool) (*datasetsv1.IngestDatapointsResponse, error)
	DeleteDatapoints(ctx context.Context, collectionID uuid.UUID, datapointIDs []uuid.UUID) (*datasetsv1.DeleteDatapointsResponse, error)
}

type TimeInterval 

type TimeInterval struct {
	// Start is the start time of the interval.
	Start time.Time
	// End is the end time of the interval.
	End time.Time

	// We use exclusive for Start and inclusive for End, because that way when both are false
	// we have a half-open interval [Start, End) which is the default behaviour we want to achieve.
	StartExclusive bool
	EndInclusive   bool
}

TimeInterval represents a time interval.

Both the Start and End time can be exclusive or inclusive.

func NewStandardTimeInterval 

func NewStandardTimeInterval(start, end time.Time) *TimeInterval

NewStandardTimeInterval creates a new TimeInterval that represents a standard interval with inclusive start and exclusive end.

func NewTimeInterval 

func NewTimeInterval(start, end time.Time, startExclusive, endInclusive bool) *TimeInterval

NewTimeInterval creates a new TimeInterval.

func (*TimeInterval) Duration 

func (t *TimeInterval) Duration() time.Duration

Duration returns the duration between the start and end times of the TimeInterval.

func (*TimeInterval) ToHalfOpen 

func (t *TimeInterval) ToHalfOpen() *TimeInterval

ToHalfOpen converts the TimeInterval to a half-open interval [Start, End).

func (*TimeInterval) ToProtoDatapointInterval 

func (t *TimeInterval) ToProtoDatapointInterval() *datasetsv1.DatapointInterval

func (*TimeInterval) ToProtoTimeInterval 

func (t *TimeInterval) ToProtoTimeInterval() *datasetsv1.TimeInterval

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.