thingfulx

package module
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 21, 2020 License: MIT Imports: 14 Imported by: 2

README

thingfulx

Library providing an interface for Thingful fetcher and parser interfaces to implement to provide new upstream source parsing capability.

wercker status GoDoc Go Report Card

Documentation

Overview

Package thingfulx provides a compatibility layer for implementing fetchers for use with the Thingful indexing system. New upstream providers can be added by implementing the core Fetcher interface contained within this library. These libraries can then be easily imported and used by Thingful.

Index

Constants

View Source 
const (
	// ReproductionPerm is "making multiple copies of a dataset
	ReproductionPerm = "cc:Reproduction"

	// DistributionPerm is "distribution, public display, and publicly performance"
	DistributionPerm = "cc:Distribution"

	// DerivativeWorksPerm is "distribution of derivative works"
	DerivativeWorksPerm = "cc:DerivativeWorks"

	// SharingPerm is "permits commercial derivatives, but only non-commercial
	// distribution"
	SharingPerm = "cc:Sharing"

	// NoticeReq is "copyright and license notices be kept intact"
	NoticeReq = "cc:Notice"

	// AttributionReq is "credit required to give to copyright holder and/or author"
	AttributionReq = "cc:Attribution"

	// ShareAlikeReq is "derivative works be licensed under the same terms or
	// compatible terms as the original work"
	ShareAlikeReq = "cc:ShareAlike"

	// SourceCodeReq is "source code (the preferred form for making modifications)
	// must be provided when exercising some rights granted by the license"
	SourceCodeReq = "cc:SourceCode"

	// CopyleftReq is "derivative and combined works must be licensed under
	// specified terms, similar to those on the original work"
	CopyleftReq = "cc:Copyleft"

	// LesserCopyleftReq is "derivative works must be licensed under specified
	// terms, with at least the same conditions as the original work;
	// combinations with the work may be licensed under different terms"
	LesserCopyleftReq = "cc:LesserCopyleft"

	// CommercialUseProhib is "exercising rights for commercial purposes"
	CommercialUseProhib = "cc:CommercialUse"

	// HighIncomeNationUseProhib is "use in a non-developing country"
	HighIncomeNationUseProhib = "cc:HighIncomeNationUse"

	// CC0V1URL is the URL identifier for CC0 1.0 Universal License
	CC0V1URL = "https://creativecommons.org/publicdomain/zero/1.0/"

	// CCByV3URL is the URL identifier for Creative Commons Attribution 3.0
	// Unported License
	CCByV3URL = "https://creativecommons.org/licenses/by/3.0/"

	// CCBySAV4URL is the URL identifier for Creative Commons
	// Attribution-ShareAlike 4.0 International
	CCBySAV4URL = "https://creativecommons.org/licenses/by-sa/4.0/"

	// CCByV4URL is the URL identifier for the Creative Commons Attribution 4.0
	// License
	CCByV4URL = "https://creativecommons.org/licenses/by/4.0/"

	// CCByNCV3URL is the URL identifier for Creative Commons
	// Atribution-NonCommercial 3.0 License
	CCByNCV3URL = "https://creativecommons.org/licenses/by-nc/3.0/"

	// CCByNDV3URL is the URL identifier for Creative Commons
	// Attribution-NoDerivs 3.0 License.
	CCByNDV3URL = "https://creativecommons.org/licenses/by-nd/3.0/"

	// CCByNCSAV3URL is the URL identifier for Creative Commons
	// Attribution-NonCommercial-Sharealike 3.0 License.
	CCByNCSAV3URL = "https://creativecommons.org/licenses/by-nc-sa/3.0/"

	// OGLV2URL is the URL identifier for the Open Government Licence version
	// 2.0 License.
	OGLV2URL = "http://www.nationalarchives.gov.uk/doc/open-government-licence/version/2/"

	// OGLV3URL is the URL identifier for the Open Government Licence version
	// 3.0 License.
	OGLV3URL = "http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/"

	// PDDLV1URL is the URL identifier for the Open Data Commons Public Domain
	// Dedication and License.
	PDDLV1URL = "https://opendatacommons.org/licenses/pddl/1.0/"

	// ODCByV1URL is the URL identifier for the Open Data Commons Attribution
	// License.
	ODCByV1URL = "https://opendatacommons.org/licenses/by/1.0/"

	// ODbLV1URL is a the URL identifier for the Open Data Commons Open Database
	// License.
	ODbLV1URL = "https://opendatacommons.org/licenses/odbl/1.0/"

	// IODLV2URL is the URL identifier for the Italian Open Data License
	IODLV2URL = "https://www.dati.gov.it/content/italian-open-data-license-v20"

	// SGODLV1 is the URL identifier for the Singapore Open Data License
	SGODLV1 = "https://data.gov.sg/open-data-licence"

	// Unknown is a placeholder identifier for an unknown or unspecified data license
	Unknown = "urn:thingful:data-license"
)
View Source 
const (
	// ErrInvalidData is a generic error that can be returned if retrieved data
	// is invalid in any way
	ErrInvalidData = Error("thingfulx: invalid data")

	// ErrInvalidLocation is an error indicating that there is a problem with the
	// location of a fetched thing
	ErrInvalidLocation = Error("thingfulx: missing or invalid location data")

	// ErrInvalidTime is an error indicating that there is a problem with parsing
	// the time value from a retrieved thing
	ErrInvalidTime = Error("thingfulx: invalid time data")

	// ErrInvalidURL is an error used to flag an invalid url
	ErrInvalidURL = Error("thingfulx: invalid url")

	// ErrNotFound is an error that can be used for a 404 not found response
	ErrNotFound = Error("thingfulx: unexpected HTTP response code, got 404")

	// ErrRobotsForbidden is an error that is returned if access to a resource is
	// forbidden by the remote site's robots.txt
	ErrRobotsForbidden = Error("thingfulx: robots.txt forbids access to this resource")
)
View Source 
const (
	// ClientToken is the key we use when setting a token on a context when making
	// a request. Note it is the responsibility of the implementing indexer to
	// determine how to send this token value to the data infrastructure (i.e.
	// Authorization header, API Key in the query string or a header).
	ClientToken = ContextKey("client-token")
)

Variables

View Source 
var DataType_name = map[int32]string{
	0: "NUMBER",
	1: "INTEGER",
	2: "STRING",
	3: "DATE_TIME",
	4: "TIME",
	5: "BOOLEAN",
	6: "BYTES",
	7: "UNKNOWN",
}
View Source 
var DataType_value = map[string]int32{
	"NUMBER":    0,
	"INTEGER":   1,
	"STRING":    2,
	"DATE_TIME": 3,
	"TIME":      4,
	"BOOLEAN":   5,
	"BYTES":     6,
	"UNKNOWN":   7,
}
View Source 
var Visibility_name = map[int32]string{
	0: "OPEN",
	1: "SHARED",
	3: "CLOSED",
}
View Source 
var Visibility_value = map[string]int32{
	"OPEN":   0,
	"SHARED": 1,
	"CLOSED": 3,
}

Functions

This section is empty.

Types

type Attribution 

type Attribution struct {
	// Name is the name of at attribution
	Name string `protobuf:"bytes,1,opt,name=Name,proto3" json:"Name,omitempty"`
	// URL  is the URL of  the attribution
	URL                  string   `protobuf:"bytes,2,opt,name=URL,proto3" json:"URL,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

Attribution is a message type used to add attributions to a Channel.

func (*Attribution) Descriptor added in v0.9.0 

func (*Attribution) Descriptor() ([]byte, []int)

func (*Attribution) GetName added in v0.9.0 

func (m *Attribution) GetName() string

func (*Attribution) GetURL added in v0.9.0 

func (m *Attribution) GetURL() string

func (*Attribution) ProtoMessage added in v0.9.0 

func (*Attribution) ProtoMessage()

func (*Attribution) Reset added in v0.9.0 

func (m *Attribution) Reset()

func (*Attribution) String added in v0.9.0 

func (m *Attribution) String() string

func (*Attribution) XXX_DiscardUnknown added in v0.9.0 

func (m *Attribution) XXX_DiscardUnknown()

func (*Attribution) XXX_Marshal added in v0.9.0 

func (m *Attribution) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Attribution) XXX_Merge added in v0.9.0 

func (m *Attribution) XXX_Merge(src proto.Message)

func (*Attribution) XXX_Size added in v0.9.0 

func (m *Attribution) XXX_Size() int

func (*Attribution) XXX_Unmarshal added in v0.9.0 

func (m *Attribution) XXX_Unmarshal(b []byte) error

type Channel 

type Channel struct {
	// URL contains a unique URL for the Channel that must be the URL at which the
	// Channel's data is available
	Url string `protobuf:"bytes,1,opt,name=url,proto3" json:"url,omitempty"`
	// Title contains a unique (if possible) title for the Channel
	Title string `protobuf:"bytes,2,opt,name=title,proto3" json:"title,omitempty"`
	// Description contains a human centered description of the Channel if
	// available.
	Description string `protobuf:"bytes,3,opt,name=description,proto3" json:"description,omitempty"`
	// Webpage contains a human centered web resource where information about the
	// specific device providing the Channel may be found
	Webpage string `protobuf:"bytes,4,opt,name=webpage,proto3" json:"webpage,omitempty"`
	// IndexedAt contains an RFC3339 timestamp showing when the Channel was
	// indexed
	IndexedAt string `protobuf:"bytes,5,opt,name=indexed_at,json=indexedAt,proto3" json:"indexed_at,omitempty"`
	// Location contains the geographical location of the Channel
	Location *Location `protobuf:"bytes,6,opt,name=location,proto3" json:"location,omitempty"`
	// Provider contains information about the data provider
	Provider *Provider `protobuf:"bytes,7,opt,name=provider,proto3" json:"provider,omitempty"`
	// Metadata contains an optional list of metadata about the Channel
	Metadata []*Metadata `protobuf:"bytes,8,rep,name=metadata,proto3" json:"metadata,omitempty"`
	// Visibility returns the visibility status of the Channel. Can be one of OPEN
	// - open API freely accessible with no authentication, SHARED - available via
	// a public API provided suitable authentication is supplied, or CLOSED - only
	// available via a private API. This is a required field.
	Visibility Visibility `protobuf:"varint,9,opt,name=visibility,proto3,enum=thingfulx.protobuf.Visibility" json:"visibility,omitempty"`
	// DataLicense returns the unique identifying URL of a specific data license
	// that the data provider has chosen to apply to their data. This is an
	// optional field.
	DataLicense *DataLicense `protobuf:"bytes,10,opt,name=data_license,json=dataLicense,proto3" json:"data_license,omitempty"`
	// Attributions contains a list of attributions we require to add to a
	// channel. Typically there will only be one, but some providers may require
	// multiple attributions.
	Attributions []*Attribution `protobuf:"bytes,11,rep,name=attributions,proto3" json:"attributions,omitempty"`
	// UpdateInterval returns the update frequency of the data resource expressed
	// as an integer number of seconds.
	UpdateInterval int32 `protobuf:"varint,12,opt,name=update_interval,json=updateInterval,proto3" json:"update_interval,omitempty"`
	// QuantityKind is used to describe the specific physical quantity kind this
	// channel measures. This will be expressed as a namespaced string, i.e.
	// m3-lite:AirTemperature where this compressed string can be expanded to URL
	// that uniquely identifies the quantity kind.
	QuantityKind string `protobuf:"bytes,13,opt,name=quantity_kind,json=quantityKind,proto3" json:"quantity_kind,omitempty"`
	// Unit is used to describe the unit in which the Channel publishes its data.
	// This will be expressed as a namespaced string, i.e. m3-lite:DegreeCelsius
	// which can be expanded to a URL that uniqely identifies the unit, and can be
	// used to obtain the symbol or to do conversions.
	Unit string `protobuf:"bytes,14,opt,name=unit,proto3" json:"unit,omitempty"`
	// DomainOfInterest is used to describe the broad category of the channel, and
	// as above will be expressed as a namespaced string, i.e.
	// m3-lite:Environment, which as above can be expanded.
	DomainOfInterest []string `protobuf:"bytes,15,rep,name=domain_of_interest,json=domainOfInterest,proto3" json:"domain_of_interest,omitempty"`
	// MeasuredBy is used to describe the type of sensor that was used to obtain
	// the channel's data. Expressed as a namespaced string, i.e.
	// m3-lite:Seismometer.
	MeasuredBy string `protobuf:"bytes,16,opt,name=measured_by,json=measuredBy,proto3" json:"measured_by,omitempty"`
	// DataType is used to encode the type of the value contained in each
	// observation.
	DataType DataType `protobuf:"varint,17,opt,name=data_type,json=dataType,proto3,enum=thingfulx.protobuf.DataType" json:"data_type,omitempty"`
	// Observations returns a list of data observation values that represent
	// concrete data recordings for the Channel.
	Observations         []*Observation `protobuf:"bytes,18,rep,name=observations,proto3" json:"observations,omitempty"`
	XXX_NoUnkeyedLiteral struct{}       `json:"-"`
	XXX_unrecognized     []byte         `json:"-"`
	XXX_sizecache        int32          `json:"-"`
}

Channel represents a single data channel being provided by an IoT device somewhere in the world.

func (*Channel) Descriptor added in v0.9.0 

func (*Channel) Descriptor() ([]byte, []int)

func (*Channel) GetAttributions added in v0.9.0 

func (m *Channel) GetAttributions() []*Attribution

func (*Channel) GetDataLicense added in v0.9.0 

func (m *Channel) GetDataLicense() *DataLicense

func (*Channel) GetDataType added in v0.9.0 

func (m *Channel) GetDataType() DataType

func (*Channel) GetDescription added in v0.9.0 

func (m *Channel) GetDescription() string

func (*Channel) GetDomainOfInterest added in v0.9.0 

func (m *Channel) GetDomainOfInterest() []string

func (*Channel) GetIndexedAt added in v0.9.0 

func (m *Channel) GetIndexedAt() string

func (*Channel) GetLocation added in v0.9.0 

func (m *Channel) GetLocation() *Location

func (*Channel) GetMeasuredBy added in v0.9.0 

func (m *Channel) GetMeasuredBy() string

func (*Channel) GetMetadata added in v0.9.0 

func (m *Channel) GetMetadata() []*Metadata

func (*Channel) GetObservations added in v0.9.0 

func (m *Channel) GetObservations() []*Observation

func (*Channel) GetProvider added in v0.10.0 

func (m *Channel) GetProvider() *Provider

func (*Channel) GetQuantityKind added in v0.9.0 

func (m *Channel) GetQuantityKind() string

func (*Channel) GetTitle added in v0.9.0 

func (m *Channel) GetTitle() string

func (*Channel) GetUnit added in v0.9.0 

func (m *Channel) GetUnit() string

func (*Channel) GetUpdateInterval added in v0.9.0 

func (m *Channel) GetUpdateInterval() int32

func (*Channel) GetUrl added in v0.9.0 

func (m *Channel) GetUrl() string

func (*Channel) GetVisibility added in v0.9.0 

func (m *Channel) GetVisibility() Visibility

func (*Channel) GetWebpage added in v0.9.0 

func (m *Channel) GetWebpage() string

func (*Channel) ProtoMessage added in v0.9.0 

func (*Channel) ProtoMessage()

func (*Channel) Reset added in v0.9.0 

func (m *Channel) Reset()

func (*Channel) String added in v0.9.0 

func (m *Channel) String() string

func (*Channel) XXX_DiscardUnknown added in v0.9.0 

func (m *Channel) XXX_DiscardUnknown()

func (*Channel) XXX_Marshal added in v0.9.0 

func (m *Channel) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Channel) XXX_Merge added in v0.9.0 

func (m *Channel) XXX_Merge(src proto.Message)

func (*Channel) XXX_Size added in v0.9.0 

func (m *Channel) XXX_Size() int

func (*Channel) XXX_Unmarshal added in v0.9.0 

func (m *Channel) XXX_Unmarshal(b []byte) error

type Client 

type Client interface {
	// DoHTTP is a thin wrapper around the `Do` method on `net/http.Client`. It
	// simply ensures we set the correct user agent header before making the
	// request. Note that unlike the wrapped standard library method we pass
	// context explicitly as the first parameter.
	DoHTTP(ctx context.Context, req *http.Request) (*http.Response, error)

	// Get is a helper function on the client interface for the most common usage
	// of the Client, making a simple Get request for a given URL, and returning a
	// slice of bytes containing the HTTP response. This uses client.DoHTTP
	// internally so that it ensures the correct user agent string is sent. Note
	// that unlike the wrapped standard library method we pass context explicitly
	// as the first parameter.
	GetHTTP(ctx context.Context, urlStr string) ([]byte, error)

	// PostHTTP defines a function for sending a POST request to a data
	// infrastructure in order to obtain data. Typically we send GET requests, but
	// some infrastructures may require POST requests. This function returns a
	// slice of bytes for any HTTP OK response, and an error for anything else. If
	// you require finer control of this you should use the DoHTTP method directly
	// which returns the http.Response for processing as required. Note that unlike
	// the wrapped standard library method we pass context explicitly as the first
	// parameter.
	PostHTTP(ctx context.Context, urlStr, contentType string, body io.Reader) ([]byte, error)
}

Client specifies an interface we will use for making outgoing requests within each Fetcher. Currently everything is HTTP, but that won't necessarily remain the case, so we add a layer of abstraction meaning we can add methods to our Client interface to support other protocols.

func NewClient 

func NewClient(userAgent string, timeout time.Duration) Client

NewClient is a constructor function that instantiates and returns a new client struct, ensuring it sets `timeout` properly on the returned http.Client

type Clock added in v0.6.0 

type Clock interface {
	// Now returns the current time as seen by the provider. This could be a fake
	// time for testing time providers.
	Now() time.Time
}

Clock is an interface that allows us to test time dependent code easily by providing an implementation of this interface that returns a 'canned' time rather than the real time.

func NewClock added in v0.6.0 

func NewClock() Clock

NewClock returns an instantiated DefaultClock instance.

func NewMockClock added in v0.6.0 

func NewMockClock() Clock

NewMockClock returns a new MockClock with the internal time set to the time at which the method was invoked.

func NewMockClockAt added in v0.6.1 

func NewMockClockAt(t time.Time) Clock

NewMockClockAt returns a new MockClock initialized to the passed in time variable.

func NewMockTimeProvider deprecated 

func NewMockTimeProvider(t time.Time) Clock

NewMockTimeProvider returns a mock time provider suitable for use in tests.

Deprecated: the TimeProvider type has been renamed to Clock, with matching mock instance constructors. New code should use the Clock versions.

type ContextKey 

type ContextKey string

ContextKey is a type alias used for context keys to avoid any issues wth clashing keys

type DataLicense 

type DataLicense struct {
	// Name is a human friendly label or name for the data license.
	Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
	// Url is a unique identifier for the license that should also be the URL
	// where the text of the license is available.
	Url string `protobuf:"bytes,2,opt,name=url,proto3" json:"url,omitempty"`
	// LegalCodeUrl is a link to a long or legal version of the license. This may
	// be the same as the above URL if no more detailed version is available.
	LegalCodeUrl string `protobuf:"bytes,3,opt,name=legal_code_url,json=legalCodeUrl,proto3" json:"legal_code_url,omitempty"`
	// Permits is a list of properties which define what usage the license
	// permits.
	Permits []string `protobuf:"bytes,4,rep,name=permits,proto3" json:"permits,omitempty"`
	// Requires is a list of properties which define what requirements the license
	// has.
	Requires []string `protobuf:"bytes,5,rep,name=requires,proto3" json:"requires,omitempty"`
	// Permits is a list of properties which define what the license prohibits
	Prohibits            []string `protobuf:"bytes,6,rep,name=prohibits,proto3" json:"prohibits,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

DataLicense is a message used to describe the properties of a data license

func GetDataLicense 

func GetDataLicense(licenseURL string) DataLicense

GetDataLicense is a function that returns a copy of a data license instance to make it easier to apply a standard license to a Thing. It takes in a license URL as a parameter, and returns an instantiated DataLicense instance representing that license or nil if the license is not defined in Thingfulx.

func (*DataLicense) Descriptor added in v0.9.0 

func (*DataLicense) Descriptor() ([]byte, []int)

func (*DataLicense) GetLegalCodeUrl added in v0.9.0 

func (m *DataLicense) GetLegalCodeUrl() string

func (*DataLicense) GetName added in v0.9.0 

func (m *DataLicense) GetName() string

func (*DataLicense) GetPermits added in v0.9.0 

func (m *DataLicense) GetPermits() []string

func (*DataLicense) GetProhibits added in v0.9.0 

func (m *DataLicense) GetProhibits() []string

func (*DataLicense) GetRequires added in v0.9.0 

func (m *DataLicense) GetRequires() []string

func (*DataLicense) GetUrl added in v0.9.0 

func (m *DataLicense) GetUrl() string

func (*DataLicense) ProtoMessage added in v0.9.0 

func (*DataLicense) ProtoMessage()

func (*DataLicense) Reset added in v0.9.0 

func (m *DataLicense) Reset()

func (*DataLicense) String added in v0.9.0 

func (m *DataLicense) String() string

func (*DataLicense) XXX_DiscardUnknown added in v0.9.0 

func (m *DataLicense) XXX_DiscardUnknown()

func (*DataLicense) XXX_Marshal added in v0.9.0 

func (m *DataLicense) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*DataLicense) XXX_Merge added in v0.9.0 

func (m *DataLicense) XXX_Merge(src proto.Message)

func (*DataLicense) XXX_Size added in v0.9.0 

func (m *DataLicense) XXX_Size() int

func (*DataLicense) XXX_Unmarshal added in v0.9.0 

func (m *DataLicense) XXX_Unmarshal(b []byte) error

type DataType added in v0.9.0 

type DataType int32

The DataType enum is used to encode information about the data type of the values returned in the repeated Observations list. 

const (
	DataType_NUMBER    DataType = 0
	DataType_INTEGER   DataType = 1
	DataType_STRING    DataType = 2
	DataType_DATE_TIME DataType = 3
	DataType_TIME      DataType = 4
	DataType_BOOLEAN   DataType = 5
	DataType_BYTES     DataType = 6
	DataType_UNKNOWN   DataType = 7
)

func (DataType) EnumDescriptor added in v0.9.0 

func (DataType) EnumDescriptor() ([]byte, []int)

func (DataType) String added in v0.9.0 

func (x DataType) String() string

type DefaultClock added in v0.6.0 

type DefaultClock struct{}

DefaultClock is an implementation of our TimeProvider interface that returns now directly from time.Now().

func (DefaultClock) Now added in v0.6.0 

func (c DefaultClock) Now() time.Time

Now returns the current time. In the default implementation this is just the value of time.Now()

type ErrBadData 

type ErrBadData struct {
	Msg string
}

ErrBadData is an error that can be returned for any bad data when parsing

func NewErrBadData 

func NewErrBadData(msg string) *ErrBadData

NewErrBadData is a constructor for creating ErrBadData instances

func (*ErrBadData) Error 

func (e *ErrBadData) Error() string

Error is the implementation of the default error interface. Returns the contained message

type ErrMissingConfig 

type ErrMissingConfig struct {
	Variable string
}

ErrMissingConfig is a simple struct for reporting a missing config variable. It contains a single Variable attribute which should contain the name of the missing variable.

func NewErrMissingConfig 

func NewErrMissingConfig(variable string) *ErrMissingConfig

NewErrMissingConfig is a constructor for creating ErrMissingConfig instances

func (*ErrMissingConfig) Error 

func (e *ErrMissingConfig) Error() string

Error is the implementation of the default error interface. Returns a simple string reporting the received status

type ErrUnexpectedResponse 

type ErrUnexpectedResponse struct {
	Status string
}

ErrUnexpectedResponse is a simple struct for reporting unexpected HTTP responses. It contains a single attribute which allows capturing the received HTTP status

func NewErrUnexpectedResponse 

func NewErrUnexpectedResponse(status string) *ErrUnexpectedResponse

NewErrUnexpectedResponse is a constructor for creating ErrUnexpectedResponse instances

func (*ErrUnexpectedResponse) Error 

func (e *ErrUnexpectedResponse) Error() string

Error is the implementation of the default error interface. Returns a simple string reporting the received status

type Error 

type Error string

Error is a type alias we use to create const possible error instances.

func (Error) Error 

func (e Error) Error() string

Error is the implementation of the error interface for our Error type.

type Indexer 

type Indexer interface {
	// Provider returns data for the current provider.
	Provider() *Provider

	// URLS returns the smallest set of URLs required to completely index the
	// upstream data provider. The returned values might be a single URL for hosts
	// that publish relatively few things that are all available via a single URL,
	// or it might be hundreds of thousands of values long for hosts that publish
	// data entirely on individual URLs.
	//
	// This function takes as parameters an instance of the Context interface for
	// request-scoped values or cancellation, a Client implementation which the
	// Indexer must use to make any outgoing requests, a delay Duration which
	// defines the minimum interval between consecutive requests to the
	// infrastructure. In addition the caller must pass in a channel by which the
	// function can return data to the caller. This is intended to allow the indexer
	// implementation to return chunks of data as it goes in order to support
	// infrastructures with hundreds of thousands of unique URLs to return.
	URLS(ctx context.Context, client Client, delay time.Duration, out chan<- URLData)

	// Fetch is our method that knows how to obtain raw data for an infrastructure,
	// and is therefore responsible for negotiating any authentication or protocol
	// weirdness when it comes to fetching data. It takes as parameters an instance
	// of the Context interface for request-scoped values and cancellation, the url
	// we want to get data from, a Client object that the indexer must use to
	// actually make the request, and a Clock to allow testing time related
	// functions. It returns the raw data from the infrastructure as a slice of
	// bytes for further parsing.
	Fetch(ctx context.Context, urlStr string, client Client, clock Clock) ([]byte, error)

	// Parse returns a slice of Channel objects extracted from that data source.
	// This function takes as parameters a slice of bytes representing the data
	// collected from the upstream data provider, the URL of the Channel being
	// indexed and a Clock instance used internally by the fetcher to record the
	// indexing time of the parser. This is to allow for easier testing. We
	// separate Parsing from Fetching as we have some systems that provide data to
	// us without having to be fetched from a remote HTTP source.
	Parse(rawData []byte, urlStr string, clock Clock) ([]*Channel, error)
}

Indexer is the main interface for things that know how to index and fetch resources from external data infrastructures and are thus responsible for handling any authentication requirements, parsing the returned data and generating normalized representations of IoT things to the caller. The interface attempts to be agnostic in terms of protocol so an implementing class is free to perform whatever steps required to get the data.

type IndexerBuilder 

type IndexerBuilder func() (Indexer, error)

IndexerBuilder is type definition for the constructor functions that new Indexer must implement.

type Location 

type Location struct {
	// Longitude contains the decimal longitude of a geographical location
	Longitude float32 `protobuf:"fixed32,1,opt,name=longitude,proto3" json:"longitude,omitempty"`
	// Latitude contains the decimal latitude of a geographical location
	Latitude             float32  `protobuf:"fixed32,2,opt,name=latitude,proto3" json:"latitude,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

Location is a message type used to describe a physical location

func (*Location) Descriptor added in v0.9.0 

func (*Location) Descriptor() ([]byte, []int)

func (*Location) GetLatitude added in v0.9.0 

func (m *Location) GetLatitude() float32

func (*Location) GetLongitude added in v0.9.0 

func (m *Location) GetLongitude() float32

func (*Location) ProtoMessage added in v0.9.0 

func (*Location) ProtoMessage()

func (*Location) Reset added in v0.9.0 

func (m *Location) Reset()

func (*Location) String added in v0.9.0 

func (m *Location) String() string

func (*Location) XXX_DiscardUnknown added in v0.9.0 

func (m *Location) XXX_DiscardUnknown()

func (*Location) XXX_Marshal added in v0.9.0 

func (m *Location) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Location) XXX_Merge added in v0.9.0 

func (m *Location) XXX_Merge(src proto.Message)

func (*Location) XXX_Size added in v0.9.0 

func (m *Location) XXX_Size() int

func (*Location) XXX_Unmarshal added in v0.9.0 

func (m *Location) XXX_Unmarshal(b []byte) error

type Metadata 

type Metadata struct {
	// Property contains a namespaced property of a channel that we wish to
	// describe.
	Property string `protobuf:"bytes,1,opt,name=property,proto3" json:"property,omitempty"`
	// Value contains the value we wish to assign to that property in order to
	// correctly describe some aspect of the Channel
	Value                string   `protobuf:"bytes,2,opt,name=value,proto3" json:"value,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

Metadata is a message type used to describe arbitrary metadata properties for a Channel

func (*Metadata) Descriptor added in v0.9.0 

func (*Metadata) Descriptor() ([]byte, []int)

func (*Metadata) GetProperty added in v0.9.0 

func (m *Metadata) GetProperty() string

func (*Metadata) GetValue added in v0.9.0 

func (m *Metadata) GetValue() string

func (*Metadata) ProtoMessage added in v0.9.0 

func (*Metadata) ProtoMessage()

func (*Metadata) Reset added in v0.9.0 

func (m *Metadata) Reset()

func (*Metadata) String added in v0.9.0 

func (m *Metadata) String() string

func (*Metadata) XXX_DiscardUnknown added in v0.9.0 

func (m *Metadata) XXX_DiscardUnknown()

func (*Metadata) XXX_Marshal added in v0.9.0 

func (m *Metadata) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Metadata) XXX_Merge added in v0.9.0 

func (m *Metadata) XXX_Merge(src proto.Message)

func (*Metadata) XXX_Size added in v0.9.0 

func (m *Metadata) XXX_Size() int

func (*Metadata) XXX_Unmarshal added in v0.9.0 

func (m *Metadata) XXX_Unmarshal(b []byte) error

type Observation 

type Observation struct {
	// RecordedAt records the timestamp at which the observation was recorded in
	// UTC and expressed as an RFC3339 string.
	RecordedAt string `protobuf:"bytes,1,opt,name=recorded_at,json=recordedAt,proto3" json:"recorded_at,omitempty"`
	// Each Observation must include its own location as this allows us to place
	// an observation in a particular place at a particular time which may be
	// different from the current location of the Channel.
	Location *Location `protobuf:"bytes,2,opt,name=location,proto3" json:"location,omitempty"`
	// Value includes the actual value of the Observation at the instant of time
	// encoded as a string.
	Value                string   `protobuf:"bytes,3,opt,name=value,proto3" json:"value,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

Observation is a messages used to record a single data value at a point in time for a channel

func (*Observation) Descriptor added in v0.9.0 

func (*Observation) Descriptor() ([]byte, []int)

func (*Observation) GetLocation added in v0.9.0 

func (m *Observation) GetLocation() *Location

func (*Observation) GetRecordedAt added in v0.9.0 

func (m *Observation) GetRecordedAt() string

func (*Observation) GetValue added in v0.9.0 

func (m *Observation) GetValue() string

func (*Observation) ProtoMessage added in v0.9.0 

func (*Observation) ProtoMessage()

func (*Observation) Reset added in v0.9.0 

func (m *Observation) Reset()

func (*Observation) String added in v0.9.0 

func (m *Observation) String() string

func (*Observation) XXX_DiscardUnknown added in v0.9.0 

func (m *Observation) XXX_DiscardUnknown()

func (*Observation) XXX_Marshal added in v0.9.0 

func (m *Observation) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Observation) XXX_Merge added in v0.9.0 

func (m *Observation) XXX_Merge(src proto.Message)

func (*Observation) XXX_Size added in v0.9.0 

func (m *Observation) XXX_Size() int

func (*Observation) XXX_Unmarshal added in v0.9.0 

func (m *Observation) XXX_Unmarshal(b []byte) error

type Provider 

type Provider struct {
	// Uid is a unique identifier for the data provider
	Uid string `protobuf:"bytes,1,opt,name=uid,proto3" json:"uid,omitempty"`
	// Name is a human friendly label for the data provider
	Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
	// Description is a longer form description of the data provider
	Description string `protobuf:"bytes,3,opt,name=description,proto3" json:"description,omitempty"`
	// Webpage should contain a link to a public webpage describing the provider
	Webpage              string   `protobuf:"bytes,4,opt,name=webpage,proto3" json:"webpage,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

Provider is a type used to describe the provider of a dataset

func (*Provider) Descriptor added in v0.10.0 

func (*Provider) Descriptor() ([]byte, []int)

func (*Provider) GetDescription added in v0.10.0 

func (m *Provider) GetDescription() string

func (*Provider) GetName added in v0.10.0 

func (m *Provider) GetName() string

func (*Provider) GetUid added in v0.10.0 

func (m *Provider) GetUid() string

func (*Provider) GetWebpage added in v0.10.0 

func (m *Provider) GetWebpage() string

func (*Provider) ProtoMessage added in v0.10.0 

func (*Provider) ProtoMessage()

func (*Provider) Reset added in v0.10.0 

func (m *Provider) Reset()

func (*Provider) String added in v0.10.0 

func (m *Provider) String() string

func (*Provider) XXX_DiscardUnknown added in v0.10.0 

func (m *Provider) XXX_DiscardUnknown()

func (*Provider) XXX_Marshal added in v0.10.0 

func (m *Provider) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Provider) XXX_Merge added in v0.10.0 

func (m *Provider) XXX_Merge(src proto.Message)

func (*Provider) XXX_Size added in v0.10.0 

func (m *Provider) XXX_Size() int

func (*Provider) XXX_Unmarshal added in v0.10.0 

func (m *Provider) XXX_Unmarshal(b []byte) error

type TimeProvider 

type TimeProvider = Clock

TimeProvider is an alias for our Clock interface for backwards compatibility

type URLData added in v0.4.0 

type URLData struct {
	// URLS is a slice of url strings representing a chunk of candidate resource
	// URLs for the target infrastructure.
	URLS []string

	// Err is used to allow the process to signal back an error to the caller. We
	// package it with the data so we can signal an error at any point.
	Err error
}

URLData is a struct used to send back data from the URLS method. We now send back data asynchronously via a channel so we want to package our data we are actually interested in (i.e. the slice of strings), along with any error that might happen.

type Visibility 

type Visibility int32

Visibility defines an access level for a resource using definitions provided by the ODI. Open means accessible without authentication, Shared is publicly accessible provided some registration and authentication process is followed, and Closed means the resource is private, only accessible with explicit permission. 

const (
	Visibility_OPEN   Visibility = 0
	Visibility_SHARED Visibility = 1
	Visibility_CLOSED Visibility = 3
)

func (Visibility) EnumDescriptor added in v0.9.0 

func (Visibility) EnumDescriptor() ([]byte, []int)

func (Visibility) String 

func (x Visibility) String() string

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.