dsref

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 7, 2021 License: GPL-3.0 Imports: 11 Imported by: 2

Documentation

Overview

Package dsref defines structure and syntax for referring to a dataset

Index

Constants

View Source 
const AllGenerations = -1

AllGenerations represents all the generations of a dataset's history

View Source 
const PuncCharacters = "`~!@#$%^&*()=+[{]}\\|;:'\",<.>/?"

PuncCharacters is the list of punctuation characters that get converted to dashes

Variables

View Source 
var (

	// ErrEmptyRef is an error for when a reference is empty
	ErrEmptyRef = fmt.Errorf("empty reference")
	// ErrParseError is an error returned when parsing fails
	ErrParseError = fmt.Errorf("could not parse ref")
	// ErrUnexpectedChar is an error when a character is unexpected, topic string must be non-empty
	ErrUnexpectedChar = fmt.Errorf("unexpected character")
	// ErrNotHumanFriendly is an error returned when a reference is not human-friendly
	ErrNotHumanFriendly = fmt.Errorf("unexpected character '@', ref can only have username/name")
	// ErrBadCaseName is the error when a bad case is used in the dataset name
	ErrBadCaseName = fmt.Errorf("dataset name may not contain any upper-case letters")
	// ErrBadCaseUsername is for when a username contains upper-case letters
	ErrBadCaseUsername = fmt.Errorf("username may not contain any upper-case letters")
	// ErrBadCaseShouldRename is the error when a dataset should be renamed to not use upper case letters
	ErrBadCaseShouldRename = fmt.Errorf("dataset name should not contain any upper-case letters, rename it to only use lower-case letters, numbers, and underscores")
	// ErrDescribeValidName is an error describing a valid dataset name
	ErrDescribeValidName = fmt.Errorf("dataset name must start with a lower-case letter, and only contain lower-case letters, numbers, dashes, and underscore. Maximum length is 144 characters")
	// ErrDescribeValidUsername describes valid username
	ErrDescribeValidUsername = fmt.Errorf("username must start with a lower-case letter, and only contain lower-case letters, numbers, dashes, and underscores")
)
View Source 
var (
	// ErrRefNotFound must be returned by a ref resolver that cannot resolve a
	// given reference
	ErrRefNotFound = errors.New("reference not found")
	// ErrPathRequired should be returned by functions that require a reference
	// have a path value, but got none.
	// ErrPathRequired should *not* be returned by implentations of the
	// ResolveRef interface
	ErrPathRequired = errors.New("reference path value is required")
)
View Source 
var ErrNoHistory = fmt.Errorf("no history")

ErrNoHistory indicates a resolved reference has no HEAD path

View Source 
var NameMaxLength = 44

NameMaxLength is the maximum length of a name that will be generated

Functions

func ConvertVersionInfoToDataset deprecated added in v0.10.0 

func ConvertVersionInfoToDataset(info *VersionInfo) *dataset.Dataset

ConvertVersionInfoToDataset builds up a dataset from all the relevant VersionInfo fields.

Deprecated: This function is needed only for supporting Search functionality. Do not add new callers if possible.

func EnsureValidName added in v0.9.8 

func EnsureValidName(text string) error

EnsureValidName returns nil if the name is valid, and an error otherwise

func EnsureValidUsername added in v0.9.8 

func EnsureValidUsername(text string) error

EnsureValidUsername is the same as EnsureValidName but returns a different error

func GenerateName added in v0.9.8 

func GenerateName(input, prefix string) string

GenerateName converts the input into a valid dataset string, which starts with a lower-case letter, and only has letters, digits, dashes and underscores.

func IsRefString added in v0.9.5 

func IsRefString(text string) bool

IsRefString returns whether the string parses as a valid reference

func IsValidName added in v0.9.5 

func IsValidName(text string) bool

IsValidName returns whether the dataset name is valid

func NewParseError added in v0.9.12 

func NewParseError(template string, args ...interface{}) error

NewParseError returns a new ParseError, its parameters are a format string and arguments

Types

type Loader added in v0.9.9 

type Loader interface {
	// LoadDataset will parse the ref string, resolve it, and load the dataset
	// from whatever store contains it
	LoadDataset(ctx context.Context, refstr string) (*dataset.Dataset, error)
}

Loader loads datasets

type MemResolver added in v0.9.9 

type MemResolver struct {
	Username string
	RefMap   map[string]string
	IDMap    map[string]VersionInfo
}

MemResolver holds maps that can do a cheap version of dataset resolution, for tests

func NewMemResolver added in v0.9.9 

func NewMemResolver(username string) *MemResolver

NewMemResolver returns a new MemResolver

func (*MemResolver) GetInfo added in v0.9.9 

func (m *MemResolver) GetInfo(initID string) *VersionInfo

GetInfo returns a VersionInfo by initID, or nil if not found

func (*MemResolver) Put added in v0.9.9 

func (m *MemResolver) Put(info VersionInfo)

Put adds a VersionInfo to the resolver

func (*MemResolver) ResolveRef added in v0.9.9 

func (m *MemResolver) ResolveRef(ctx context.Context, ref *Ref) (string, error)

ResolveRef finds the identifier & head path for a dataset reference implements resolve.NameResolver interface

type ParseError added in v0.9.12 

type ParseError struct {
	Message string
}

ParseError is an error for when a dataset reference fails to parse

func (*ParseError) Error added in v0.9.12 

func (e *ParseError) Error() string

Error renders the ParseError as a string

type Ref 

type Ref struct {
	// InitID is the canonical identifer for a dataset history
	InitID string `json:"initID,omitempty"`
	// Username of dataset owner
	Username string `json:"username,omitempty"`
	// ProfileID of dataset owner
	// deprecated - avoid using this field, we're working towards removing it
	// generally profile IDs should come from request scopes, or be fetched from
	// stores of identity info (profile.Store)
	ProfileID string `json:"profileID,omitempty"`
	// Unique name reference for this dataset
	Name string `json:"name,omitempty"`
	// Content-addressed path for this dataset
	Path string `json:"path,omitempty"`
}

Ref is a reference to a dataset

func MustParse added in v0.9.7 

func MustParse(text string) Ref

MustParse parses a dsref from a string, or panics if it fails

func Parse added in v0.9.5 

func Parse(text string) (Ref, error)

Parse a reference from a string

func ParseHumanFriendly added in v0.9.5 

func ParseHumanFriendly(text string) (Ref, error)

ParseHumanFriendly parses a reference that only has a username and a dataset name

func ParsePeerRef added in v0.10.0 

func ParsePeerRef(text string) (Ref, error)

ParsePeerRef a reference from a string where the dataset name is non-mandatory

func (Ref) Alias 

func (r Ref) Alias() (s string)

Alias returns the alias components of a Ref as a string

func (Ref) Complete added in v0.9.10 

func (r Ref) Complete() bool

Complete returns true if all fields are populated

func (Ref) Copy added in v0.9.9 

func (r Ref) Copy() Ref

Copy duplicates a reference

func (Ref) Equals added in v0.9.5 

func (r Ref) Equals(t Ref) bool

Equals returns whether the reference equals another

func (Ref) Human added in v0.9.9 

func (r Ref) Human() string

Human returns the human-friendly representation of the reference example: some_user/my_dataset

func (Ref) IsEmpty added in v0.9.5 

func (r Ref) IsEmpty() bool

IsEmpty returns whether the reference is empty

func (Ref) IsPeerRef added in v0.10.0 

func (r Ref) IsPeerRef() bool

IsPeerRef returns true if only Peername is set

func (Ref) LegacyProfileIDString added in v0.10.0 

func (r Ref) LegacyProfileIDString() (s string)

LegacyProfileIDString serializes a ref in the form 

Username/Name@ProfileID/Path

Deprecated - don't add callers, use String or raw ref fields instead

func (Ref) String 

func (r Ref) String() (s string)

String implements the Stringer interface for Ref

func (Ref) VersionInfo added in v0.9.9 

func (r Ref) VersionInfo() VersionInfo

VersionInfo creates a new sparse VersionInfo from a reference

type Resolver added in v0.9.9 

type Resolver interface {
	// ResolveRef uses ref as an outParam, setting ref.ID and ref.Path on success
	// some implementations of name resolution may make network calls
	// the returned "source" value should be either an empty string, indicating
	// the source was resolved locally, or the multiaddress of the network
	// address that performed the resolution
	ResolveRef(ctx context.Context, ref *Ref) (source string, err error)
}

Resolver finds the identifier and HEAD path for a dataset reference

func ParallelResolver added in v0.9.9 

func ParallelResolver(resolvers ...Resolver) Resolver

ParallelResolver composes multiple resolvers into one resolver that runs in parallel when called, using the first resolver that doesn't return ErrRefNotFound

func SequentialResolver added in v0.9.9 

func SequentialResolver(resolvers ...Resolver) Resolver

SequentialResolver composes multiple resolvers into one that runs each resolver in sequence, using the first resolver that doesn't return ErrorNotFound

type Rev 

type Rev struct {
	// field scopt, currently can only be a component name, or the entire dataset
	Field string
	// the nth-generational ancestor of a history
	Gen int
}

Rev names a field of a dataset at a snapshot relative to the latest version in a history Much of this is inspired by git revisions: https://git-scm.com/docs/gitrevisions

Unlike git, Qri is aware of the underlying data model it's selecting against, so revisions can have conventional names for specifying fields of a dataset

func NewAllRevisions 

func NewAllRevisions() *Rev

NewAllRevisions returns a Rev struct that represents all revisions.

func ParseRev 

func ParseRev(rev string) (*Rev, error)

ParseRev turns a string into a revision

func ParseRevs 

func ParseRevs(str string) (revs []*Rev, err error)

ParseRevs turns a comma-separated list of revisions into a slice of revisions

type State added in v0.9.8 

type State int

State is used to handle the state machine that processes strings 

const (
	// StateNone is for being outside of a word, looking at spaces
	StateNone State = 0
	// StateLowerWord is for looking at a word made-up of lower-case letters
	StateLowerWord State = 1
	// StateFirstUpper is for looking at the first upper-case letter of a word
	StateFirstUpper State = 2
	// StateCapsWord is for looking at a word made up of all upper-case letters
	StateCapsWord State = 3
	// StateNumber is for looking at a sequence of digits
	StateNumber State = 4
	// StatePunc is for looking at punctuation characters
	StatePunc State = 5
	// StateExit is set by flush in order to exit the top-level loop immediately
	StateExit State = 6
)

type VersionInfo added in v0.9.5 

type VersionInfo struct {
	//
	// Key as a stable identifier
	//
	// InitID is derived from the logbook for the dataset
	InitID string `json:"initID,omitempty"`
	//
	// Fields from dsref.Ref
	//
	// Username of dataset owner
	Username string `json:"username,omitempty"`
	// ProfileID of dataset owner
	ProfileID string `json:"profileID,omitempty"`
	// Unique name reference for this dataset
	Name string `json:"name,omitempty"`
	// Content-addressed path for this dataset
	Path string `json:"path,omitempty"`
	//
	// State about the dataset that can change
	//
	// If true, this dataset has published versions
	Published bool `json:"published,omitempty"`
	// If true, this reference doesn't exist locally. Only makes sense if path is set, as this
	// flag refers to specific versions, not to entire dataset histories.
	Foreign bool `json:"foreign,omitempty"`
	//
	// Meta fields
	//
	// Title from the meta structure
	MetaTitle string `json:"metaTitle,omitempty"`
	// List of themes from the meta structure, comma-separated list
	ThemeList string `json:"themeList,omitempty"`
	//
	// Structure fields
	//
	// Size of the body in bytes
	BodySize int `json:"bodySize,omitempty"`
	// Num of rows in the body
	BodyRows int `json:"bodyRows,omitempty"`
	// Format of the body, such as "csv" or "json"
	BodyFormat string `json:"bodyFormat,omitempty"`
	// Number of errors from the structure
	NumErrors int `json:"numErrors,omitempty"`
	//
	// Commit fields
	//
	// Timestamp field from the commit
	CommitTime time.Time `json:"commitTime,omitempty"`
	// Title field from the commit
	CommitTitle string `json:"commitTitle,omitempty"`
	// Message field from the commit
	CommitMessage string `json:"commitMessage,omitempty"`
	//
	// About the dataset's history and location
	//
	// Number of versions that the dataset has
	NumVersions int `json:"numVersions,omitempty"`
	// FSIPath is this dataset's link to the local filesystem if one exists
	FSIPath string `json:"fsiPath,omitempty"`
	//
	// Run Fields
	//
	// RunID is derived from from either the Commit.RunID, field or the runID of a
	// failed run. In the latter case the Path value will be empty
	RunID string `json:"runID,omitempty"`
	// RunStatus is a string version of the run.Status enumeration. This value
	// will always be one of:
	//    ""|"waiting"|"running"|"succeeded"|"failed"|"unchanged"|"skipped"
	// RunStatus is not stored on a dataset version, and instead must come from
	// either run state or a cache of run state
	// it's of type string to follow the "plain old data" pattern
	RunStatus string `json:"runStatus,omitempty"`
	// RunDuration is how long the run took/has currently taken in nanoseconds
	// default value of 0 means no duration data is available.
	// RunDuration is not stored on a dataset version, and instead must come from
	// either run state or a cache of run state
	RunDuration int64 `json:"runDuration,omitempty"`
}

VersionInfo is an aggregation of fields from a dataset version for caching & listing purposes. VersionInfos are typically used when showing a list of datasets or a list of dataset versions ("qri list" and "qri log").

VersionInfos can also describe dataset versions that are being created or failed to create. In these cases the calculated VersionInfo.Path value must always equal the empty string.

If any fields are added to this struct, keep it in sync with: 

dscache/def.fbs       dscache
dscache/fill_info.go  func fillInfoForDatasets
repo/ref/convert.go   func ConvertToVersionInfo

If you are considering making major changes to VersionInfo, read this synopsis first: 

https://github.com/qri-io/qri/pull/1641#issuecomment-778521313

func ConvertDatasetToVersionInfo deprecated added in v0.9.5 

func ConvertDatasetToVersionInfo(ds *dataset.Dataset) VersionInfo

ConvertDatasetToVersionInfo assigns values form a dataset to a VersionInfo This function is a shim while we work on building up dscache as a store of VersionInfo.

Deprecated: Don't use this function for new code. Instead reference a VersionInfo that is stored somewhere, or write a function that builds a VersionInfo without needing a dataset

func NewVersionInfoFromRef added in v0.9.10 

func NewVersionInfoFromRef(ref Ref) VersionInfo

NewVersionInfoFromRef creates a sparse-populated VersionInfo from a dsref.Ref

func (*VersionInfo) Alias added in v0.9.5 

func (v *VersionInfo) Alias() string

Alias returns the alias components of a Ref as a string

func (VersionInfo) SimpleRef added in v0.9.5 

func (v VersionInfo) SimpleRef() Ref

SimpleRef returns a simple dsref.Ref

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.