repo

package
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 9, 2018 License: GPL-3.0 Imports: 16 Imported by: 2

Documentation

Overview

Package repo represents a repository of qri information Analogous to a git repository, repo expects a rigid structure filled with rich types specific to qri. Lots of things in here take inspiration from the ipfs datastore interface: github.com/ipfs/go-datastore

Index

Constants

View Source 
const (
	// ETDsCreated represents a peer creating a dataset (either updating an existing or a new dataset)
	ETDsCreated = EventType("ds_created")
	// ETDsDeleted represents destroying a dataset. Peers should respect this and remove locally as well
	ETDsDeleted = EventType("ds_deleted")
	// ETDsRenamed represents changing a dataset's name. Peers should update their refstore
	ETDsRenamed = EventType("ds_renamed")
	// ETDsPinned represents a peer pinning a dataset to their local storage
	ETDsPinned = EventType("ds_pinned")
	// ETDsUnpinned represents a peer unpinnning a dataset from local storage
	ETDsUnpinned = EventType("ds_unpinned")
	// ETDsAdded represents adding a reference to another peer's dataset to their node
	ETDsAdded = EventType("ds_added")
	// ETTransformExecuted represents running a transformation
	ETTransformExecuted = EventType("tf_executed")
)

Variables

View Source 
var (
	// ErrNotFound is the err implementers should return when stuff isn't found
	ErrNotFound = fmt.Errorf("repo: not found")
	// ErrPeerIDRequired is for when a peerID is missing-but-expected
	ErrPeerIDRequired = fmt.Errorf("repo: peerID is required")
	// ErrPeernameRequired is for when a peername is missing-but-expected
	ErrPeernameRequired = fmt.Errorf("repo: peername is required")
	// ErrNameRequired is for when a name is missing-but-expected
	ErrNameRequired = fmt.Errorf("repo: name is required")
	// ErrPathRequired is for when a path is missing-but-expected
	ErrPathRequired = fmt.Errorf("repo: path is required")
	// ErrNameTaken is for when a name name is already taken
	ErrNameTaken = fmt.Errorf("repo: name already in use")
	// ErrRepoEmpty is for when the repo has no datasets
	ErrRepoEmpty = fmt.Errorf("repo: this repo contains no datasets")
	// ErrNotPinner is for when the repo doesn't have the concept of pinning as a feature
	ErrNotPinner = fmt.Errorf("repo: backing store doesn't support pinning")
	// ErrNoRegistry indicates no regsitry is currently configured
	ErrNoRegistry = fmt.Errorf("no configured registry")
	// ErrEmptyRef indicates that the given reference is empty
	ErrEmptyRef = fmt.Errorf("repo: empty dataset reference")
)
View Source 
var ErrRefSelectionNotSupported = fmt.Errorf("selection not supported")

ErrRefSelectionNotSupported is the expected error for when RefSelector interface is *not* implemented

Functions

func CanonicalizeDatasetRef added in v0.1.1 

func CanonicalizeDatasetRef(r Repo, ref *DatasetRef) error

CanonicalizeDatasetRef uses the user's repo to turn any local aliases into full dataset references using known canonical peernames and paths. If the provided reference is not in the local repo, still do the work of handling aliases, but return a repo.ErrNotFound error, which callers can respond to by possibly contacting remote repos.

func CanonicalizeProfile added in v0.3.0 

func CanonicalizeProfile(r Repo, ref *DatasetRef, need *NeedPeernameRenames) error

CanonicalizeProfile populates dataset DatasetRef ProfileID and Peername properties, changing aliases to known names, and adding ProfileID from a peerstore

func CompareDatasetRef 

func CompareDatasetRef(a, b DatasetRef) error

CompareDatasetRef compares two Dataset References, returning an error describing any difference between the two references

func DataNodes 

func DataNodes(nodes map[string]*dsgraph.Node) (ds map[string]bool)

DataNodes returns a map[path]bool of all raw data nodes

func Graph 

func Graph(r Repo) (map[string]*dsgraph.Node, error)

Graph generates a map of all paths on this repository pointing to dsgraph.Node structs with all links configured. This is potentially expensive to calculate. Best to do some caching.

func WalkRepoDatasets 

func WalkRepoDatasets(r Repo, visit func(depth int, ref *DatasetRef, err error) (bool, error)) error

WalkRepoDatasets visits every dataset in the history of a user's namespace Yes, this potentially a very expensive function to call, use sparingly.

Types

type DatasetRef 

type DatasetRef struct {
	// Peername of dataset owner
	Peername string `json:"peername,omitempty"`
	// ProfileID of dataset owner
	ProfileID profile.ID `json:"profileID,omitempty"`
	// Unique name reference for this dataset
	Name string `json:"name,omitempty"`
	// Content-addressed path for this dataset
	Path string `json:"path,omitempty"`
	// Dataset is a pointer to the dataset being referenced
	Dataset *dataset.DatasetPod `json:"dataset,omitempty"`
	// Published indicates whether this reference is listed as an available dataset
	Published bool `json:"published"`
}

DatasetRef encapsulates a reference to a dataset. This needs to exist to bind ways of referring to a dataset to a dataset itself, as datasets can't easily contain their own hash information, and names are unique on a per-repository basis. It's tempting to think this needs to be "bigger", supporting more fields, keep in mind that if the information is important at all, it should be stored as metadata within the dataset itself.

func MustParseDatasetRef added in v0.5.2 

func MustParseDatasetRef(refstr string) DatasetRef

MustParseDatasetRef panics if the reference is invalid. Useful for testing

func ParseDatasetRef 

func ParseDatasetRef(ref string) (DatasetRef, error)

ParseDatasetRef decodes a dataset reference from a string value It’s possible to refer to a dataset in a number of ways. The full definition of a dataset reference is as follows: 

dataset_reference = peer_name/dataset_name@peer_id/network/hash

we swap in defaults as follows, all of which are represented as empty strings: 

network - defaults to /ipfs/
hash - tip of version history (latest known commit)

these defaults are currently enforced by convention. TODO - make Dataset Ref parsing the responisiblity of the Repo interface, replacing empty strings with actual defaults

dataset names & hashes are disambiguated by checking if the input parses to a valid multihash after base58 decoding. through defaults & base58 checking the following should all parse: 

peer_name/dataset_name
/network/hash
peername
peer_id
@peer_id
@peer_id/network/hash

see tests for more exmples

TODO - add validation that prevents peernames from being valid base58 multihashes and makes sure hashes are actually valid base58 multihashes TODO - figure out how IPFS CID's play into this

func (DatasetRef) AliasString added in v0.2.0 

func (r DatasetRef) AliasString() (s string)

AliasString returns the alias components of a DatasetRef as a string

func (DatasetRef) DecodeDataset added in v0.3.1 

func (r DatasetRef) DecodeDataset() (*dataset.Dataset, error)

DecodeDataset returns a dataset.Dataset from the stored CodingDataset field

func (DatasetRef) Equal added in v0.1.1 

func (r DatasetRef) Equal(b DatasetRef) bool

Equal returns true only if Peername Name and Path are equal

func (DatasetRef) IsEmpty added in v0.1.1 

func (r DatasetRef) IsEmpty() bool

IsEmpty returns true if none of it's fields are set

func (DatasetRef) IsPeerRef 

func (r DatasetRef) IsPeerRef() bool

IsPeerRef returns true if only Peername is set

func (DatasetRef) Match added in v0.1.1 

func (r DatasetRef) Match(b DatasetRef) bool

Match checks returns true if Peername and Name are equal, and/or path is equal

func (DatasetRef) String 

func (r DatasetRef) String() (s string)

String implements the Stringer interface for DatasetRef

type Event added in v0.3.0 

type Event struct {
	Time   time.Time
	Type   EventType
	Ref    DatasetRef
	PeerID peer.ID
	Params interface{}
}

Event is a list of details for logging a query

type EventLog added in v0.3.0 

type EventLog interface {
	LogEvent(t EventType, ref DatasetRef) error
	Events(limit, offset int) ([]*Event, error)
	EventsSince(time.Time) ([]*Event, error)
}

EventLog keeps logs

type EventType added in v0.3.0 

type EventType string

EventType classifies types of events that can be logged

type MemEventLog added in v0.3.0 

type MemEventLog []*Event

MemEventLog is an in-memory implementation of the EventLog interface

func (MemEventLog) Events added in v0.3.0 

func (log MemEventLog) Events(limit, offset int) ([]*Event, error)

Events grabs a set of Events from the store

func (MemEventLog) EventsSince added in v0.3.0 

func (log MemEventLog) EventsSince(t time.Time) ([]*Event, error)

EventsSince produces a slice of all events since a given time

func (*MemEventLog) LogEvent added in v0.3.0 

func (log *MemEventLog) LogEvent(t EventType, ref DatasetRef) error

LogEvent adds a query entry to the store

func (*MemEventLog) LogEventDetails added in v0.3.2 

func (log *MemEventLog) LogEventDetails(t EventType, when int64, peerID peer.ID, ref DatasetRef, params interface{}) error

LogEventDetails adds an entry to the log TODO: Update LogEvent to work like this, update callers.

type MemRefstore added in v0.1.1 

type MemRefstore []DatasetRef

MemRefstore is an in-memory implementation of the Namestore interface

func (*MemRefstore) DeleteRef added in v0.1.1 

func (r *MemRefstore) DeleteRef(del DatasetRef) error

DeleteRef removes a name from the store

func (MemRefstore) GetRef added in v0.1.1 

func (r MemRefstore) GetRef(get DatasetRef) (ref DatasetRef, err error)

GetRef completes a reference with , refs can have either Path or Peername & Name specified, GetRef should fill out the missing pieces

func (*MemRefstore) PutRef added in v0.1.1 

func (r *MemRefstore) PutRef(put DatasetRef) error

PutRef adds a reference to the namestore. Only complete references may be added

func (MemRefstore) RefCount added in v0.1.1 

func (r MemRefstore) RefCount() (int, error)

RefCount returns the total number of names in the store

func (MemRefstore) References added in v0.1.1 

func (r MemRefstore) References(limit, offset int) ([]DatasetRef, error)

References grabs a set of names from the Store's namespace

type MemRepo 

type MemRepo struct {
	*MemRefstore
	*MemEventLog
	// contains filtered or unexported fields
}

MemRepo is an in-memory implementation of the Repo interface

func NewMemRepo 

func NewMemRepo(p *profile.Profile, store cafs.Filestore, ps profile.Store, rc *regclient.Client) (*MemRepo, error)

NewMemRepo creates a new in-memory repository

func (*MemRepo) Graph 

func (r *MemRepo) Graph() (map[string]*dsgraph.Node, error)

Graph gives the graph of objects in this repo

func (*MemRepo) PrivateKey added in v0.3.0 

func (r *MemRepo) PrivateKey() crypto.PrivKey

PrivateKey returns this repo's private key

func (*MemRepo) Profile 

func (r *MemRepo) Profile() (*profile.Profile, error)

Profile returns the peer profile for this repository

func (*MemRepo) Profiles added in v0.3.0 

func (r *MemRepo) Profiles() profile.Store

Profiles gives this repo's Peer interface implementation

func (*MemRepo) RefCache added in v0.3.0 

func (r *MemRepo) RefCache() Refstore

RefCache gives access to the ephemeral Refstore

func (*MemRepo) Registry added in v0.3.2 

func (r *MemRepo) Registry() *regclient.Client

Registry returns a client for interacting with a federated registry if one exists, otherwise nil

func (*MemRepo) SelectedRefs added in v0.5.0 

func (r *MemRepo) SelectedRefs() ([]DatasetRef, error)

SelectedRefs gives the current reference selection

func (*MemRepo) SetProfile added in v0.3.0 

func (r *MemRepo) SetProfile(p *profile.Profile) error

SetProfile updates this repo's profile

func (*MemRepo) SetSelectedRefs added in v0.5.0 

func (r *MemRepo) SetSelectedRefs(sel []DatasetRef) error

SetSelectedRefs sets the current reference selection

func (*MemRepo) Store 

func (r *MemRepo) Store() cafs.Filestore

Store returns the underlying cafs.Filestore for this repo

type NeedPeernameRenames added in v0.5.2 

type NeedPeernameRenames struct {
	Renames map[string]string
}

NeedPeernameRenames represents which peernames need to be renamed, and to what.

func NewNeedPeernameRenames added in v0.5.2 

func NewNeedPeernameRenames() NeedPeernameRenames

NewNeedPeernameRenames returns a new NeedPeerNameRenames struct.

type NodeList 

type NodeList struct {
	Nodes map[string]*dsgraph.Node
}

NodeList is a collection of nodes

type ProfileRef added in v0.3.0 

type ProfileRef struct {
	Peername  string     `json:"peername,omitempty"`
	ProfileID profile.ID `json:"profileID,omitempty"`
	// Profile data
	Profile *profile.Profile
}

ProfileRef encapsulates a reference to a peer profile It's main job is to connect peernames / profile ID's to profiles

func (ProfileRef) String added in v0.3.0 

func (r ProfileRef) String() (s string)

String implements the Stringer interface for PeerRef

type RefSelector added in v0.5.0 

type RefSelector interface {
	SetSelectedRefs([]DatasetRef) error
	SelectedRefs() ([]DatasetRef, error)
}

RefSelector is an interface for supporting reference selection a reference selection is a slice of references intended for using in dataset operations

type Refstore added in v0.1.1 

type Refstore interface {
	// PutRef adds a reference to the store. References must be complete with
	// Peername, Name, and Path specified
	PutRef(ref DatasetRef) error
	// GetRef "completes" a passed in alias (DatasetRef with at least Peername
	// and Name field specified), filling in missing fields with a stored ref
	// TODO - should we rename this to "CompleteRef"?
	GetRef(ref DatasetRef) (DatasetRef, error)
	// DeleteRef removes a reference from the store
	DeleteRef(ref DatasetRef) error
	// References returns a set of references from the store
	References(limit, offset int) ([]DatasetRef, error)
	// RefCount returns the number of references in the store
	RefCount() (int, error)
}

Refstore keeps a collection of dataset references, Refstores require complete references (with both alias and identifiers), and can carry only one of a given alias eg: putting peer/dataset@a/ipfs/b when a ref with alias peer/dataset is already in the store will overwrite the stored reference

type Repo 

type Repo interface {
	// All repositories wraps a content-addressed filestore as the cannonical
	// record of this repository's data. Store gives direct access to the
	// cafs.Filestore instance any given repo is using.
	Store() cafs.Filestore

	// All Repos must keep a Refstore, defining a store of known datasets
	Refstore
	// EventLog keeps a log of Profile activity for this repo
	EventLog

	// A repository must maintain profile information about the owner of this dataset.
	// The value returned by Profile() should represent the peer.
	Profile() (*profile.Profile, error)
	// SetProfile sets this repo's current profile. Profiles must contain a private key
	SetProfile(*profile.Profile) error
	// PrivateKey hands over this repo's private key
	// TODO - this is needed to create action structs, any way we can make this
	// privately-negotiated or created at init?
	PrivateKey() crypto.PrivKey
	// A repository must maintain profile information about encountered peers.
	// Decsisions regarding retentaion of peers is left to the the implementation
	// TODO - should rename this to "profiles" to separate from the networking
	// concept of a peer
	Profiles() profile.Store
	// Registry returns a client for interacting with a federated registry
	// if one exists, otherwise nil
	Registry() *regclient.Client
}

Repo is the interface for working with a qri repository qri repos are stored graph of resources:datasets, known peers, analytics data, change requests, etc. Repos are connected to a single peer profile. Repos must wrap an underlying cafs.Filestore, which is intended to act as the canonical store of state across all peers that this repo may interact with.

type SearchParams 

type SearchParams struct {
	Q             string
	Limit, Offset int
}

SearchParams encapsulates parameters provided to Searchable.Search

type Searchable 

type Searchable interface {
	Search(p SearchParams) ([]DatasetRef, error)
}

Searchable is an opt-in interface for supporting repository search

Source Files

View all Source files

Directories

Path Synopsis
Package gen contains routines that perform expensive cryptographic operations.
 Package gen contains routines that perform expensive cryptographic operations.
Package test contains a set of tests to ensure a repo implementation conforms to expected behaviors, calling RunRepoTests on a given repo implementation should pass all checks in order to properly work with Qri.
 Package test contains a set of tests to ensure a repo implementation conforms to expected behaviors, calling RunRepoTests on a given repo implementation should pass all checks in order to properly work with Qri.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.