repo

package
v0.9.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 30, 2020 License: GPL-3.0 Imports: 20 Imported by: 2

Documentation

Overview

Package repo represents a repository of qri information Analogous to a git repository, repo expects a rigid structure filled with rich types specific to qri.

Index

Constants

This section is empty.

Variables

View Source 
var (

	// DefaultQriLocation is where qri data defaults to storing. The keyword $HOME
	// (and only $HOME) will be replaced with the current user home directory
	DefaultQriLocation = "$HOME/.qri"

	// ErrNotFound is the err implementers should return when stuff isn't found
	ErrNotFound = fmt.Errorf("repo: not found")
	// ErrNoHistory is the err implementers should return when no versions exist in history
	ErrNoHistory = fmt.Errorf("repo: no history")
	// ErrPeerIDRequired is for when a peerID is missing-but-expected
	ErrPeerIDRequired = fmt.Errorf("repo: peerID is required")
	// ErrPeernameRequired is for when a peername is missing-but-expected
	ErrPeernameRequired = fmt.Errorf("repo: peername is required")
	// ErrNoRepo is for when a repo is missing-but-expected
	ErrNoRepo = fmt.Errorf("repo: no repo found")
	// ErrNameRequired is for when a name is missing-but-expected
	ErrNameRequired = fmt.Errorf("repo: name is required")
	// ErrPathRequired is for when a path is missing-but-expected
	ErrPathRequired = fmt.Errorf("repo: path is required")
	// ErrNameTaken is for when a name name is already taken
	ErrNameTaken = fmt.Errorf("repo: name already in use")
	// ErrRepoEmpty is for when the repo has no datasets
	ErrRepoEmpty = fmt.Errorf("repo: this repo contains no datasets")
	// ErrNotPinner is for when the repo doesn't have the concept of pinning as a feature
	ErrNotPinner = fmt.Errorf("repo: backing store doesn't support pinning")
	// ErrNoRegistry indicates no regsitry is currently configured
	ErrNoRegistry = fmt.Errorf("no configured registry")
	// ErrEmptyRef indicates that the given reference is empty
	ErrEmptyRef = fmt.Errorf("repo: empty dataset reference")
)

Functions

func CanonicalizeDatasetRef added in v0.1.1 

func CanonicalizeDatasetRef(r Repo, ref *reporef.DatasetRef) error

CanonicalizeDatasetRef uses the user's repo to turn any local aliases into full dataset references using known canonical peernames and paths. If the provided reference is not in the local repo, still do the work of handling aliases, but return a repo.ErrNotFound error, which callers can respond to by possibly contacting remote repos.

func CanonicalizeProfile added in v0.3.0 

func CanonicalizeProfile(r Repo, ref *reporef.DatasetRef) error

CanonicalizeProfile populates dataset reporef.DatasetRef ProfileID and Peername properties, changing aliases to known names, and adding ProfileID from a peerstore

func CompareDatasetRef 

func CompareDatasetRef(a, b reporef.DatasetRef) error

CompareDatasetRef compares two Dataset References, returning an error describing any difference between the two references

func DeleteVersionInfoShim added in v0.9.9 

func DeleteVersionInfoShim(r Repo, ref dsref.Ref) (*dsref.VersionInfo, error)

DeleteVersionInfoShim is a shim for getting away from the old stack of DatasetRef, CanonicalizeDatasetRef, and RefStore while still safely interacting with the repo.Refstore API

func FlatbufferBytes added in v0.9.5 

func FlatbufferBytes(rs RefList) []byte

FlatbufferBytes turns refs into a byte slice of flatbuffer data

func GetVersionInfoShim added in v0.9.9 

func GetVersionInfoShim(r Repo, ref dsref.Ref) (*dsref.VersionInfo, error)

GetVersionInfoShim is a shim for getting away from the old DatasetRef, CanonicalizeDatasetRef, RefStore stack

func ListDatasetsShim added in v0.9.9 

func ListDatasetsShim(r Repo, offset, limit int) ([]dsref.VersionInfo, error)

ListDatasetsShim gets away from the old stack of DatasetRef, CanonicalizeDatasetRef and RefStore

func ListVersionInfoShim added in v0.9.10 

func ListVersionInfoShim(r Repo, offset, limit int) ([]dsref.VersionInfo, error)

ListVersionInfoShim wraps a call to References, converting results to a slice of VersionInfos

func MarshalFlatbuffer added in v0.9.5 

func MarshalFlatbuffer(r reporef.DatasetRef, builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes a ref to a builder

func ParseDatasetRef 

func ParseDatasetRef(ref string) (reporef.DatasetRef, error)

ParseDatasetRef decodes a dataset reference from a string value It’s possible to refer to a dataset in a number of ways. The full definition of a dataset reference is as follows: 

dataset_reference = peer_name/dataset_name@peer_id/network/hash

we swap in defaults as follows, all of which are represented as empty strings: 

network - defaults to /ipfs/
hash - tip of version history (latest known commit)

these defaults are currently enforced by convention. TODO - make Dataset Ref parsing the responisiblity of the Repo interface, replacing empty strings with actual defaults

dataset names & hashes are disambiguated by checking if the input parses to a valid multihash after base58 decoding. through defaults & base58 checking the following should all parse: 

peer_name/dataset_name
/network/hash
peername
peer_id
@peer_id
@peer_id/network/hash

see tests for more exmples

TODO - add validation that prevents peernames from being valid base58 multihashes and makes sure hashes are actually valid base58 multihashes TODO - figure out how IPFS CID's play into this

func PutVersionInfoShim added in v0.9.9 

func PutVersionInfoShim(r Repo, vi *dsref.VersionInfo) error

PutVersionInfoShim is a shim for getting away from old stack of DatasetRef, CanonicalizeDatasetRef, and RefStores while still safely interacting with the repo.Refstore API

func UnmarshalFlatbuffer added in v0.9.5 

func UnmarshalFlatbuffer(rfb *repofb.DatasetRef) (r reporef.DatasetRef, err error)

UnmarshalFlatbuffer decodes a job from a flatbuffer

Types

type MemRefstore added in v0.1.1 

type MemRefstore []reporef.DatasetRef

MemRefstore is an in-memory implementation of the Namestore interface

func (*MemRefstore) DeleteRef added in v0.1.1 

func (r *MemRefstore) DeleteRef(del reporef.DatasetRef) error

DeleteRef removes a name from the store

func (MemRefstore) GetRef added in v0.1.1 

func (r MemRefstore) GetRef(get reporef.DatasetRef) (ref reporef.DatasetRef, err error)

GetRef completes a reference with , refs can have either Path or Peername & Name specified, GetRef should fill out the missing pieces

func (*MemRefstore) PutRef added in v0.1.1 

func (r *MemRefstore) PutRef(put reporef.DatasetRef) error

PutRef adds a reference to the namestore. Only complete references may be added

func (MemRefstore) RefCount added in v0.1.1 

func (r MemRefstore) RefCount() (int, error)

RefCount returns the total number of names in the store

func (MemRefstore) References added in v0.1.1 

func (r MemRefstore) References(offset, limit int) ([]reporef.DatasetRef, error)

References grabs a set of names from the Store's namespace

type MemRepo 

type MemRepo struct {
	*MemRefstore
	// contains filtered or unexported fields
}

MemRepo is an in-memory implementation of the Repo interface

func NewMemRepo 

func NewMemRepo(ctx context.Context, p *profile.Profile, fs *muxfs.Mux, bus event.Bus) (*MemRepo, error)

NewMemRepo creates a new in-memory repository TODO (b5) - this constructor should have an event.bus argument

func (*MemRepo) Bus added in v0.9.10 

func (r *MemRepo) Bus() event.Bus

Bus accesses the repo's event bus

func (*MemRepo) Done added in v0.9.9 

func (r *MemRepo) Done() <-chan struct{}

Done returns a channel that the repo will send on when the repo is closed

func (*MemRepo) DoneErr added in v0.9.9 

func (r *MemRepo) DoneErr() error

DoneErr gives an error that occured during the shutdown process

func (*MemRepo) Dscache added in v0.9.5 

func (r *MemRepo) Dscache() *dscache.Dscache

Dscache returns a dscache

func (*MemRepo) Filesystem added in v0.7.0 

func (r *MemRepo) Filesystem() *muxfs.Mux

Filesystem gives access to the underlying filesystem

func (*MemRepo) Logbook added in v0.9.1 

func (r *MemRepo) Logbook() *logbook.Book

Logbook accesses the mem repo logbook

func (*MemRepo) PrivateKey added in v0.3.0 

func (r *MemRepo) PrivateKey() crypto.PrivKey

PrivateKey returns this repo's private key

func (*MemRepo) Profile 

func (r *MemRepo) Profile() (*profile.Profile, error)

Profile returns the peer profile for this repository

func (*MemRepo) Profiles added in v0.3.0 

func (r *MemRepo) Profiles() profile.Store

Profiles gives this repo's Peer interface implementation

func (*MemRepo) RefCache added in v0.3.0 

func (r *MemRepo) RefCache() Refstore

RefCache gives access to the ephemeral Refstore

func (*MemRepo) RemoveLogbook added in v0.9.1 

func (r *MemRepo) RemoveLogbook()

RemoveLogbook drops a MemRepo's logbook pointer. MemRepo gets used in tests a bunch, where logbook manipulation is helpful

func (*MemRepo) ResolveRef added in v0.9.9 

func (r *MemRepo) ResolveRef(ctx context.Context, ref *dsref.Ref) (string, error)

ResolveRef implements the dsref.Resolver interface

func (*MemRepo) SetFilesystem added in v0.8.0 

func (r *MemRepo) SetFilesystem(fs *muxfs.Mux)

SetFilesystem implements QFSSetter, currently used during lib contstruction

func (*MemRepo) SetLogbook added in v0.9.1 

func (r *MemRepo) SetLogbook(book *logbook.Book)

SetLogbook assigns MemRepo's logbook. MemRepo gets used in tests a bunch, where logbook manipulation is helpful

func (*MemRepo) SetProfile added in v0.3.0 

func (r *MemRepo) SetProfile(p *profile.Profile) error

SetProfile updates this repo's profile

func (*MemRepo) Store 

func (r *MemRepo) Store() cafs.Filestore

Store returns the underlying cafs.Filestore for this repo

type QFSSetter added in v0.8.0 

type QFSSetter interface {
	SetFilesystem(qfs.Filesystem)
}

QFSSetter sets a qfs.Filesystem the whole interface is a short-term hack that should only need to be called in one context: when lib is setting up a Repo TODO (b5): either decouple repo & qfs completely, or merge them

type RefList added in v0.9.5 

type RefList []reporef.DatasetRef

RefList is a list of refs

func UnmarshalRefsFlatbuffer added in v0.9.0 

func UnmarshalRefsFlatbuffer(data []byte) (ls RefList, err error)

UnmarshalRefsFlatbuffer turns a repo flatbuffer into a list of refs

func (RefList) Len added in v0.9.5 

func (rs RefList) Len() int

Len returns the length of refs. Used for sort.Interface

func (RefList) Less added in v0.9.5 

func (rs RefList) Less(i, j int) bool

Less returns true if i comes before j. Used for sort.Interface

func (RefList) Swap added in v0.9.5 

func (rs RefList) Swap(i, j int)

Swap flips the positions of i and j. Used for sort.Interface

type Refstore added in v0.1.1 

type Refstore interface {
	// PutRef adds a reference to the store. References must be complete with
	// Peername, Name, and Path/FSIPath specified
	PutRef(ref reporef.DatasetRef) error
	// GetRef "completes" a passed in alias (reporef.DatasetRef with at least Peername
	// and Name field specified), filling in missing fields with a stored ref
	GetRef(ref reporef.DatasetRef) (reporef.DatasetRef, error)
	// DeleteRef removes a reference from the store
	DeleteRef(ref reporef.DatasetRef) error
	// References returns a set of references from the store
	References(offset, limit int) ([]reporef.DatasetRef, error)
	// RefCount returns the number of references in the store
	RefCount() (int, error)
}

Refstore keeps a collection of dataset references, Refstores require complete references (with both alias and identifiers), and can carry only one of a given alias eg: putting peer/dataset@a/ipfs/b when a ref with alias peer/dataset is already in the store will overwrite the stored reference

type Repo 

type Repo interface {
	// repos are created with an event bus, and provid an accessor
	Bus() event.Bus

	// All repositories wraps a content-addressed filestore as the cannonical
	// record of this repository's data. Store gives direct access to the
	// cafs.Filestore instance any given repo is using.
	// TODO (b5) - deprecate store in favour of the Filesystem interface
	Store() cafs.Filestore

	// Filesystem is currently a read-only source of Filesystem-like data
	// Filestores can multiplex to read from multiple sources like the local
	// filesystem, over http, or content-addressed filesystems.
	// the long term-plan is to merge Filestore & Store
	Filesystem() *muxfs.Mux

	// A Repo can resolve dataset references it knows about locally from its
	// Refstore
	// NOTE (b5): this implementation will be dropped when the Refstore interface
	// is removed from the repo, delegating local ref resolution to dscache
	dsref.Resolver
	// All Repos must keep a Refstore, defining a store of known datasets
	// NOTE(dlong): Refstore is going away soon, everything is going to move to Dscache
	Refstore
	// Dscache is a cache of datasets that have been built according to logbook
	Dscache() *dscache.Dscache

	// Repos have a logbook for recording & storing operation logs
	Logbook() *logbook.Book

	// A repository must maintain profile information about the owner of this dataset.
	// The value returned by Profile() should represent the peer.
	Profile() (*profile.Profile, error)
	// SetProfile sets this repo's current profile. Profiles must contain a private key
	SetProfile(*profile.Profile) error
	// PrivateKey hands over this repo's private key
	PrivateKey() crypto.PrivKey
	// A repository must maintain profile information about encountered peers.
	// Decsisions regarding retentaion of peers is left to the the implementation
	Profiles() profile.Store

	// Done returns a channel that the repo will send on when the repo is closed
	Done() <-chan struct{}
	// DoneErr gives any error that occured in the shutdown process
	DoneErr() error
}

Repo is the interface for working with a qri repository qri repos are stored graph of resources:datasets, known peers, analytics data, change requests, etc. Repos are connected to a single peer profile. Repos must wrap an underlying cafs.Filestore, which is intended to act as the canonical store of state across all peers that this repo may interact with.

type SearchParams 

type SearchParams struct {
	Q             string
	Limit, Offset int
}

SearchParams encapsulates parameters provided to Searchable.Search

type Searchable 

type Searchable interface {
	Search(p SearchParams) ([]reporef.DatasetRef, error)
}

Searchable is an opt-in interface for supporting repository search

Source Files

View all Source files

Directories

Path Synopsis
Package buildrepo initializes a qri repo
 Package buildrepo initializes a qri repo
Package fsrepo is a file-system implementation of repo
 Package fsrepo is a file-system implementation of repo
Package gen contains routines that perform expensive cryptographic operations.
 Package gen contains routines that perform expensive cryptographic operations.
Package profile defines a qri peer profile
 Package profile defines a qri peer profile
spec
Package spec contains a set of tests to ensure a repo implementation conforms to expected behaviors, calling RunRepoTests on a given repo implementation should pass all checks in order to properly work with Qri.
 Package spec contains a set of tests to ensure a repo implementation conforms to expected behaviors, calling RunRepoTests on a given repo implementation should pass all checks in order to properly work with Qri.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.