remote

package
v0.9.13 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 12, 2020 License: GPL-3.0 Imports: 42 Imported by: 1

Documentation

Overview

Package remote implements syncronization between qri instances

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNoRemoteClient is returned when no client is allocated
	ErrNoRemoteClient = fmt.Errorf("remote: no client to make remote requests")
	// ErrRemoteNotFound indicates a specified remote couldn't be located
	ErrRemoteNotFound = fmt.Errorf("remote not found")
)
View Source 
var ErrNotImplemented = fmt.Errorf("not implemented")

ErrNotImplemented is returned for methods that are not implemented

Functions

func Address 

func Address(cfg *config.Config, name string) (addr string, err error)

Address extracts the address of a remote from a configuration for a given remote name

func OplogFromContext added in v0.9.1 

func OplogFromContext(ctx context.Context) (l *oplog.Log, ok bool)

OplogFromContext pulls an oplog value from

func PrintProgressBarsOnPushPull added in v0.9.10 

func PrintProgressBarsOnPushPull(w io.Writer, bus event.Bus)

PrintProgressBarsOnPushPull writes progress data to the given writer on push & pull. requires the event bus that a remote.Client is publishing on

func VerifySigParams added in v0.9.1 

func VerifySigParams(pubkey crypto.PubKey, params map[string]string) (bool, error)

VerifySigParams takes a public key and a map[string]string params and verifies the the signature is correct TODO (ramfox): should be refactored to be private once remotes have their own keystore and can make the replation between a pid and a public key on their own

Types

type Client 

type Client interface {
	// Feeds gets a named set of dataset feeds from a remote, for example a
	// "recent" feed containing a list of datasets the remote has added ordered
	// newest to oldest
	Feeds(ctx context.Context, remoteAddr string) (map[string][]dsref.VersionInfo, error)
	// Feed fetches a named feed of datasets
	Feed(ctx context.Context, remoteAddr, feedName string, page, pageSize int) ([]dsref.VersionInfo, error)
	// Preview fetches a size-bounded subset of a single dataset version,
	// summarizing the contents of the dataset version
	PreviewDatasetVersion(ctx context.Context, ref dsref.Ref, remoteAddr string) (*dataset.Dataset, error)
	// FetchLogs downloads logbook data on a dataset without storing the results
	// locally
	FetchLogs(ctx context.Context, ref dsref.Ref, remoteAddr string) (*oplog.Log, error)
	// NewRemoteRefResolver creates RefResolver backed by network requests to a
	// single remote
	NewRemoteRefResolver(addr string) dsref.Resolver

	// PushDataset synchronizes a dataset with a remote, synchronizing logbook
	// data  and pulling the dataset version specified by ref.Path
	PushDataset(ctx context.Context, ref dsref.Ref, remoteAddr string) error
	// PullDataset fetches & stores a dataset from a remote, synchronizing logbook
	// data and pulling the dataset version data associated with ref.Path
	PullDataset(ctx context.Context, ref *dsref.Ref, remoteAddr string) (*dataset.Dataset, error)
	// RemoveDataset removes a dataset from a remote entirely, delete logbook data
	// on the remote and requesting the remote drop all stored dataset versions
	RemoveDataset(ctx context.Context, ref dsref.Ref, remoteAddr string) error
	// RemoveDatasetVersion asks a remote to stop storing version data for a
	// dataset
	RemoveDatasetVersion(ctx context.Context, ref dsref.Ref, remoteAddr string) error

	// Done returns a channel that the client will send on when the client is
	// closed
	Done() <-chan struct{}
	// DoneErr gives any error that occured in the shutdown process
	DoneErr() error
	// Shutdown ends the client process early
	Shutdown() <-chan struct{}
}

Client connects to remotes to perform synchronization tasks

func NewClient 

func NewClient(ctx context.Context, node *p2p.QriNode, pub event.Publisher) (c Client, err error)

NewClient creates a remote client suitable for syncing peers

func NewMockClient added in v0.9.3 

func NewMockClient(ctx context.Context, node *p2p.QriNode, book *logbook.Book) (c Client, err error)

NewMockClient returns a mock remote client. context passed to NewMockClient MUST use the `Shutdown` method or cancel externally for proper cleanup

type Feeds added in v0.9.5 

type Feeds interface {
	// Feeds returns a set of feeds keyed by name, the number of results in each
	// feed, and the number of feeds themselves is up to the server
	Feeds(ctx context.Context, userID string) (map[string][]dsref.VersionInfo, error)
	// Feed fetches a bounded set of VersionInfos for a given feed name
	Feed(ctx context.Context, userID, name string, offset, limit int) ([]dsref.VersionInfo, error)
}

Feeds accesses streams of dataset VersionInfo's to browse. Feeds should be named by their defining characteristic (eg: "popular", "recent", etc.) to distinguish their intention. Feed names must be unique.

The precise behaviour of feeds if left up to the responder. Remotes can make any number of feeds available, and update those feeds with any frequency. A remote may construct feeds of datasets that they don't have data for, simply to assist in dataset discovery.

The userID argument is planned for future use. The Qri roadmap includes plans to implement access control some day, providing an identifier for the user requesting a feed will allow the provider to tailor feeds to show datasets that user may have priviledged access to.

type Hook 

type Hook func(ctx context.Context, pid profile.ID, ref dsref.Ref) error

Hook is a function called at specific points in the sync cycle hook contexts may be populated with request parameters

type LocalPreviews added in v0.9.10 

type LocalPreviews struct {
	repo.Repo
	// contains filtered or unexported fields
}

LocalPreviews implements the previews interface with a local repo

func (LocalPreviews) Preview added in v0.9.10 

func (rp LocalPreviews) Preview(ctx context.Context, _, refStr string) (*dataset.Dataset, error)

Preview gets a preview for a reference

func (LocalPreviews) PreviewComponent added in v0.9.10 

func (rp LocalPreviews) PreviewComponent(ctx context.Context, _, refStr, component string) (interface{}, error)

PreviewComponent gets a component for a reference & component name

type MockClient added in v0.9.3 

type MockClient struct {
	// contains filtered or unexported fields
}

MockClient is a remote client suitable for tests

func (*MockClient) Done added in v0.9.10 

func (c *MockClient) Done() <-chan struct{}

Done returns a channel that the client will send on when finished closing

func (*MockClient) DoneErr added in v0.9.10 

func (c *MockClient) DoneErr() error

DoneErr gives an error that occured during the shutdown process

func (*MockClient) Feed added in v0.9.10 

func (c *MockClient) Feed(ctx context.Context, remoteAddr, feedName string, page, pageSize int) ([]dsref.VersionInfo, error)

Feed is not implemented

func (*MockClient) Feeds added in v0.9.5 

func (c *MockClient) Feeds(ctx context.Context, remoteAddr string) (map[string][]dsref.VersionInfo, error)

Feeds is not implemented

func (*MockClient) FetchLogs added in v0.9.5 

func (c *MockClient) FetchLogs(ctx context.Context, ref dsref.Ref, remoteAddr string) (*oplog.Log, error)

FetchLogs is not implemented

func (*MockClient) NewRemoteRefResolver added in v0.9.9 

func (c *MockClient) NewRemoteRefResolver(addr string) dsref.Resolver

NewRemoteRefResolver mocks a ref resolver off a foreign logbook

func (*MockClient) PreviewDatasetVersion added in v0.9.10 

func (c *MockClient) PreviewDatasetVersion(ctx context.Context, ref dsref.Ref, remoteAddr string) (*dataset.Dataset, error)

PreviewDatasetVersion is not implemented

func (*MockClient) PullDataset added in v0.9.3 

func (c *MockClient) PullDataset(ctx context.Context, ref *dsref.Ref, remoteAddr string) (*dataset.Dataset, error)

PullDataset adds a reference to a dataset using test peer info

func (*MockClient) PullDatasetVersion added in v0.9.10 

func (c *MockClient) PullDatasetVersion(ctx context.Context, ref *dsref.Ref, remoteAddr string) error

PullDatasetVersion is not implemented

func (*MockClient) PushDataset added in v0.9.3 

func (c *MockClient) PushDataset(ctx context.Context, ref dsref.Ref, remoteAddr string) error

PushDataset is not implemented

func (*MockClient) PushLogs added in v0.9.3 

func (c *MockClient) PushLogs(ctx context.Context, ref dsref.Ref, remoteAddr string) error

PushLogs is not implemented

func (*MockClient) RemoveDataset added in v0.9.3 

func (c *MockClient) RemoveDataset(ctx context.Context, ref dsref.Ref, remoteAddr string) error

RemoveDataset is not implemented

func (*MockClient) RemoveDatasetVersion added in v0.9.10 

func (c *MockClient) RemoveDatasetVersion(ctx context.Context, ref dsref.Ref, remoteAddr string) error

RemoveDatasetVersion is not implemented

func (*MockClient) RemoveLogs added in v0.9.3 

func (c *MockClient) RemoveLogs(ctx context.Context, ref dsref.Ref, remoteAddr string) error

RemoveLogs is not implemented

func (*MockClient) Shutdown added in v0.9.12 

func (c *MockClient) Shutdown() <-chan struct{}

Shutdown allows you to close the client before the parent context closes

type Options 

type Options struct {
	// called when a client requests to push a dataset, before any data has been
	// received
	DatasetPushPreCheck Hook
	// called when a dataset has been pushed, but before it's saved
	DatasetPushFinalCheck Hook
	// called after successfully publishing a dataset version
	DatasetPushed Hook
	// called when a client has unpublished a dataset version
	DatasetRemovePreCheck Hook
	// called after a dataset version has been removed
	DatasetRemoved Hook
	// called before a version pull is permitted
	DatasetPullPreCheck Hook
	// called when a client pulls a dataset
	DatasetPulled Hook

	// called before any log data is accepted from a client
	LogPushPreCheck Hook
	// called after a log has been received by a client, before it's saved
	LogPushFinalCheck Hook
	// called after a log has been pushed
	LogPushed Hook
	// called before a log pull is allowed
	LogPullPreCheck Hook
	// called after a log has been pulled
	LogPulled Hook
	// called before a log remove is performed
	LogRemovePreCheck Hook
	// called after a log has been removed
	LogRemoved Hook

	// called before any feed data request is processed
	FeedPreCheck Hook
	// called before a preview request is processed
	PreviewPreCheck Hook

	// Use a custom feeds interface implementation. Default creates a Feeds
	// instance from node.Repo
	Feeds
	// Use a custom previews interface implementation. Default creates a
	// Previews instance from node.Repo
	Previews
	// Policy defines the access control for the remote
	Policy *access.Policy
}

Options encapsulates runtime configuration for a remote

type OptionsFunc added in v0.9.12 

type OptionsFunc func(o *Options)

OptionsFunc adjusts the behavior of the a remote when passed to NewRemote

func OptLoadPolicyFileIfExists added in v0.9.12 

func OptLoadPolicyFileIfExists(filename string) OptionsFunc

OptLoadPolicyFileIfExists checks for a policy at the given path and populates the remote.Options.Policy if so

func OptPolicy added in v0.9.12 

func OptPolicy(p *access.Policy) OptionsFunc

OptPolicy adds a policy to the remote options

type OtherPeer added in v0.9.12 

type OtherPeer struct {
	// contains filtered or unexported fields
}

OtherPeer represents another peer which the MockClient connects to

type Previews added in v0.9.5 

type Previews interface {
	Preview(ctx context.Context, userID, refStr string) (*dataset.Dataset, error)
	PreviewComponent(ctx context.Context, userID, refStr, component string) (interface{}, error)
}

Previews is an interface for generating constant-size summaries of dataset data

type Remote 

type Remote struct {
	Feeds    Feeds
	Previews Previews

	FeedPreCheck    Hook
	PreviewPreCheck Hook
	// contains filtered or unexported fields
}

Remote receives requests from other qri nodes to perform actions on their behalf

func NewRemote 

func NewRemote(node *p2p.QriNode, cfg *config.Remote, localResolver dsref.Resolver, opts ...OptionsFunc) (*Remote, error)

NewRemote creates a remote

func (*Remote) AddDefaultRoutes added in v0.9.5 

func (r *Remote) AddDefaultRoutes(mux *http.ServeMux)

AddDefaultRoutes attaches routes a remote client will expect to an HTTP muxer

func (*Remote) ComponentHTTPHandler added in v0.9.5 

func (r *Remote) ComponentHTTPHandler(prefix string) http.HandlerFunc

ComponentHTTPHandler handles dataset component requests over HTTP

func (*Remote) DsyncHTTPHandler 

func (r *Remote) DsyncHTTPHandler() http.HandlerFunc

DsyncHTTPHandler provides an http handler for dsync

func (*Remote) FeedHTTPHandler added in v0.9.5 

func (r *Remote) FeedHTTPHandler(prefix string) http.HandlerFunc

FeedHTTPHandler gives access a feed VersionInfos constructed by a remote

func (*Remote) FeedsHTTPHandler added in v0.9.5 

func (r *Remote) FeedsHTTPHandler() http.HandlerFunc

FeedsHTTPHandler provides access to the home feed

func (*Remote) GoOnline added in v0.9.10 

func (r *Remote) GoOnline(ctx context.Context) error

GoOnline abstracts startDsyncServer, which starts the remote http dsync server and adds the dsync protocol to the underlying host

func (*Remote) LogsyncHTTPHandler added in v0.9.1 

func (r *Remote) LogsyncHTTPHandler() http.HandlerFunc

LogsyncHTTPHandler provides an http handler for synchronizing logs

func (*Remote) Node added in v0.9.5 

func (r *Remote) Node() *p2p.QriNode

Node exposes this remote's QriNode

func (*Remote) Policy added in v0.9.12 

func (r *Remote) Policy() *access.Policy

Policy exposes this remote's access control policy

func (*Remote) PreviewHTTPHandler added in v0.9.5 

func (r *Remote) PreviewHTTPHandler(prefix string) http.HandlerFunc

PreviewHTTPHandler handles dataset preview requests over HTTP

func (*Remote) RefsHTTPHandler 

func (r *Remote) RefsHTTPHandler() http.HandlerFunc

RefsHTTPHandler handles requests for dataset references

func (*Remote) RemoveDataset added in v0.9.1 

func (r *Remote) RemoveDataset(ctx context.Context, params map[string]string) error

RemoveDataset handles requests to remove a dataset currently removes all versions of a dataset TODO (ramfox): add `gen` params that indicates how many versions of the dataset, starting with the most recent version, we should remove. This should remove the latest version of the dataset ref from the refstore and add the (n + 1)th to the refstore gen = -1 should indicate that we remove all the dataset versions

type RepoFeeds added in v0.9.5 

type RepoFeeds struct {
	repo.Repo
}

RepoFeeds implements the feed interface with a Repo

func (RepoFeeds) Feed added in v0.9.5 

func (rf RepoFeeds) Feed(ctx context.Context, userID, name string, offset, limit int) ([]dsref.VersionInfo, error)

Feed fetches a portion of an individual named feed

func (RepoFeeds) Feeds added in v0.9.5 

func (rf RepoFeeds) Feeds(ctx context.Context, userID string) (map[string][]dsref.VersionInfo, error)

Feeds returns a set of feeds keyed by name, fetching a few references for each available feed

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.