pod

Documentation

Index

• Variables
• func NewAddCatalog() tasks.Taskable
• type AddCatalog
  • func (t *AddCatalog) Do(pch chan tasks.Progress)
  • func (t *AddCatalog) SetDatastore(store datastore.Datastore)
  • func (t *AddCatalog) Valid() error

Variables

var IpfsApiServerUrl = ""

Functions

func NewAddCatalog

func NewAddCatalog() tasks.Taskable

Types

type AddCatalog

type AddCatalog struct {
	// title for collection if no collection present
	CollectionTitle string
	// url that points to catalog
	Url string `json:"url"`
	// paginate into dataset list, zero is no pagination / offset
	Limit int
	// offset to start archiving at
	Offset int
	// how many fetching goroutines to spin up. max 5
	Parallelism int
	// skip items that already have a hash value
	SkipArchived bool
	// how long to sleep between requests in seconds(inside of parallel routines)
	CrawDelay time.Duration
	// contains filtered or unexported fields
}

AddCatalog injests a a collection to IPFS, it iterates through each setting hashes on collection urls and, eventually, generates a cdxj index of the archive

func (*AddCatalog) Do

func (t *AddCatalog) Do(pch chan tasks.Progress)

func (*AddCatalog) SetDatastore

func (t *AddCatalog) SetDatastore(store datastore.Datastore)

AddCatalog task needs to talk to an underlying database it's expected that the task executor will call this method before calling Do

func (*AddCatalog) Valid

func (t *AddCatalog) Valid() error