metax

package
v0.10.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 28, 2020 License: GPL-3.0 Imports: 18 Imported by: 0

Documentation

Overview

Package metax provides a client for the CSC MetaX API.

Index

Constants

View Source
const (
	Verbose = false

	DatasetsEndpoint                = "/rest/datasets/"
	ChangeCumulativeStateEndpoint   = "/rpc/datasets/change_cumulative_state"
	RefreshDirectoryContentEndpoint = "/rpc/datasets/refresh_directory_content"
)
View Source
const (
	SchemaIda = "metax-ida"
	SchemaAtt = "metax-att"
)

These constants are used to identify the schema a dataset is using. When creating a new dataset, the schema will also determine the catalog the dataset belongs to.

View Source
const (
	// MetaxDatasetFamily is the dataset type for Fairdata datasets.
	MetaxDatasetFamily = 2
)

Variables

View Source
var (
	ErrInvalidContentType = errors.New("invalid content-type: expected json")
	ErrNotFound           = errors.New("not found")
	ErrIdRequired         = errors.New("dataset without id and not allowed to create")
	ErrInvalidId          = errors.New("invalid dataset id")
)
View Source
var (
	// IdentifierKey is the key pointing to the primary identifier, if any.
	IdentifierKey = "identifier"

	// NewVersionKey is the key to the value indicating a new version has been created.
	NewVersionKey = "new_version_created"

	// NewVersionId is the key holding the identifier of the new version or an empty string.
	NewVersionIdKey = "new_version_created.identifier"

	// Editor is the key pointing to Qvain's private object inside the dataset.
	EditorKey = "editor"

	// QvainId is the key with the Qvain record id.
	QvainIdKey = "record_id"

	// QvainId is the key with the identifier "qvain".
	QvainIdentifierKey = "identifier"

	// DateCreatedKey is the key for the Metax dataset creation timestamp.
	DateCreatedKey = "date_created"

	// DateModifiedKey is the key for the Metax dataset modification timestamp.
	DateModifiedKey = "date_modified"

	// DateDeprecatedKey is the key for the Metax dataset deprecation timestamp.
	DateDeprecatedKey = "date_deprecated"

	// DateRemovedKey is the key for the Metax dataset removal timestamp.
	DateRemovedKey = "date_removed"
)
View Source
var (
	// CatalogIdentifiers are used to identify schema in datasets imported from Metax.
	CatalogIdentifiers = map[string]string{
		"urn:nbn:fi:att:data-catalog-ida": SchemaIda,
		"urn:nbn:fi:att:data-catalog-att": SchemaAtt,
		"urn:nbn:fi:att:data-catalog-pas": SchemaIda,
	}
)

Functions

func CreatedNewVersion

func CreatedNewVersion(blob []byte) bool

CreatedNewVersion returns a boolean indicating whether the new version created key exists. Note this doesn't check if its value is valid.

func DisableHttps

func DisableHttps(svc *MetaxService)

func GetIdentifier

func GetIdentifier(blob []byte) string

func GetIdentifiers

func GetIdentifiers(blob []byte) (string, string, string)

GetIdentifiers returns the Metax identifier, Metax "new version" identifier and Qvain record_id fields.

func GetModificationDate

func GetModificationDate(blob []byte) time.Time

GetModificationDate returns the date of last modification (created, modified, deprecated, removed) for a dataset.

func GetQvainId

func GetQvainId(blob []byte) string

GetQvainId returns the Qvain id of the dataset from the editor object or empty string.

func IsPublished

func IsPublished(blob []byte) bool

func LoadMetaxDataset

func LoadMetaxDataset(ds *models.Dataset) models.TypedDataset

LoadMetaxDataset constructs an existing MetaxDataset from an existing base dataset.

func MaybeNewVersionId

func MaybeNewVersionId(blob []byte) string

MaybeNewVersionId returns the id of a new Metax version if a new version was created, otherwise an empty string.

func NewMetaxDataset

func NewMetaxDataset(creator uuid.UUID) (models.TypedDataset, error)

NewMetaxDataset creates a Metax dataset.

func WithLatestVersion

func WithLatestVersion(svc *MetaxService)

func WithStreaming

func WithStreaming(req *http.Request)

WithStreaming forces the API to return a streaming response without pagination.

Types

type ApiError

type ApiError struct {
	// contains filtered or unexported fields
}

func (*ApiError) Error

func (e *ApiError) Error() string

func (*ApiError) OriginalError

func (e *ApiError) OriginalError() json.RawMessage

func (*ApiError) StatusCode

func (e *ApiError) StatusCode() int

type DataCatalog

type DataCatalog struct {
	Identifier *string `json:"identifier"`
}

DataCatalog contains the catalog identifier

type DatasetOption

type DatasetOption func(*http.Request)

func Since

func Since(t time.Time) DatasetOption

Since is a dataset option that adds an if-modified-since header to dataset queries.

func WithOwner

func WithOwner(uid string) DatasetOption

WithOwner is a dataset option that restricts dataset queries to the Qvain owner set in the dataset's editor object.

func WithRemoved

func WithRemoved() DatasetOption

WithRemoved is a dataset option that filters dataset queries to those that have been removed.

func WithUser

func WithUser(id string) DatasetOption

WithUser is a dataset option that filters dataset queries to those that have the given user identifier set in the dataset.

type Editor

type Editor struct {
	Identifier *string `json:"identifier"`
	RecordId   *string `json:"record_id"`
	CreatorId  *string `json:"creator_id,omitempty"`
	OwnerId    *string `json:"owner_id,omitempty"`
	ExtId      *string `json:"fd_id,omitempty"`
}

Editor is the Go representation of the Editor object in a Metax record.

type LinkingError

type LinkingError struct {
	// contains filtered or unexported fields
}

LinkingError is a custom error type that adds the missing field name.

func NewLinkingError

func NewLinkingError(field ...string) *LinkingError

NewLinkingError creates a new LinkingError with the field name set to the argument, or marks the whole metadata block as missing if no argument given.

func (*LinkingError) Error

func (e *LinkingError) Error() string

Error satisfies the Error interface.

func (*LinkingError) Field

func (e *LinkingError) Field() string

Field returns the missing field name.

func (*LinkingError) IsNotMine

func (e *LinkingError) IsNotMine() bool

IsNotMine returns a boolean value indicating if the whole Qvain metadata block was missing.

type MetaxDataset

type MetaxDataset struct {
	*models.Dataset
}

MetaxDataset wraps a models.Dataset.

func (*MetaxDataset) CreateData

func (dataset *MetaxDataset) CreateData(family int, schema string, blob []byte, extra map[string]string) error

CreateData creates a dataset from template and merges set fields.

func (*MetaxDataset) UpdateData

func (dataset *MetaxDataset) UpdateData(family int, schema string, blob []byte, extra map[string]string) error

UpdateData creates a partial dataset JSON blob to patch an existing one with.

func (*MetaxDataset) ValidateCreated added in v0.10.3

func (dataset *MetaxDataset) ValidateCreated() error

ValidateCreated performs checks on created dataset.

func (*MetaxDataset) ValidateUpdated added in v0.10.3

func (dataset *MetaxDataset) ValidateUpdated(updated *models.Dataset) error

ValidateUpdated checks that updated dataset can be saved.

type MetaxOption

type MetaxOption func(*MetaxService)

func WithCredentials

func WithCredentials(user, pass string) MetaxOption

func WithInsecureCertificates

func WithInsecureCertificates(enable bool) MetaxOption

func WithLogger

func WithLogger(logger zerolog.Logger) MetaxOption

func WithUserAgent

func WithUserAgent(ua string) MetaxOption

type MetaxRawRecord

type MetaxRawRecord struct {
	json.RawMessage
}

MetaxRawRecord embeds a json.RawMessage containing an unparsed JSON []byte slice with the Metax dataset.

func (MetaxRawRecord) GetQvainId

func (raw MetaxRawRecord) GetQvainId(mrec *MetaxRecord) (*uuid.UUID, error)

IsNewDataset checks if the Metax record should be treated as a new dataset. A dataset is new if it doesn't have a Qvain id.

func (MetaxRawRecord) Record

func (raw MetaxRawRecord) Record() (*MetaxRecord, error)

Record unmarshals the raw JSON and validates it, returning either a partially parsed MetaxRecord or an error.

-wvh- NOTE: (2019-03-28) Validation disabled to allow creating new datasets.

func (MetaxRawRecord) ToQvain

func (raw MetaxRawRecord) ToQvain() (*models.Dataset, bool, error)

ToQvain converts a Metax record in raw JSON to a Qvain record using the values in the Editor object.

If the Editor metadata contains valid data, consider the dataset ours and populate (all) the Dataset struct fields; boolean New is false. If the Editor metadata does not exist, consider the dataset new and let the caller handle creation; the ids and ownership fields are not set. If the Editor metadata is invalid, return an error.

type MetaxRecord

type MetaxRecord struct {
	Id         int64  `json:"id"`
	Identifier string `json:"identifier"`

	DataCatalog *DataCatalog `json:"data_catalog"`

	MetadataProviderUser *string `json:"metadata_provider_user"`

	DateCreated  *time.Time `json:"date_created"`
	DateModified *time.Time `json:"date_modified"`

	Removed bool `json:"removed"`

	Editor *Editor `json:"editor"`

	ResearchDataset json.RawMessage `json:"research_dataset"`
	Contract        json.RawMessage `json:"contract"`
}

MetaxRecord is a helper struct to parse the fields we need from a Metax dataset.

func (*MetaxRecord) Validate

func (record *MetaxRecord) Validate() error

Validate checks if the Metax record contains the fields we need to identify the record (those below the `editor` key).

-wvh- NOTE: (2019-03-28) Deprecated to allow creating new datasets if there is no existing application metadata in the dataset.

type MetaxService

type MetaxService struct {
	// contains filtered or unexported fields
}

MetaxService represents the Metax API server.

func NewMetaxService

func NewMetaxService(host string, params ...MetaxOption) *MetaxService

NewMetaxService returns a Metax API client.

func (*MetaxService) ChangeCumulativeState added in v0.10.3

func (api *MetaxService) ChangeCumulativeState(ctx context.Context, identifier string, cumulativeState string) (newMetaxId string, err error)

ChangeCumulativeState calls Metax RPC for changing cumulative state of a dataset with given Metax identifier. Returns the new Metax identifier if a new dataset version was created.

func (*MetaxService) Create

func (api *MetaxService) Create(ctx context.Context, blob json.RawMessage) (json.RawMessage, error)

Create makes new datasets at the API endpoint. Deprecated: use Store().

func (*MetaxService) Datasets

func (api *MetaxService) Datasets(params ...DatasetOption) (*PaginatedResponse, error)

func (*MetaxService) Delete

func (api *MetaxService) Delete(ctx context.Context, blob json.RawMessage) error

Delete marks a dataset as removed in Metax.

func (*MetaxService) GetId

func (api *MetaxService) GetId(id string) (json.RawMessage, error)

GetId queries the dataset endpoint for a dataset with the given id.

func (*MetaxService) GetIdRemoved

func (api *MetaxService) GetIdRemoved(id string) (json.RawMessage, error)

GetId queries the dataset endpoint for a dataset with the given id.

func (*MetaxService) ReadStream deprecated

func (api *MetaxService) ReadStream(params ...DatasetOption) ([]MetaxRecord, error)

ReadStream queries the dataset endpoint with an unpaged request.

Deprecated: use ReadStreamChannel() for actual asynchronous stream processing.

func (*MetaxService) ReadStreamChannel

func (api *MetaxService) ReadStreamChannel(ctx context.Context, params ...DatasetOption) (int, chan *MetaxRawRecord, chan error, error)

ReadStreamChannel queries the dataset endpoint streaming the resulting datasets asynchronously throught a channel.

func (*MetaxService) RefreshDirectoryContent added in v0.10.3

func (api *MetaxService) RefreshDirectoryContent(ctx context.Context,
	datasetIdentifier string, directoryIdentifier string) (newMetaxId string, err error)

RefreshDirectoryContent calls Metax RPC for updating directory contents on a published dataset. If the directory has new files that aren't yet in the dataset, they will be added. Returns the new Metax identifier if a new dataset version was created.

func (*MetaxService) Store

func (api *MetaxService) Store(ctx context.Context, blob json.RawMessage, owner *models.User) (json.RawMessage, error)

Store sends – or "publishes" – a dataset to the Metax dataset API. If the dataset has no identifier yet, it is POSTed to the dataset endpoint as a new dataset; otherwise it is PUT to the endpoint for that specific dataset identifier. If the request was successful, the dataset will be returned; if the request failed, and the response was in JSON format, the API error will include that error body.

func (*MetaxService) UrlForId

func (api *MetaxService) UrlForId(id string) string

type PaginatedResponse

type PaginatedResponse struct {
	Count    int            `json:"count"`
	Next     string         `json:"next"`
	Previous string         `json:"previous"`
	Results  []*MetaxRecord `json:"results"`
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL