dsv1

package
v2.0.0-a1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 24, 2022 License: BSD-3-Clause Imports: 13 Imported by: 0

README

dsv1

This module provides limited read support for v1 based dataset collections. It is intended to enable migration of data from a version 1 dataset collection to the current version of the dataset collection.

Notes

Version 1 dataset collections were exclusively pairtree based. The JSON objects are stored in the pairtree and the attached documents under that directory. The operational metadata as well as the general metadata is maintained in the collection.json file in the root collection folder. Frame data is also in the root folder. Frames do not need to migrate, they tend to be operationally empheral in practice. The JSON documents and their attachments do need to migrate.

If the version 1 collection is in good working order then the collections.json file can be used to remap the old collection to the new one. On the otherhand if it is not in good working order the pairtree itself can be used to derive the keys and for the objects in the collection. Leverage the pairtree itself is safeist way to proceed.

Documentation

Overview

dsv1 is a submodule of dataset package.

Authors R. S. Doiel, <rsdoiel@library.caltech.edu> and Tom Morrel, <tmorrell@library.caltech.edu>

Copyright (c) 2022, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Package dataset includes the operations needed for processing collections of JSON documents and their attachments.

Authors R. S. Doiel, <rsdoiel@library.caltech.edu> and Tom Morrel, <tmorrell@library.caltech.edu>

Copyright (c) 2022, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Index

Constants

View Source 
const (
	// Version 1.1.1-legacy
	Version = `1.1.1-legacy`

	// Asc is used to identify ascending sorts
	Asc = iota
	// Desc is used to identify descending sorts
	Desc = iota
)

Variables

This section is empty.

Functions

func Analyzer 

func Analyzer(collectionName string, verbose bool) error

Analyzer checks the collection version and analyzes current state of collection reporting on errors.

func IsCollection 

func IsCollection(p string) bool

IsCollection checks to see if a given path contains a collection.json file

func SetupV1TestCollection 

func SetupV1TestCollection(cName string, records map[string]map[string]interface{}) error

Types

type Collection 

type Collection struct {
	// DatasetVersion of the collection
	DatasetVersion string `json:"dataset,omitempty"`

	// Name (filename) of collection
	Name string `json:"name"`

	// KeyMap holds the document key to path in the collection
	KeyMap map[string]string `json:"keymap,omitempty"`

	// FrameMap is a list of frame names and with rel path to the frame defined in the collection
	FrameMap map[string]string `json:"frames,omitempty"`

	// Description describes what is in the collection.
	Description string `json:"description,omitempty"`

	// Created is the date/time the init command was run in
	// RFC1123 format.
	Created string `json:"created,omitempty"`

	// Version of collection being stored in semvar notation
	Version string `json:"version,omitempty"`

	// Contact info
	Contact string `json:"contact,omitempty"`

	// Author holds a list of PersonOrOrg
	Author []*PersonOrOrg `json:"author,omitempty"`

	// Contributors holds a list of PersonOrOrg
	Contributor []*PersonOrOrg `json:"contributor,omitempty"`

	// Funder holds a list of PersonOrOrg
	Funder []*PersonOrOrg `json:"funder,omitempty"`

	// DOI holds the digital object identifier if defined.
	DOI string `json:"doi,omitempty"`

	// License holds a pointer to the license information for
	// the collection. E.g. CC0 URL
	License string `json:"license,omitempty"`

	// Annotation is a map to any addition metadata associated with
	// the Collection's metadata.
	Annotation map[string]interface{} `json:"annotation,omitempty"`

	// Who is the person(s)/organization(s) that created the collection
	Who []string `json:"who,omitempty"`
	// What - description of collection
	What string `json:"what,omitempty"`
	// When - date associated with collection (e.g. 2021,
	// 2021-10, 2021-10-02), should map to an approx date like in
	// archival work.
	When string `json:"when,omitempty"`
	// Where - location (e.g. URL, address) of collection
	Where string `json:"where,omitempty"`
	// contains filtered or unexported fields
}

Collection is the container holding a pairtree containing JSON docs

func Open 

func Open(name string) (*Collection, error)

Open reads in a collection's metadata and returns and new collection structure or error.

``` 

var (
   c *Collection
   err error
)
c, err = dataset.Open("collection.ds")
if err != nil {
   // ... handle error
}
defer c.Close()

```

func (*Collection) Close 

func (c *Collection) Close() error

Close closes a collection, writing the updated keys to disc Close removes the "lock.pid" file in the collection root. Close is often called in conjunction with "defer" keyword.

``` 

c, err := dataset.Open("my_collection.ds")
if err != nil { // .. handle error ...
}
// do some stuff with the collection
if err := c.Close(); err != nil {
   // ... handle closing error ...
}

```

func (*Collection) DocPath 

func (c *Collection) DocPath(name string) (string, error)

DocPath returns a full path to a key or an error if not found

``` 

c, err := dataset.Open("my_collection.ds")
if err != nil {
   // ... handle error ...
}
defer c.Close()
key := "my-object-key"
docPath := c.DocPath(key)

```

func (*Collection) IsKeyNotFound 

func (c *Collection) IsKeyNotFound(e error) bool

IsKeyNotFound checks an error message and returns true if it is a key not found error.

func (*Collection) KeyExists 

func (c *Collection) KeyExists(key string) bool

KeyExists returns true if key is in collection's KeyMap, false otherwise 

var (
   c *dataset.Collection
)
// ... collection previously opened and assigned to "c" ...

key := "object-1"
if c.KeyExists(key) == true {
   // ... do something with the key ...
}

```

func (*Collection) Keys 

func (c *Collection) Keys() []string

Keys returns a list of keys in a collection 

var (
   c *dataset.Collection
   keys []string
)
// ... collection previously opened and assigned to "c" ...

keys := c.Keys()
for _, key := range keys {
   // ... do something with the list of keys ...
}

```

func (*Collection) Length 

func (c *Collection) Length() int

Length returns the number of keys in a collection 

var (
   c *dataset.Collection
)
// ... collection previously opened and assigned to "c" ...

l := c.Length()
// ... do something with the number of itemsin the collection ...

```

func (*Collection) MetadataJSON 

func (c *Collection) MetadataJSON() []byte

MetadataJSON() returns a collection's metadata fields as a JSON encoded byte array.

func (*Collection) Read 

func (c *Collection) Read(name string, data map[string]interface{}, cleanObject bool) error

Read finds the record in a collection, updates the data interface provide and if problem returns an error name must exist or an error is returned

``` 

var (
   c *dataset.Collection
)
// ... collection previously opened and assigned to "c" ...
key := "object-2"
obj, err := c.Read(key)
if err != nil { // ... handle error ...
}

```

func (*Collection) ReadJSON 

func (c *Collection) ReadJSON(name string) ([]byte, error)

ReadJSON finds a the record in the collection and returns the JSON source or an error.

``` 

var (
   c *Collection
)
// ... collection previously opened and assigned to "c" ...
key := "object-1"
src, err := c.ReadJSON(key)
if err != nil {
   // ... handle error ...
}
// ... do something with the JSON encoded "src" value ...

```

type PersonOrOrg 

type PersonOrOrg struct {
	// Type is either "Person" or "Organization"
	Type string `json:"@type,omitempty"`
	// ID is either an ORCID or ROR
	ID string `json:"@id,omitempty"`
	// Name of an organization, empty if person
	Name string `json:"name,omitempty"`
	// Given name for a person, empty of organization
	GivenName string `json:"givenName,omitempty"`
	// Family name for a person, empty of organization
	FamilyName string `json:"familyName,omitempty"`
	// Affiliation holds the intitution affiliation of a person.
	Affiliation []*PersonOrOrg `json:"affiliation,omitempty"`
	// Annotation holds custom fields, e.g. a grant number of a funder
	Annotation map[string]interface{} `json:"annotation,omitempty"`
}

PersonOrOrg holds a the description of a person or organizaion associated with the dataset collection. e.g. author, contributor or funder.

Source Files

View all Source files

Directories

Path Synopsis
tbl.go provides some utility functions to move string one and two demensional slices into/out of one and two deminsional slices.
 tbl.go provides some utility functions to move string one and two demensional slices into/out of one and two deminsional slices.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.