irdmtools

package module
v0.0.37 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 16, 2023 License: BSD-3-Clause Imports: 21 Imported by: 0

README

Institutional Repository Data Management Tools

This is a proof of concept set tools and Go packages for working with institutional repositories. Initial target is Invenio RDM.

The proof of concept is being developed around RDM's web services (e.g. REST API and OAI-PMH), PostgreSQL database and external metadata services (e.g. CrossRef, DataCite).

Caltech Library is using testing the prototype with continuous content migration, aggregation and metadata analysis.

Tools

rdmutil

This tool is for interacting with an Invenio RDM repository via RDM's REST and OAI-PMH API. It covers most the JSON API documented at https://inveniordm.docs.cern.ch/. This includes listing, submitting and managing records and draft records.

rdmutil configuration is read either from the envinronment or a JSON formated configuration file. See the man page for details.

eprint2rdm

This tool is migrating content from an EPrints repository via the EPrint REST API. It will retrieve an EPrint XML representation of the EPrint record and transform it into a JSON encded simplified record nearly compatible with Invenio RDM. See the man page for details.

doi2rdm

This tool will query the CrossRef API and convert a works record into a JSON structure compatible with an RDM record (e.g. to be inserted via an RDM API call). See the man page for details.

Requirements

  • An Invenio RDM deployment
  • To building the software and documentation
    • git
    • Go >= 1.20.4
    • Make (e.g. GNU Make)
    • Pandoc >= 3
  • For harvesting content

Quick install

If you're running on Linux, macOS or Raspberry Pi OS you may be able to installed irdmtools with the following curl command

curl https://caltechlibrary.github.io/irdmtools/installer.sh | sh

Installation from source

This codebase is speculative. It is likely to change and as issues are identified. To install you need to download the source code and compile it. Here's the steps I take to install irdmtools.

git clone git@github.com:caltechlibrary/irdmtools
cd irdmtools
make
make test
make install

Documentation

Overview

irdmtools is a package for working with institutional repositories and data management systems. Current implementation targets Invenio-RDM.

@author R. S. Doiel, <rsdoiel@caltech.edu> @author Tom Morrell, <tmorrell@caltech.edu>

Copyright (c) 2023, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

/ irdmtools is a package for working with institutional repositories and data management systems. Current implementation targets Invenio-RDM.

@author R. S. Doiel, <rsdoiel@caltech.edu> @author Tom Morrell, <tmorrell@caltech.edu>

Copyright (c) 2023, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

irdmtools is a package for working with institutional repositories and data management systems. Current implementation targets Invenio-RDM.

@author R. S. Doiel, <rsdoiel@caltech.edu> @author Tom Morrell, <tmorrell@caltech.edu>

Copyright (c) 2023, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

irdmtools is a package for working with institutional repositories and data management systems. Current implementation targets Invenio-RDM.

@author R. S. Doiel, <rsdoiel@caltech.edu> @author Tom Morrell, <tmorrell@caltech.edu>

Copyright (c) 2023, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

irdmtools is a package for working with institutional repositories and data management systems. Current implementation targets Invenio-RDM.

@author R. S. Doiel, <rsdoiel@caltech.edu> @author Tom Morrell, <tmorrell@caltech.edu>

Copyright (c) 2023, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

irdmtools is a package for working with institutional repositories and data management systems. Current implementation targets Invenio-RDM.

@author R. S. Doiel, <rsdoiel@caltech.edu> @author Tom Morrell, <tmorrell@caltech.edu>

Copyright (c) 2023, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

irdmtools is a package for working with institutional repositories and data management systems. Current implementation targets Invenio-RDM.

@author R. S. Doiel, <rsdoiel@caltech.edu> @author Tom Morrell, <tmorrell@caltech.edu>

Copyright (c) 2023, Caltech All rights not granted herein are expressly reserved by Caltech.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Index

Constants

View Source 
const (
	// Version number of release
	Version = "0.0.37"

	// ReleaseDate, the date version.go was generated
	ReleaseDate = "2023-08-16"

	// ReleaseHash, the Git hash when version.go was generated
	ReleaseHash = "f1279b1"

	LicenseText = `` /* 1430-byte string literal not displayed */

)

Variables

This section is empty.

Functions

func AddAdditionalTitles added in v0.0.3 

func AddAdditionalTitles(rec *simplified.Record, title *simplified.TitleDetail) error

func AddDate added in v0.0.5 

func AddDate(rec *simplified.Record, dt *simplified.DateType) error

func AddFunder 

func AddFunder(rec *simplified.Record, funder *simplified.Funder) error

func AddKeyword 

func AddKeyword(rec *simplified.Record, keyword string) error

func AddRelatedIdentifier added in v0.0.25 

func AddRelatedIdentifier(rec *simplified.Record, scheme string, identifier string) error

func AddRelatedIdentifiers added in v0.0.3 

func AddRelatedIdentifiers(rec *simplified.Record, identifiers []*simplified.Identifier) error

func AddRights added in v0.0.5 

func AddRights(rec *simplified.Record, rights []*simplified.Right) error

func AddSubject 

func AddSubject(rec *simplified.Record, subject string) error

func AddSubjects added in v0.0.5 

func AddSubjects(rec *simplified.Record, subjects []*simplified.Subject) error

func CheckDOI added in v0.0.34 

func CheckDOI(cfg *Config, doi string) ([]map[string]interface{}, error)

CheckDOI takes a DOI and does a lookup to see if there are any matching .pids.doi.indentifier values.

``` doi := "10.1126/science.82.2123.219" records, err := CheckDOI(cfg, doi) 

if err != nil {
   // ... handle error ...
}

for _, rec := ranges {
   // ... process results ...
}

```

func CheckWaitInterval added in v0.0.4 

func CheckWaitInterval(iTime time.Time, wait time.Duration) (time.Time, bool)

CheckWaitInterval checks to see if an interval of time has been met or exceeded. It returns the remaining time interval (possibly reset) and a boolean. The boolean is true when the time interval has been met or exceeded, false otherwise.

``` tot := len(something) // calculate the total number of items to process t0 := time.Now() iTime := time.Now() reportProgress := false 

for i, key := range records {
    // ... process stuff ...
    if iTime, reportProgress = CheckWaitInterval(rptTime, (30 * time.Second)); reportProgress {
        log.Printf("%s", ProgressETR(t0, i, tot))
    }
}

```

func CrosswalkCrossRefWork 

func CrosswalkCrossRefWork(cfg *Config, work *crossrefapi.Works, resourceTypeMap map[string]string, contributorTypeMap map[string]string) (*simplified.Record, error)

CrosswalkCrossRefWork takes a Works object from the CrossRef API and maps the fields into an simplified Record struct return a new struct or error.

func CrosswalkEPrintToRecord 

func CrosswalkEPrintToRecord(eprint *eprinttools.EPrint, rec *simplified.Record, resourceTypes map[string]string, contributorTypes map[string]string) error

CrosswalkEPrintToRecord implements a crosswalk between an EPrint 3.x EPrint XML record as struct to a Invenio RDM record as struct.

func DeleteEndpoint added in v0.0.31 

func DeleteEndpoint(cfg *Config, p string) ([]byte, error)

DeleteEndpoint takes an access token and endpoint path along with JSON source as payload and returns JSON source and error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") p := "api/records/qez01-2309a/draft" src, err := DeleteEndpoint(cfg.InvenioToken, p) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func DeleteFiles added in v0.0.29 

func DeleteFiles(cfg *Config, recordId string, filenames []string, debug bool) ([]byte, error)

DeleteFiles takes a configuration object and record id, and list of files and removes from a draft.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" fNames := ["article.pdf", "data.zip" ] // add files to draft record in JSON src, _ := os.ReadFile(fName) draft, err := DeleteFiles(cfg, id, fNames) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%+v\n", draft) ```

func DiscardDraft added in v0.0.29 

func DiscardDraft(cfg *Config, recordId string, debug bool) (map[string]interface{}, error)

DiscardDraft takes a configuration object and record id, contacts an RDM instance and deletes a draft of a record and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" debug := true _, err := DiscardDraft(cfg, id, debug) 

if err != nil {
   // ... handle error ...
}

```

func FmtHelp added in v0.0.11 

func FmtHelp(src string, appName string, version string, releaseDate string, releaseHash string) string

FmtHelp lets you process a text block with simple curly brace markup.

func GetAccess added in v0.0.29 

func GetAccess(cfg *Config, recordId string, accessType string) ([]byte, error)

GetAccess takes an acces token, a record id and optionally a access type. Returns either the access object or attribute if type is specified. Also returns an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" src, err := GetAccess(cfg.InvenioToken, id, "") 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func GetDraft added in v0.0.28 

func GetDraft(cfg *Config, id string) (map[string]interface{}, error)

GetDraft takes a configuration object and record id, contacts an RDM instance retrieves an existing draft of a record and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" draft, err := GetDraft(cfg, id) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%+v\n", draft) ```

func GetDraftFiles added in v0.0.31 

func GetDraftFiles(cfg *Config, recordId string, debug bool) (map[string]interface{}, error)

GetDraftFiles takes a configuration object and record id, contacts an RDM instance and returns the files metadata and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" debug := true entries, err := GetDraftFiles(cfg, id, debug) 

if err != nil {
   // ... handle error ...
}

```

func GetEPrint added in v0.0.3 

func GetEPrint(baseURL string, eprintID int, timeout time.Duration, retryCount int) (*eprinttools.EPrints, error)

GetEPrint fetches a single EPrint record via the EPrint REST API.

func GetEndpoint added in v0.0.29 

func GetEndpoint(cfg *Config, p string) ([]byte, error)

GetEndpoint takes an access token and endpoint path and returns JSON source and error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") p := "api/records/qez01-2309a/draft" src, err := GetEndpoint(cfg.InvenioToken, p) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func GetFile added in v0.0.28 

func GetFile(cfg *Config, id string, fName string) (*simplified.Entry, error)

GetFile takes a configuration object, record id and filename, contacts an RDM instance and returns the specific file metadata and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" fName := "article.pdf" entry, err := GetFile(cfg, id, fName) 

if err != nil {
   // ... handle error ...
}

```

func GetFiles added in v0.0.28 

func GetFiles(cfg *Config, recordId string, debug bool) (map[string]interface{}, error)

GetFiles takes a configuration object and record id, contacts an RDM instance and returns the files metadata and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" debug := true entries, err := GetFiles(cfg, id, debug) 

if err != nil {
   // ... handle error ...
}

```

func GetKeys added in v0.0.3 

func GetKeys(baseURL string, timeout time.Duration, retryCount int) ([]int, error)

GetKeys returns a list of eprint record ids from the EPrints REST API

func GetModifiedRecordIds 

func GetModifiedRecordIds(cfg *Config, start string, end string) ([]string, error)

GetModifiedRecordIds takes a configuration object, contacts am RDM instance and returns a list of ids created, deleted or updated in the time range specififed. I problem is encountered returns an error.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

NOTE: This method relies on OAI-PMH, this is a rate limited process so results can take quiet some time.

func GetRawRecord added in v0.0.5 

func GetRawRecord(cfg *Config, id string) (map[string]interface{}, error)

GetRawRecord takes a configuration object and record id, contacts an RDM instance and returns a map[string]interface{} record

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" mapRecord, err := GetRawRecord(cfg, id) 

if err != nil {
    // ... handle error ...
}

```

func GetRecord 

func GetRecord(cfg *Config, id string, draft bool) (*simplified.Record, error)

GetRecord takes a configuration object and record id, contacts an RDM instance and returns a simplified record and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" record, err := GetRecord(cfg, id, false) 

if err != nil {
   // ... handle error ...
}

```

func GetRecordIds 

func GetRecordIds(cfg *Config) ([]string, error)

GetRecordIds takes a configuration object, contacts am RDM instance and returns a list of ids and error.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

NOTE: This method relies on OAI-PMH, this is a rate limited process so results can take quiet some time.

func GetReview added in v0.0.31 

func GetReview(cfg *Config, recordId string, debug bool) (map[string]interface{}, error)

GetReview takes a configuration object, record id and returns an review object (which includes a request id) and error code.

func GetVersionLatest added in v0.0.28 

func GetVersionLatest(cfg *Config, id string) (map[string]interface{}, error)

GetVersionLatest takes a configuration object and record id, contacts an RDM instance and returns the versons metadata and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" versions, err := GetVersionLatest(cfg, id) 

if err != nil {
   // ... handle error ...
}

```

func GetVersions added in v0.0.28 

func GetVersions(cfg *Config, id string) (map[string]interface{}, error)

GetVersions takes a configuration object and record id, contacts an RDM instance and returns the versons metadata and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" versions, err := GetVersions(cfg, id) 

if err != nil {
   // ... handle error ...
}

```

func Harvest 

func Harvest(cfg *Config, fName string, debug bool) error

func LoadTypesMap added in v0.0.3 

func LoadTypesMap(fName string, mapTypes map[string]string) error

```

func NewDraft added in v0.0.30 

func NewDraft(cfg *Config, recordId string) (map[string]interface{}, error)

NewDraft takes a configuration object and record id, contacts an RDM instance and create a draft of an existing record and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id = "38rg4-36m04" draft, err := NewDraft(cfg, id) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%+v\n", draft) ```

func NewRecord added in v0.0.30 

func NewRecord(cfg *Config, src []byte) (map[string]interface{}, error)

NewRecord takes a configuration object and JSON record values. It contacts an RDM instance and create a new record return the JSON for the newly created record with a record id. When records are created they are in "draft" state.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") fName := "new_record.json" // A new record in JSON src, _ := os.ReadFile(fName) record, err := NewRecord(cfg, src) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%+v\n", record) ```

func NewRecordVersion added in v0.0.30 

func NewRecordVersion(cfg *Config, recordId string) (map[string]interface{}, error)

NewRecordVersion takes a configuration object and record id to create the new version draft. The returns JSON record values includes the new record id identifying the new version.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id = "38rg4-36m04" record, err := NewRecordVersion(cfg, id) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%+v\n", record) ```

func PatchEndpoint added in v0.0.30 

func PatchEndpoint(cfg *Config, p string, payload []byte) ([]byte, error)

PatchEndpoint takes an access token and endpoint path along with JSON source as payload and returns JSON source and error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") p := "api/records/qez01-2309a/draft" data := os.ReadFile("draft.json") src, err := PatchEndpoint(cfg.InvenioToken, p, data) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func PostEndpoint added in v0.0.30 

func PostEndpoint(cfg *Config, p string, payload []byte) ([]byte, error)

PostEndpoint takes an access token and endpoint path along with JSON source as payload and returns JSON source and error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") p := "api/records/qez01-2309a/draft" data := os.ReadFile("draft.json") src, err := PostEndpoint(cfg.InvenioToken, p, data) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func ProgressETR added in v0.0.4 

func ProgressETR(t0 time.Time, i int, tot int) string

ProgressETR returns a string with the percentage processed and estimated time remaining. It requires the a counter of records processed, the total count of records and a time zero value.

``` tot := len(something) // calculate the total number of items to process t0 := time.Now() iTime := time.Now() reportProgress := false 

for i, key := range records {
    // ... process stuff ...
    if iTime, reportProgress = CheckWaitInterval(rptTime, (30 * time.Second)); reportProgress {
        log.Printf("%s", ProgressETR(t0, i, tot))
    }
}

```

func ProgressIPS added in v0.0.4 

func ProgressIPS(t0 time.Time, i int, timeUnit time.Duration) string

ProgressIPS returns a string with the elapsed time and increments per second. Takes a time zero, a counter and time unit. Returns a string with count, running time and increments per time unit. ``` t0 := time.Now() iTime := time.Now() reportProgress := false 

for i, key := range records {
    // ... process stuff ...
    if iTime, reportProgress = CheckWaitInterval(rptTime, (30 * time.Second)); reportProgress || i = 0 {
        log.Printf("%s", ProgressIPS(t0, i, time.Second))
    }
}

```

func PublishRecordVersion added in v0.0.32 

func PublishRecordVersion(cfg *Config, recordId string, version string, pubDate string, debug bool) (map[string]interface{}, error)

PublishRecordVersion takes a configuration object and record id of a new version draft and publishes it. NOTE: creating a new version will clear .metadata.publication_date, version label and DOI. These can be replace when publishing in the version and pubDate parameter. If those values are empty string no change is made to the draft before publishing.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id = "38rg4-36m04" version, pubDate := "internal", "2022-08" record, err := PublicRecordVersion(cfg, id, version, pubDate) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%+v\n", record) ```

func PutEndpoint added in v0.0.30 

func PutEndpoint(cfg *Config, p string, payload []byte) ([]byte, error)

PutEndpoint takes an access token and endpoint path along with JSON source as payload and returns JSON source and error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") p := "api/records/qez01-2309a/draft" data := os.ReadFile("draft.json") src, err := PutEndpoint(cfg.InvenioToken, p, data) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func Query 

func Query(cfg *Config, q string, sort string) ([]map[string]interface{}, error)

Query takes a query string and returns the paged object results as a slice of `map[string]interface{}`

``` records, err := Query(cfg, "Geological History in Southern California", "newest") 

if err != nil {
    // ... handle error ...
}

for _, rec := ranges {
    // ... process results ...
}

```

func QueryCrossRefWork 

func QueryCrossRefWork(cfg *Config, doi string, mailTo string, dotInitials bool, downloadDocument bool) (*crossrefapi.Works, error)

func QueryDataCite 

func QueryDataCite(cfg *Config, doi string, mailTo string, dotInitials bool, downloadDocument bool, debug bool) (map[string]interface{}, error)

func RetrieveFile added in v0.0.28 

func RetrieveFile(cfg *Config, id string, fName string) ([]byte, error)

RetrieveFile takes a configuration object, record id and filename, contacts an RDM instance and returns the specific file and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" fName := "article.pdf" data, err := RetrieveFile(cfg, id, fName) 

if err != nil {
   // ... handle error ...
}

os.WriteFile(fName, data, 0664) ```

func ReviewRequest added in v0.0.31 

func ReviewRequest(cfg *Config, recordId string, decision string, comment string, debug bool) (map[string]interface{}, error)

ReviewRequest takes a configuration object and record id, a decision, and optional comment contacts an RDM instance and updates the review status for the submitted draft record.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" debug := true _, err := ReviewDraft(cfg, id, "accept", "", debug) 

if err != nil {
   // ... handle error ...
}

```

func SampleConfig 

func SampleConfig(configFName string) ([]byte, error)

SampleConfig display a minimal configuration for the rdmutil cli. The minimal values in the configuration are "invenio_api" url and "invenio_token" holding the access token.

``` 

src, err := SampleConfig("irdmtools.json")
if err != nil {
    // ... handle error ...
}
fmt.Printf("%s\n", src)

```

func SendToCommunity added in v0.0.30 

func SendToCommunity(cfg *Config, recordId string, communityId string, debug bool) (map[string]interface{}, error)

SendToCommunity sends a draft to an RDM community. This will trigger the review step need for publication. You need the record id and a community id (looks like a UUID). Returns a map[string]interface{} and error values.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" comminityId := ... // this is a UUID like value debug := true src, err := SendToCommunity(cfg, id, communityId, debug) 

if err != nil {
   // ... handle error ...
}

```

func SetAccess added in v0.0.29 

func SetAccess(cfg *Config, recordId string, accessType string, accessValue string, debug bool) ([]byte, error)

SetAccess takes an access token, record id, a access type and value. Returns the updated access object and error value.

FIXME: Current this method only supports setting record and files attributes to "public" and "restricted". Future implementations may add support to set record embargos.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" debug := true src, err := SetAccess(cfg.InvenioToken, id, "", debug) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func SetArticleNumber 

func SetArticleNumber(rec *simplified.Record, articleNo string) error

func SetContributors added in v0.0.3 

func SetContributors(rec *simplified.Record, contributors []*simplified.Creator) error

func SetCreators added in v0.0.3 

func SetCreators(rec *simplified.Record, creators []*simplified.Creator) error

func SetCustomField added in v0.0.23 

func SetCustomField(rec *simplified.Record, customField string, key string, value interface{}) error

func SetDOI 

func SetDOI(rec *simplified.Record, doi string) error

Wraps the simplified package with crosswalks

func SetDescription 

func SetDescription(rec *simplified.Record, description string) error

func SetEdition 

func SetEdition(rec *simplified.Record, edition string) error

func SetFilesEnable added in v0.0.30 

func SetFilesEnable(cfg *Config, recordId string, enable bool, debug bool) (map[string]interface{}, error)

SetFilesEnable will set the metadata.files.enable value.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" enabled := true debug := true _, err := SetFilesEnable(cfg, id, enabled, debug) 

if err != nil {
   // ... handle error ...
}

```

func SetFullTextStatus 

func SetFullTextStatus(rec *simplified.Record, status bool) error

func SetFunding 

func SetFunding(rec *simplified.Record, funding []*simplified.Funder) error

func SetImprintField added in v0.0.25 

func SetImprintField(rec *simplified.Record, key string, value interface{}) error

func SetIssue added in v0.0.16 

func SetIssue(rec *simplified.Record, issue string) error

func SetJournalField added in v0.0.23 

func SetJournalField(rec *simplified.Record, key string, value interface{}) error

func SetLanguages added in v0.0.27 

func SetLanguages(rec *simplified.Record, key string, value interface{}) error

func SetMonographType 

func SetMonographType(rec *simplified.Record, monographType string) error

func SetPageRange 

func SetPageRange(rec *simplified.Record, pageRange string) error

func SetPresentationType added in v0.0.3 

func SetPresentationType(rec *simplified.Record, presentationType string) error

func SetProject 

func SetProject(rec *simplified.Record, project string) error

func SetPubDate added in v0.0.32 

func SetPubDate(cfg *Config, recordId string, pubDate string, debug bool) (map[string]interface{}, error)

SetPubDate will set the metadata.publication_date value.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" pubDate := "2016" debug := true _, err := SetPublicationDate(cfg, id, pubDate, debug) 

if err != nil {
   // ... handle error ...
}

```

func SetPublication 

func SetPublication(rec *simplified.Record, publication string) error

func SetPublicationDate added in v0.0.5 

func SetPublicationDate(rec *simplified.Record, pubDate string) error

func SetPublicationDateByType added in v0.0.25 

func SetPublicationDateByType(rec *simplified.Record, dt string, publicationType string) error

func SetPublisher 

func SetPublisher(rec *simplified.Record, publisher string) error

func SetPublisherLocation 

func SetPublisherLocation(rec *simplified.Record, place string) error

func SetReferred 

func SetReferred(rec *simplified.Record, referred bool) error

func SetResourceType 

func SetResourceType(rec *simplified.Record, resourceType string, resourceTypeMap map[string]string) error

func SetSeries 

func SetSeries(rec *simplified.Record, series string) error

func SetTitle added in v0.0.3 

func SetTitle(rec *simplified.Record, title string) error

func SetVersion added in v0.0.32 

func SetVersion(cfg *Config, recordId string, version string, debug bool) (map[string]interface{}, error)

SetVersion will set the metadata.version value.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" versoin := 'public files' debug := true _, err := SetVersion(cfg, id, version, debug) 

if err != nil {
   // ... handle error ...
}

```

func SetVolume 

func SetVolume(rec *simplified.Record, volume string) error

func UpdateDraft added in v0.0.28 

func UpdateDraft(cfg *Config, recordId string, payloadSrc []byte, debug bool) (map[string]interface{}, error)

UpdateDraft takes a configuration object and record id, contacts an RDM instance and create a draft of a record and an error value.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" fName := "draft.json" // An updated draft record in JSON src, _ := os.ReadFile(fName) debug := true draft, err := UpdateDraft(cfg, id, src, debug) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%+v\n", draft) ```

func UploadFiles added in v0.0.29 

func UploadFiles(cfg *Config, recordId string, filenames []string, debug bool) (map[string]interface{}, error)

UploadFiles takes a configuration object and record id, and a map to filename and paths contacts an RDM instance and adds the files to a draft record.

The configuration object must have the InvenioAPI and InvenioToken attributes set.

``` cfg, _ := LoadConfig("config.json") id := "qez01-2309a" fNames := ["article.pdf", "data.zip" ] // add files to draft record in JSON src, _ := os.ReadFile(fName) draft, err := UploadFiles(cfg, id, fNames) 

if err != nil {
   // ... handle error ...
}

fmt.Printf("%+v\n", draft) ```

Types

type Config 

type Config struct {
	// Debug is set true then methods with access to the Config obect
	// can use this flag to implement addition logging to standard err
	Debug bool `json:"-"`
	// Repository Name, e.g. CaltechAUTHORS, CaltechTHESIS, CaltechDATA
	RepoName string `json:"repo_name,omitempty"`
	// InvenioAPI holds the URL to the InvenioAPI
	InvenioAPI string `json:"rdm_url,omitempty"`
	// InvenioToken is holds the token string to access the API
	InvenioToken string `json:"rdmtok,omitempty"`
	// Invenio DSN holds the data source name for the Postgres database storing the invenio records
	InvenioDSN string `json:"rdm_dsn,omitempty"`
	// InvenioStorage holds the URI to the default storage of Invenio RDM objects, e.g. local file system or S3 bucket
	InvenioStorage string `json:"rdm_storage,omitempty"`
	// CName holds the dataset collection name used when harvesting content
	CName string `json:"c_name,omitempty"`
	// MailTo holds an email address to use when an email (e.g. CrossRef API access) is needed
	MailTo string `json:"mailto,omitempty"`
	// contains filtered or unexported fields
}

Config holds the common configuration used by all irdmtools

func NewConfig added in v0.0.5 

func NewConfig() *Config

NewConfig generates an empty configuration struct.

func (*Config) LoadConfig 

func (cfg *Config) LoadConfig(configFName string) error

LoadConfig reads the configuration file and initializes the attributes in the Config struct. It returns an error if problem were encounter. NOTE: It does NOT merge the settings in the environment.

``` 

cfg := NewConfig()
if err := cfg.LoadConfig("irdmtools.json"); err != nil {
   // ... handle error ...
}
fmt.Printf("Invenio RDM API UTL: %q\n", cfg.IvenioAPI)
fmt.Printf("Invenio RDM token: %q\n", cfg.InvenioToken)
fmt.Printf("Dataset Collection: %q\n", cfg.CName)
fmt.Printf("MailTo: %q\n", cfg.MailTo)

```

func (*Config) LoadEnv 

func (cfg *Config) LoadEnv(prefix string) error

LoadEnv checks the environment for configuration values if not previusly sets them. It will apply a prefix to the expected environment variable names if one is provided.

``` 

cfg := new(Config)
if err := cfg.LoadEnv("TEST_"); err != nil {
      // ... error handle ...
}

```

type Doi2Rdm 

type Doi2Rdm struct {
	Cfg *Config
}

Doi2Rdm holds the configuration for doi2rdm cli.

func (*Doi2Rdm) Configure 

func (app *Doi2Rdm) Configure(configFName string, envPrefix string, debug bool) error

Configure reads the configuration file and environtment initialing the Cfg attribute of a Doi2Rdm object. It returns an error if problem were encounter.

``` 

app := new(irdmtools.Doi2Rdm)
if err := app.Configure("irdmtools.json", "TEST_"); err != nil {
   // ... handle error ...
}
fmt.Printf("Invenio RDM API UTL: %q\n", app.Cfg.IvenioAPI)
fmt.Printf("Invenio RDM token: %q\n", app.Cfg.InvenioToken)

```

func (*Doi2Rdm) Run 

func (app *Doi2Rdm) Run(in io.Reader, out io.Writer, eout io.Writer, options map[string]string, doi string) error

Run implements the doi2rdm cli behaviors. With the exception of the "setup" action you should call `app.LoadConfig()` before execute Run.

``` 

app := new(irdmtools.Doi2Rdm)
if err := app.LoadConfig("irdmtools.json"); err != nil {
   // ... handle error ...
}
recordId := "wx0w-2231"
src, err := app.Run(os.Stdin, os.Stdout, os.Stderr,
                     "get_record", []string{recordId})
if err != nil {
    // ... handle error ...
}
fmt.Printf("%s\n", src)

```

type EPrint2Rdm 

type EPrint2Rdm struct {
	Cfg *Config
}

EPrint2Rdm holds the configuration for rdmutil cli.

func (*EPrint2Rdm) Run 

func (app *EPrint2Rdm) Run(in io.Reader, out io.Writer, eout io.Writer, username string, password string, host string, eprintId string, resourceTypesFName string, contributorTypesFName string, allIds bool, idList string, cName string, debug bool) error

Run implements the eprint2rdm cli behaviors.

``` 

	app := new(irdmtools.EPrint2Rdm)
	eprintUsername := os.Getenv("EPRINT_USERNAME")
	eprintPassword := os.Getenv("EPRINT_PASSWORD")
	eprintHost := "eprints.example.edu"
	eprintId := "11822"
	resourceTypes := map[string]string{}
 if err := LoadTypesMap("resource-types.csv", resourceTypes);
		err != nil {
		// ... handle error ...
	}
 contributorTypes := map[string]string{}
 if err := LoadTypesMap("contributor-types.csv", contributorTypes);
		err != nil {
		// ... handle error ...
	}
	src, err := app.Run(os.Stdin, os.Stdout, os.Stderr,
						eprintUser, eprintPassword,
						eprintHost, eprintId,
                     resourceTypes, contributorsTypes,
						debug)
	if err != nil {
		// ... handle error ...
	}
	fmt.Printf("%s\n", src)

```

type EPrintKeysPage added in v0.0.2 

type EPrintKeysPage struct {
	XMLName xml.Name `xml:"html"`
	Anchors []string `xml:"body>ul>li>a"`
}

EPrintKeysPage holds the structure of the HTML page with the EPrint IDs embedded from the EPrint REST API.

type Hits 

type Hits struct {
	Hits  []map[string]interface{} `json:"hits,omitempty"`
	Total int                      `json:"total,omitempty"`
}
type Links struct {
	Self string `json:"self,omitempty"`
	Next string `json:"next,omitempty"`
	Prev string `json:"prev,omitempty"`
}

type OAIHeader 

type OAIHeader struct {
	Status     string   `xml:"status,attr,omitempty" json:"status,omitempty"`
	Identifier string   `xml:"identifier,omitempty" json:"identifier,omitempty"`
	DateStamp  string   `xml:"datestamp,omitempty" json:"datestamp,omitempty"`
	SetSpec    []string `xml:"setSpec,omitempty" json:"set_spec,omitempty"`
}

OAIHeader holds the response items for

type OAIListIdentifiers 

type OAIListIdentifiers struct {
	Headers         []OAIHeader `xml:"header,omitempty" json:"header,omitempty"`
	ResumptionToken string      `xml:"resumptionToken,omitempty" json:"resumption_token,omitempty"`
}

type OAIListIdentifiersResponse 

type OAIListIdentifiersResponse struct {
	XMLName         xml.Name            `xml:"OAI-PMH" json:"-"`
	XMLNS           string              `xml:"xmlns,attr,omitempty" json:"xmlns,omitempty"`
	ResponseDate    string              `xml:"responseDate,omitempty" json:"response_date,omitempty"`
	Request         string              `xml:"request,omitempty" json:"request,omitempty"`
	RequestAttr     map[string]string   `xml:"request,attr,omitempty" json:"request_attr,omitempty"`
	ListIdentifiers *OAIListIdentifiers `xml:"ListIdentifiers,omitempty" json:"list_identifiers,omitempty"`
}

OAIListIdendifiersResponse

type QueryResponse 

type QueryResponse struct {
	//
	Hits   *Hits  `json:"hits,omitepmty"`
	Links  *Links `json:"links,omitempty"`
	SortBy string `json:"sortBy,omitempty"`
}

QueryResponse holds the response to /api/records?q=...

type RateLimit 

type RateLimit struct {
	// Limit maps to X-RateLimit-Limit
	Limit int `json:"limit,omitempty"`
	// OldLimit holds the last value of rate limit before change.
	OldLimit int `json:"-"`
	// Remaining maps to X-RateLimit-Remaining
	Remaining int `json:"remaining,omitempty"`
	// Reset maps to X-RateLimit-Reset
	Reset int `json:"reset,omitempty"`
}

RateLimit holds the values used to play nice with OAI-PMH or REST API. It normally is extracted from the response header.

func (*RateLimit) Fprintf 

func (rl *RateLimit) Fprintf(out io.Writer)

func (*RateLimit) FromHeader 

func (rl *RateLimit) FromHeader(header http.Header)

FromHeader takes an http.Header (e.g. http.Response.Header) and updates a rate limit struct.

``` rl := new(RateLimit) rl.FromHeader(header) ```

func (*RateLimit) FromResponse 

func (rl *RateLimit) FromResponse(resp *http.Response)

FromResponse takes an http.Response struct and extracts the header values realated to rate limits (e.g. X-RateLite-Limit)

``` rl := new(RateLimit) rl.FromResponse(response) ```

func (*RateLimit) ResetString 

func (rl *RateLimit) ResetString() string

func (*RateLimit) String 

func (rl *RateLimit) String() string

func (*RateLimit) Throttle 

func (rl *RateLimit) Throttle(i int, tot int)

Throttle looks at the rate limit structure and implements an appropriate sleep time based on rate limits.

``` 

 i, tot := 0, 1000 // This ith' iteration and total number of records
	rl := new(RateLimit)
	// Set our rate limit from
	rl.FromResponse(response)
 rl.Throttle(i, tot)

```

func (*RateLimit) TimeToReset 

func (rl *RateLimit) TimeToReset() (time.Duration, time.Time)

func (*RateLimit) TimeToWait 

func (rl *RateLimit) TimeToWait(unit time.Duration) time.Duration

SecondsToWait returns the number of seconds (as a time.Duratin) to wait to avoid a http status code 429 and a ratio (float64) of remaining per request limit.

``` rl := new(RateLimit) rl.FromHeader(response.Header) timeToWait := rl.TimeToWait() time.Sleep(timeToWait) ```

type RdmUtil 

type RdmUtil struct {
	Cfg   *Config
	Debug bool
}

RdmUtil holds the configuration for rdmutil cli.

func (*RdmUtil) CheckDOI added in v0.0.34 

func (app *RdmUtil) CheckDOI(doi string) ([]byte, error)

CheckDOI checks the .pids.doi.identifier and returns a record from the match DOI.

``` 

	app := new(irdmtools.RdmUtil)
	if err := app.LoadConfig("irdmtools.json"); err != nil {
	   // ... handle error ...
	}
 doi := '10.1126/science.82.2123.219'
	src, err := app.CheckDOI(doi)
	if err != nil {
	    // ... handle error ...
	}
	fmt.Printf("%s\n", src)

```

func (*RdmUtil) Configure 

func (app *RdmUtil) Configure(configFName string, envPrefix string, debug bool) error

Configure reads the configuration file and environtment initialing the Cfg attribute of a RdmUtil object. It returns an error if problem were encounter.

``` 

app := new(irdmtools.RdmUtil)
if err := app.Configure("irdmtools.json", "TEST_"); err != nil {
   // ... handle error ...
}
fmt.Printf("Invenio RDM API UTL: %q\n", app.Cfg.IvenioAPI)
fmt.Printf("Invenio RDM token: %q\n", app.Cfg.InvenioToken)

```

func (*RdmUtil) DeleteEndpoint added in v0.0.31 

func (app *RdmUtil) DeleteEndpoint(p string) ([]byte, error)

DeleteEndpoint performs a DELETE on the endpoint indicated by PATH provided.

func (*RdmUtil) DeleteFiles added in v0.0.29 

func (app *RdmUtil) DeleteFiles(recordId string, filenames []string) ([]byte, error)

DeleteFiles takes a RDM record id an list of files and removes them from a draft.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

id := "woie-x0121" filenames := []string{ "article.pdf", "charts.zip", "data.zip" } src, err := app.DeleteFiles(id, filenames) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) DiscardDraft added in v0.0.29 

func (app *RdmUtil) DiscardDraft(recordId string) ([]byte, error)

DiscardDraft takes a record id and delete the draft.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

id := "woie-x0121" src, err := app.DiscardDraft(id) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetAccess added in v0.0.29 

func (app *RdmUtil) GetAccess(id string, accessType string) ([]byte, error)

GetAccess returns the JSON for the access attribute in a record if accessType parameter is an empty string or the specific access requested if not (e.g. "files", "record"). An error value is also returned.

``` app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId := "woie-x0121" accessType := "" // accessType = "record" // accessType := "files" src, err := app.GetAccess(recordId, accessType) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func (*RdmUtil) GetDraft added in v0.0.28 

func (app *RdmUtil) GetDraft(id string) ([]byte, error)

GetDraft returns an existing draft of a record.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId := "woie-x0121" src, err := app.GetDraft(recordId) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetDraftFiles added in v0.0.31 

func (app *RdmUtil) GetDraftFiles(recordId string) ([]byte, error)

GetDraftFiles returns the metadata for a draft's files

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId := "woie-x0121" src, err := app.GetFiles(recordId) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetEndpoint added in v0.0.29 

func (app *RdmUtil) GetEndpoint(p string) ([]byte, error)

GetEndpoint performs a GET on the endpoint indicated by PATH provided.

func (*RdmUtil) GetFile added in v0.0.28 

func (app *RdmUtil) GetFile(id string, fName string) ([]byte, error)

GetFile returns the metadata for a file

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId := "woie-x0121" fName := "article.pdf" src, err := app.GetFile(recordId, fName) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetFiles added in v0.0.28 

func (app *RdmUtil) GetFiles(recordId string) ([]byte, error)

GetFiles returns the metadata for working with files

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId := "woie-x0121" src, err := app.GetFiles(recordId) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetModifiedIds 

func (app *RdmUtil) GetModifiedIds(start string, end string) ([]byte, error)

GetModified returns a byte slice for a JSON encode list of record ids modified (created, updated, deleted) in the given time range. If a problem occurs an error is returned.

``` 

app := new(irdmtools.RdmUtil)
if err := app.LoadConfig("irdmtools.json"); err != nil {
   // ... handle error ...
}
src, err := app.GetModifiedIds("2020-01-01", "2020-12-31")
if err != nil {
    // ... handle error ...
}
fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetRawRecord added in v0.0.5 

func (app *RdmUtil) GetRawRecord(id string) ([]byte, error)

GetRawRecord returns a byte slice for a JSON encoded record as a `map[string]interface{}` retrieved from the RDM API.

``` 

app := new(irdmtools.RdmUtil)
if err := app.LoadConfig("irdmtools.json"); err != nil {
   // ... handle error ...
}
recordId := "woie-x0121"
src, err := app.GetRawRecord(recordId)
if err != nil {
    // ... handle error ...
}
fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetRecord 

func (app *RdmUtil) GetRecord(id string) ([]byte, error)

GetRecord returns a byte slice for a JSON encoded record or an error.

``` 

app := new(irdmtools.RdmUtil)
if err := app.LoadConfig("irdmtools.json"); err != nil {
   // ... handle error ...
}
recordId := "woie-x0121"
src, err := app.GetRecord(recordId)
if err != nil {
    // ... handle error ...
}
fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetRecordIds 

func (app *RdmUtil) GetRecordIds() ([]byte, error)

GetRecordIds returns a byte slice for a JSON encode list of record ids or an error.

``` 

app := new(irdmtools.RdmUtil)
if err := app.LoadConfig("irdmtools.json"); err != nil {
   // ... handle error ...
}
src, err := app.GetRecordIds()
if err != nil {
    // ... handle error ...
}
fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetReview added in v0.0.31 

func (app *RdmUtil) GetReview(recordId string) ([]byte, error)

GetReview takes a record id and returns information about review requests.

``` app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

id := "woie-x0121" src, err := app.GetReview(id, "accept", "") 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func (*RdmUtil) GetVersionLatest added in v0.0.28 

func (app *RdmUtil) GetVersionLatest(id string) ([]byte, error)

GetVersionLatest returns the latest version metadata for a record.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId := "woie-x0121" src, err := app.GetVersionLatest(recordId) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) GetVersions added in v0.0.28 

func (app *RdmUtil) GetVersions(id string) ([]byte, error)

GetVersions returns the versioning metadata for a record.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId := "woie-x0121" src, err := app.GetVersions(recordId) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) Harvest 

func (app *RdmUtil) Harvest(fName string) error

Harvest takes a JSON file contianing a list of record ids and harvests them into a dataset v2 collection. The dataset collection must exist and be configured in either the environment or configuration file.

func (*RdmUtil) NewDraft added in v0.0.30 

func (app *RdmUtil) NewDraft(recordId string) ([]byte, error)

NewDraft returns the a new draft of an existing record.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId = "woie-x0121" src, err := app.NewDraft(recordId) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) NewRecord added in v0.0.30 

func (app *RdmUtil) NewRecord(src []byte) ([]byte, error)

NewRecord create a new record from JSON source. It returns a created record including a record id.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

jsonSrc, _ := os.ReadFile("new_record.json") src, err := app.NewRecord(jsonSrc) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) NewRecordVersion added in v0.0.30 

func (app *RdmUtil) NewRecordVersion(recordId string) ([]byte, error)

NewRecordVersion create a new record version using record id. It returns a created record including a record id.

``` app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId = "woie-x0121" src, err := app.NewRecordVersion(recordId) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func (*RdmUtil) PatchEndpoint added in v0.0.30 

func (app *RdmUtil) PatchEndpoint(p string, data []byte) ([]byte, error)

PatchEndpoint performs a PATCH on the endpoint indicated by PATH provided.

func (*RdmUtil) PostEndpoint added in v0.0.30 

func (app *RdmUtil) PostEndpoint(p string, data []byte) ([]byte, error)

PostEndpoint performs a POST on the endpoint indicated by PATH provided.

func (*RdmUtil) PublishRecordVersion added in v0.0.32 

func (app *RdmUtil) PublishRecordVersion(recordId string, version string, pubDate string) ([]byte, error)

PublishRecordVersion publish a new version draft using the version's record id.

``` app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId = "woie-x0121" version, pubDate := "internal", "2022-08" src, err := app.PublishRecordVersion(recordId) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func (*RdmUtil) PutEndpoint added in v0.0.30 

func (app *RdmUtil) PutEndpoint(p string, data []byte) ([]byte, error)

PutEndpoint performs a PUT on the endpoint indicated by PATH provided.

func (*RdmUtil) Query 

func (app *RdmUtil) Query(q string, sort string) ([]byte, error)

Query returns a byte slice for a JSON encode list of record summaries or an error.

``` 

app := new(irdmtools.RdmUtil)
if err := app.LoadConfig("irdmtools.json"); err != nil {
   // ... handle error ...
}
src, err := app.Query("My favorite book", -1, "newest")
if err != nil {
    // ... handle error ...
}
fmt.Printf("%s\n", src)

```

func (*RdmUtil) RetrieveFile added in v0.0.28 

func (app *RdmUtil) RetrieveFile(id string, fName string) ([]byte, error)

RetrieveFile retrieves the file from an RDM instance.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

recordId := "woie-x0121" fName := "article.pdf" data, err := app.RetrieveFile(recordId, fName) 

if err != nil {
  // ... handle error ...
}

os.WriteFile("article.pdf", data, 0664)

```

func (*RdmUtil) ReviewRequest added in v0.0.31 

func (app *RdmUtil) ReviewRequest(recordId string, decision string, comment string) ([]byte, error)

ReviewRequest takes a record id, a decision and a comment and submits it to the review process.

``` app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

id := "woie-x0121" src, err := app.ReviewPublishDraft(id, "accept", "") 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src) ```

func (*RdmUtil) Run 

func (app *RdmUtil) Run(in io.Reader, out io.Writer, eout io.Writer, action string, params []string) error

Run implements the irdmapp cli behaviors. With the exception of the Run.

``` 

app := new(irdmtools.RdmUtil)
if err := app.LoadConfig("irdmtools.json"); err != nil {
   // ... handle error ...
}
recordId := "wx0w-2231"
src, err := app.Run(os.Stdin, os.Stdout, os.Stderr,
                     "get_record", []string{recordId})
if err != nil {
    // ... handle error ...
}
fmt.Printf("%s\n", src)

```

func (*RdmUtil) SendToCommunity added in v0.0.30 

func (app *RdmUtil) SendToCommunity(recordId string, communityId string) ([]byte, error)

SendToCommunity takes a RDM record id and community UUID. It populates the the parent element approriately for draft to be submitted to a specific community.

func (*RdmUtil) SetAccess added in v0.0.29 

func (app *RdmUtil) SetAccess(id string, accessType string, accessValue string) ([]byte, error)

SetAccess sets the access attribute for a record. The access type can be either record or files. The value can be either "public" or "restricted". An error value is also returned with the function.

``` 

	app := new(irdmtools.RdmUtil)
	if err := app.LoadConfig("irdmtools.json"); err != nil {
	   // ... handle error ...
	}
	recordId := "woie-x0121"
 accessType := "record"
	src, err := app.SetAccess(recordId, accessType, "public")
	if err != nil {
	    // ... handle error ...
	}
	fmt.Printf("%s\n", src)
 accessType = "files"
	src, err := app.SetAccess(recordId, accessType, "restricted")
	if err != nil {
	    // ... handle error ...
	}
	fmt.Printf("%s\n", src)

```

func (*RdmUtil) SetFilesEnable added in v0.0.30 

func (app *RdmUtil) SetFilesEnable(recordId string, enable bool) ([]byte, error)

SetFilesEnable takes a RDM record id and either boolean setting the files.enabled value in a draft record. Returns the draft record and an error value.

func (*RdmUtil) SetPubDate added in v0.0.32 

func (app *RdmUtil) SetPubDate(recordId string, pubDate string) ([]byte, error)

SetPubDate takes a RDM record id and publication date string setting the .metadata.publication_date in a draft record.

func (*RdmUtil) SetVersion added in v0.0.32 

func (app *RdmUtil) SetVersion(recordId string, version string) ([]byte, error)

SetVersion takes a RDM record id and version string setting the .metadata.version in a draft record.

func (*RdmUtil) UpdateDraft added in v0.0.28 

func (app *RdmUtil) UpdateDraft(recordId string, src []byte) ([]byte, error)

UpdateDraft returns takes a record id and returns a draft record.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

id := "woie-x0121" jsonSrc, _ := os.ReadFile("draft.json") src, err := app.UpdateDraft(id, jsonSrc) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

func (*RdmUtil) UploadFiles added in v0.0.29 

func (app *RdmUtil) UploadFiles(recordId string, filenames []string) ([]byte, error)

UploadFiles takes a RDM record id an list of files and uploads them to a draft.

```

app := new(irdmtools.RdmUtil) 

if err := app.LoadConfig("irdmtools.json"); err != nil {
  // ... handle error ...
}

id := "woie-x0121" filenames := []string{ "article.pdf", "charts.zip", "data.zip" } src, err := app.UploadFiles(id, filenames) 

if err != nil {
  // ... handle error ...
}

fmt.Printf("%s\n", src)

```

Source Files

View all Source files

Directories

Path Synopsis
cmd
doi2rdm
doi2rdm is a command line program for harvesting DOI metadata from CrossRef and DataCite returning a JSON documentument sutiable for import into Invenio RDM.
 doi2rdm is a command line program for harvesting DOI metadata from CrossRef and DataCite returning a JSON documentument sutiable for import into Invenio RDM.
eprint2rdm
eprint2rdm is a command line program for harvesting an EPrint metadata record and return a Invenio RDM style record.
 eprint2rdm is a command line program for harvesting an EPrint metadata record and return a Invenio RDM style record.
people2vocabulary
rdmutil
rdmutil is a command line program for working with Invenio RDM.
 rdmutil is a command line program for working with Invenio RDM.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.