gorods

package module
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 12, 2016 License: BSD-3-Clause Imports: 8 Imported by: 1

README

GoRods

Golang binding for iRods C API. Requires go version >= 1.5 for cgo compile flags variable support.

Notice: This package is incomplete and still under heavy development. API is subject to change without warning until a stable version is released.

Installation

Dependencies (http://irods.org/download/): irods-dev-4.1.8

$ go get github.com/jjacquay712/GoRods
Docs

https://godoc.org/github.com/jjacquay712/GoRods

Example Usage

package main

import (
	"fmt"
	"github.com/jjacquay712/GoRods"
)

func main() {

	// Connect to server, error provided by second parameter
	irods, _ := gorods.New(gorods.ConnectionOptions {

		// Or gorods.EnvironmentDefined to use the systems preconfigured environment
		Type: gorods.UserDefined, 

		Host: "localhost",
		Port: 1247,
		Zone: "tempZone",

		Username: "admin",
		Password: "password",
	})

	// Open collection, preload sub collections into memory
	homeDir, _ := irods.Collection("/tempZone/home/admin", true)

	buildFile := homeDir.Cd("gorods").Get("build.sh")

	// Returns MetaCollection containing all metadata for buildFile DataObject
	metas, _ := buildFile.Meta()

	// Returns pointer to Meta struct
	metas.Get("MyAttribute")

	// Add a meta AVU triple
	metas.Add(gorods.Meta {
		Attribute: "add-test",
		Value: "test",
		Units: "string",
	})

	// Or use a shortcut
	myAttr, _ := buildFile.Attribute("MyAttribute")

	myAttr.SetValue("New Value")

	myAttr.SetUnits("myUnit")

	myAttr.Set("New Value", "myUnit")

	myAttr.Rename("AnotherAttribute")

	// Delete the metadata AVU triple
	myAttr.Delete()
	
	// Returns true/false if checksum matches
	buildFile.Verify("GdU5GXvmky9/rw7rduk4JaEtEdlhhhhGufiez+2aI4o=")
	
	// Download remote file
	buildFile.DownloadTo("build.sh")

	// Read file from /tempZone/home/admin/gorods/build.sh
	contents, _ := buildFile.Read()

	// Read file in 5 byte chunks
	var wholeFile []byte

	buildFile.ReadChunk(5, func(chunk []byte) {
		wholeFile = append(wholeFile, chunk...)
	})

	fmt.Printf(string(wholeFile))

	// Print []Byte as string
	fmt.Printf(string(contents))

	// Add local file to collection
	remoteFile, _ := homeDir.Put("local_file.txt")

	// Copy file to gorods directory
	remoteFile.CopyTo("gorods")
	// or
	//
	// gorodsDir := homeDir.Cd("gorods")
	// remoteFile.CopyTo(gorodsDir)

	// Move file
	remoteFile.MoveTo("gorods/local_file2.txt")

	// Rename file
	remoteFile.Rename("local_file3.txt")

	// Create file in home directory, overwrite if it exists
	test, _ := gorods.CreateDataObj(gorods.DataObjOptions {
		Name: "test.txt",
		Mode: 0750,
		Force: true,
	}, homeDir)

	// Write string to test file
	test.Write([]byte("This is a test!"))

	// Write 5 copies of "test" to file
	// Will start writing at last offset (seek) position (typically 0)
	for n := 0; n < 5; n++ {
		test.WriteBytes([]byte("test\n"))
	}

	// We must close the file explicitly after calling WriteBytes()
	test.Close()

	// Stat the test.txt file
	fmt.Printf("%v \n", test.Stat())

	// Read the contents back, print to screen
	fmt.Printf("%v \n", string(test.Read()))

	// Delete the file
	test.Delete()

}

Contributing

Send me a pull request!

Todo

iRods API Coverage
  • Implement Connection: ilocate, interface to query meta (imeta qu)
  • Implement Collection: CreateCollection(), MoveTo(), CopyTo(), DownloadTo()
  • Implement: User, Group?, Resource, Zone?
  • Implement access control (rcModAccessControl) and tickets
  • Implement DataObj: MoveToResource(), Replicate(), ReplSettings()
Code Polish
  • Complete unit tests

Known Issues

Copyright (c) 2016, University of Florida Research Foundation, Inc. All Rights Reserved.

GoRods is released under a 3-clause BSD License. For more information please refer to the LICENSE.md file

Documentation

Overview

Package gorods is a Golang binding for the iRods C API (iRods client library). GoRods uses cgo to call iRods client functions.

Index

Constants

View Source 
const (
	EnvironmentDefined = iota
	UserDefined
)

EnvironmentDefined and UserDefined constants are used when calling gorods.New(ConnectionOptions{ Type: ... }) When EnvironmentDefined is specified, the options stored in ~/.irods/irods_environment.json will be used. When UserDefined is specified you must also pass Host, Port, Username, and Zone. Password should be set regardless.

View Source 
const (
	DataObjType = iota
	CollectionType
	ResourceType
	ResourceGroupType
	UserType
)

Used when calling Type() on different gorods objects

View Source 
const (
	Info = iota
	Warn
	Fatal
)

Log level constants

Variables

This section is empty.

Functions

This section is empty.

Types

type Collection 

type Collection struct {
	Path        string
	Name        string
	DataObjects IRodsObjs
	MetaCol     *MetaCollection
	Con         *Connection
	Col         *Collection
	Recursive   bool
	Init        bool
	Type        int
	// contains filtered or unexported fields
}

Collection structs contain information about single collections in an iRods zone.

func (*Collection) AddMeta added in v0.1.1 

func (col *Collection) AddMeta(m Meta) (newMeta *Meta, err error)

AddMeta adds a single Meta triple struct

func (*Collection) All 

func (col *Collection) All() (IRodsObjs, error)

Returns generic interface slice containing both data objects and collections combined

func (*Collection) Attribute 

func (col *Collection) Attribute(attr string) (*Meta, error)

Attribute gets specific metadata AVU triple for Collection

func (*Collection) Cd 

func (col *Collection) Cd(path string) *Collection

Cd is a shortcut for calling collection.GetCollections().Find(path). It effectively returns (or changes to) the sub collection you specify collection-relatively or absolutely.

func (*Collection) Close 

func (col *Collection) Close() error

Close closes the Collection connection and resets the handle

func (*Collection) CreateDataObj 

func (col *Collection) CreateDataObj(opts DataObjOptions) (*DataObj, error)

CreateDataObj creates a data object within the collection using the options specified

func (*Collection) DeleteMeta added in v0.1.1 

func (col *Collection) DeleteMeta(attr string) (*MetaCollection, error)

DeleteMeta deletes a single Meta triple struct, identified by Attribute field

func (*Collection) Exists 

func (col *Collection) Exists(path string) bool

Exists returns true of false depending on whether the DataObj or Collection is found

func (*Collection) Find 

func (col *Collection) Find(path string) IRodsObj

Find returns either a DataObject or Collection using the collection-relative or absolute path specified.

func (*Collection) FindCol added in v0.1.1 

func (col *Collection) FindCol(path string) *Collection

func (*Collection) FindObj added in v0.1.1 

func (col *Collection) FindObj(path string) *DataObj

func (*Collection) FindRecursive added in v0.1.1 

func (col *Collection) FindRecursive(path string) IRodsObj

Find returns either a DataObject or Collection using the collection-relative or absolute path specified.

func (*Collection) Get 

func (col *Collection) Get(path string) *DataObj

Get is a shortcut for calling collection.GetDataObjs().Find(path). It effectively returns the DataObj you specify collection-relatively or absolutely.

func (*Collection) GetCol added in v0.1.1 

func (col *Collection) GetCol() *Collection

GetCol returns the *Collection of the collection

func (*Collection) GetCollections added in v0.1.1 

func (col *Collection) GetCollections() (response IRodsObjs, err error)

GetCollections returns only the IRodsObjs that represent collections

func (*Collection) GetCon added in v0.1.1 

func (col *Collection) GetCon() *Connection

Connection returns the *Connection used to get collection

func (*Collection) GetDataObjs added in v0.1.1 

func (col *Collection) GetDataObjs() (response IRodsObjs, err error)

GetDataObjs returns only the data objects contained within the collection

func (*Collection) GetName added in v0.1.1 

func (col *Collection) GetName() string

GetName returns the Name of the collection

func (*Collection) GetPath added in v0.1.1 

func (col *Collection) GetPath() string

GetPath returns the Path of the collection

func (*Collection) GetType added in v0.1.1 

func (col *Collection) GetType() int

Type gets the type

func (*Collection) IsRecursive added in v0.1.1 

func (col *Collection) IsRecursive() bool

IsRecursive returns true or false

func (*Collection) Meta 

func (col *Collection) Meta() (*MetaCollection, error)

Meta returns collection of all metadata AVU triples for Collection

func (*Collection) Open 

func (col *Collection) Open() error

Open connects to iRods and sets the handle for Collection. Usually called by Collection.init()

func (*Collection) Put 

func (col *Collection) Put(localFile string) (*DataObj, error)

Put adds a local file to the remote iRods collection

func (*Collection) ReadCollection 

func (col *Collection) ReadCollection() error

ReadCollection reads data (overwrites) into col.DataObjects field.

func (*Collection) Refresh 

func (col *Collection) Refresh() error

Refresh is an alias of ReadCollection()

func (*Collection) String 

func (obj *Collection) String() string

String shows the contents of the collection.

d = DataObj

C = Collection

Sample output: 

Collection: /tempZone/home/admin/gorods
	d: build.sh
	C: bin
	C: pkg
	C: src

type Connection 

type Connection struct {
	Connected  bool
	Options    *ConnectionOptions
	OpenedObjs IRodsObjs
	// contains filtered or unexported fields
}

func New 

func New(opts ConnectionOptions) (*Connection, error)

New creates a connection to an iRods iCAT server. EnvironmentDefined and UserDefined constants are used in ConnectionOptions{ Type: ... }). When EnvironmentDefined is specified, the options stored in ~/.irods/irods_environment.json will be used. When UserDefined is specified you must also pass Host, Port, Username, and Zone. Password should be set unless using an anonymous user account with tickets.

func (*Connection) Collection 

func (con *Connection) Collection(startPath string, recursive bool) (*Collection, error)

Collection initializes and returns an existing iRods collection using the specified path

func (*Connection) DataObject added in v0.1.1 

func (con *Connection) DataObject(dataObjPath string) (dataobj *DataObj, err error)

DataObject directly returns a specific DataObj without the need to traverse collections. Must pass full path of data object.

func (*Connection) Disconnect 

func (con *Connection) Disconnect() error

Disconnect closes connection to iRods iCAT server, returns error on failure or nil on success

func (*Connection) QueryMeta added in v0.1.1 

func (con *Connection) QueryMeta(dataObjPath string) (dataobj *DataObj, err error)

QueryMeta

func (*Connection) SearchDataObjects added in v0.1.1 

func (con *Connection) SearchDataObjects(dataObjPath string) (dataobj *DataObj, err error)

SearchDataObjects searchs for and returns DataObjs slice based on a search string. Use '%' as a wildcard. Equivalent to ilocate command

func (*Connection) SetTicket added in v0.1.2 

func (con *Connection) SetTicket(t string) error

SetTicket is equivalent to using the -t flag with icommands

func (*Connection) String 

func (obj *Connection) String() string

String provides connection status and options provided during initialization (gorods.New)

type ConnectionOptions 

type ConnectionOptions struct {
	Type int

	Host string
	Port int
	Zone string

	Username string
	Password string
	Ticket   string
}

ConnectionOptions are used when creating iRods iCAT server connections see gorods.New() docs for more info.

type DataObj 

type DataObj struct {
	Path   string
	Name   string
	Size   int64
	Offset int64
	Type   int

	MetaCol *MetaCollection

	// Con field is a pointer to the Connection used to fetch the data object
	Con *Connection

	// Col field is a pointer to the Collection containing the data object
	Col *Collection
	// contains filtered or unexported fields
}

DataObj structs contain information about single data objects in an iRods zone.

func CreateDataObj 

func CreateDataObj(opts DataObjOptions, coll *Collection) (*DataObj, error)

CreateDataObj creates and adds a data object to the specified collection using provided options. Returns the newly created data object.

func (*DataObj) AddMeta added in v0.1.1 

func (obj *DataObj) AddMeta(m Meta) (nm *Meta, err error)

AddMeta adds a single Meta triple struct

func (*DataObj) Attribute 

func (obj *DataObj) Attribute(attrName string) (*Meta, error)

Attribute returns a single Meta triple struct found by the attributes name

func (*DataObj) Chksum 

func (obj *DataObj) Chksum() (string, error)

Chksum returns md5 hash string of data object

func (*DataObj) Close 

func (obj *DataObj) Close() error

Close closes the data object, resets handler

func (*DataObj) CopyTo 

func (obj *DataObj) CopyTo(iRodsCollection interface{}) error

CopyTo copies the data object to the specified collection. Supports Collection struct or string as input. Also refreshes the destination collection automatically to maintain correct state. Returns error.

func (*DataObj) Delete 

func (obj *DataObj) Delete() error

Delete deletes the data object from the iRods server with a force flag

func (*DataObj) DeleteMeta added in v0.1.1 

func (obj *DataObj) DeleteMeta(attr string) (*MetaCollection, error)

DeleteMeta deletes a single Meta triple struct, identified by Attribute field

func (*DataObj) DownloadTo 

func (obj *DataObj) DownloadTo(localPath string) error

DownloadTo downloads and writes the entire data object to the provided path. Don't use this with large files unless you have RAM to spare, use ReadChunk() instead. Returns error.

func (*DataObj) GetCol added in v0.1.1 

func (obj *DataObj) GetCol() *Collection

GetName returns the *Collection of the data object

func (*DataObj) GetCon added in v0.1.1 

func (obj *DataObj) GetCon() *Connection

Connection returns the *Connection used to get data object

func (*DataObj) GetName added in v0.1.1 

func (obj *DataObj) GetName() string

GetName returns the Name of the data object

func (*DataObj) GetPath added in v0.1.1 

func (obj *DataObj) GetPath() string

GetName returns the Path of the data object

func (*DataObj) GetType added in v0.1.1 

func (obj *DataObj) GetType() int

Type gets the type

func (*DataObj) LSeek 

func (obj *DataObj) LSeek(offset int64) error

LSeek sets the read/write offset pointer of a data object, returns error

func (*DataObj) Meta 

func (obj *DataObj) Meta() (*MetaCollection, error)

Meta returns collection of Meta AVU triple structs of the data object

func (*DataObj) MoveTo 

func (obj *DataObj) MoveTo(iRodsCollection interface{}) error

MoveTo moves the data object to the specified collection. Supports Collection struct or string as input. Also refreshes the source and destination collections automatically to maintain correct state. Returns error.

func (*DataObj) MoveToResource 

func (obj *DataObj) MoveToResource(destinationResource string) *DataObj

NEED TO IMPLEMENT

func (*DataObj) Open 

func (obj *DataObj) Open() error

Open opens a connection to iRods and sets the data object handle

func (*DataObj) Read 

func (obj *DataObj) Read() ([]byte, error)

Read reads the entire data object into memory and returns a []byte slice. Don't use this for large files.

func (*DataObj) ReadBytes 

func (obj *DataObj) ReadBytes(pos int64, length int) ([]byte, error)

ReadBytes reads bytes from a data object at the specified position and length, returns []byte slice and error.

func (*DataObj) ReadChunk 

func (obj *DataObj) ReadChunk(size int64, callback func([]byte)) error

ReadChunk reads the entire data object in chunks (size of chunk specified by size parameter), passing the data into a callback function for each chunk. Use this to read/write large files.

func (*DataObj) Rename 

func (obj *DataObj) Rename(newFileName string) error

Rename is equivalent to the Linux mv command except that the data object must stay within the current collection (directory), returns error.

func (*DataObj) ReplSettings 

func (obj *DataObj) ReplSettings(resource map[string]interface{}) *DataObj

NEED TO IMPLEMENT

func (*DataObj) Replicate 

func (obj *DataObj) Replicate(targetResource string) *DataObj

NEED TO IMPLEMENT

func (*DataObj) Stat 

func (obj *DataObj) Stat() (map[string]interface{}, error)

Stat returns a map (key/value pairs) of the system meta information. The following keys can be used with the map:

"objSize"

"dataMode"

"dataId"

"chksum"

"ownerName"

"ownerZone"

"createTime"

"modifyTime"

func (*DataObj) String 

func (obj *DataObj) String() string

String returns path of data object 

func (obj *DataObj) Unlink() error

Unlink deletes the data object from the iRods server, no force flag is used

func (*DataObj) Verify 

func (obj *DataObj) Verify(md5Checksum string) bool

Verify returns true or false depending on whether the checksum md5 string matches

func (*DataObj) Write 

func (obj *DataObj) Write(data []byte) error

Write writes the data to the data object, starting from the beginning. Returns error.

func (*DataObj) WriteBytes 

func (obj *DataObj) WriteBytes(data []byte) error

WriteBytes writes to the data object wherever the object's offset pointer is currently set to. It advances the pointer to the end of the written data for supporting subsequent writes. Be sure to call obj.LSeek(0) before hand if you wish to write from the beginning. Returns error.

type DataObjOptions 

type DataObjOptions struct {
	Name     string
	Size     int64
	Mode     int
	Force    bool
	Resource string
}

DataObjOptions is used for passing options to the CreateDataObj function

type GoRodsError 

type GoRodsError struct {
	LogLevel int
	Message  string
	Time     time.Time
}

GoRodsError stores information about errors

func (*GoRodsError) Error 

func (err *GoRodsError) Error() string

Error returns error string, alias of String(). Sample output: 

2016-04-22 10:02:30.802355258 -0400 EDT: Fatal - iRods Connect Failed: rcConnect failed

func (*GoRodsError) String 

func (err *GoRodsError) String() string

String returns error string. Sample output: 

2016-04-22 10:02:30.802355258 -0400 EDT: Fatal - iRods Connect Failed: rcConnect failed

type IRodsObj 

type IRodsObj interface {
	GetType() int
	GetName() string
	GetPath() string
	GetCol() *Collection
	GetCon() *Connection

	Meta() (*MetaCollection, error)
	Attribute(string) (*Meta, error)
	AddMeta(Meta) (*Meta, error)
	DeleteMeta(string) (*MetaCollection, error)

	String() string
	Open() error
	Close() error
}

IRodsObj is a generic interface used to detect the object type and access common fields

type IRodsObjs added in v0.1.1 

type IRodsObjs []IRodsObj

func (IRodsObjs) Exists added in v0.1.1 

func (objs IRodsObjs) Exists(path string) bool

Exists checks to see if a collection exists in the slice and returns true or false

func (IRodsObjs) Find added in v0.1.1 

func (objs IRodsObjs) Find(path string) IRodsObj

Find gets a collection from the slice and returns nil if one is not found. Both the collection name or full path can be used as input.

func (IRodsObjs) FindRecursive added in v0.1.1 

func (objs IRodsObjs) FindRecursive(path string) IRodsObj

FindRecursive acts just like Find, but also searches sub collections recursively. If the collection was not explicitly loaded recursively, only the first level of sub collections will be searched.

type Meta 

type Meta struct {
	Attribute string
	Value     string
	Units     string
	Parent    *MetaCollection
}

Meta structs contain information about a single iRods metadata attribute-value-units (AVU) triple

func (*Meta) Delete added in v0.1.1 

func (m *Meta) Delete() (*MetaCollection, error)

Delete deletes the current Meta struct from iRods object

func (*Meta) Rename added in v0.1.1 

func (m *Meta) Rename(attributeName string) (*Meta, error)

Rename will modify metadata AVU attribute name only

func (*Meta) Set added in v0.1.1 

func (m *Meta) Set(value string, units string) (*Meta, error)

Set will modify metadata AVU value & units

func (*Meta) SetAll added in v0.1.1 

func (m *Meta) SetAll(attributeName string, value string, units string) (newMeta *Meta, e error)

SetAll will modify metadata AVU with all three paramaters (Attribute, Value, Unit)

func (*Meta) SetUnits added in v0.1.1 

func (m *Meta) SetUnits(units string) (*Meta, error)

SetUnits will modify metadata AVU units only

func (*Meta) SetValue added in v0.1.1 

func (m *Meta) SetValue(value string) (*Meta, error)

SetValue will modify metadata AVU value only

func (*Meta) String 

func (m *Meta) String() string

String shows the contents of the meta struct.

Sample output: 

Attr1: Val (unit: foo)

type MetaCollection 

type MetaCollection struct {
	Metas Metas
	Obj   IRodsObj
	Con   *Connection
}

MetaCollection is a collection of metadata AVU triples for a single data object

func (*MetaCollection) Add added in v0.1.1 

func (mc *MetaCollection) Add(m Meta) (*Meta, error)

Add creates a new meta AVU triple, returns pointer to the created Meta struct

func (*MetaCollection) All added in v0.1.2 

func (mc *MetaCollection) All() (Metas, error)

All

func (*MetaCollection) Delete added in v0.1.1 

func (mc *MetaCollection) Delete(attr string) (err error)

Delete deletes the meta AVU triple from the data object, identified by it's Attribute field

func (*MetaCollection) Each added in v0.1.2 

func (mc *MetaCollection) Each(iterator func(*Meta)) error

Each

func (*MetaCollection) Get 

func (mc *MetaCollection) Get(attr string) (*Meta, error)

Get finds a single Meta struct by it's Attribute field. Similar to Attribute() function of other types.

func (*MetaCollection) ReadMeta added in v0.1.1 

func (mc *MetaCollection) ReadMeta() error

ReadMeta clears existing metadata triples and grabs updated copy from iCAT server.

func (*MetaCollection) Refresh added in v0.1.1 

func (mc *MetaCollection) Refresh() error

Refresh clears existing metadata triples and grabs updated copy from iCAT server. It's an alias of ReadMeta()

func (*MetaCollection) String 

func (mc *MetaCollection) String() string

String shows the contents of the meta collection.

Sample output: 

Attr1: Val (unit: )
Attr2: Yes (unit: bool)

type Metas added in v0.1.2 

type Metas []*Meta

Notes

Bugs

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.