llapi

package
v0.0.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 13, 2021 License: MIT Imports: 17 Imported by: 4

Documentation

Overview

Package llapi provides access to many of the functions avialable in liblustreapi.

Index

Constants

View Source 
const (
	OpMark     = C.CL_MARK
	OpCreate   = C.CL_CREATE   /* namespace */
	OpMkdir    = C.CL_MKDIR    /* namespace */
	OpHardlink = C.CL_HARDLINK /* namespace */
	OpSoftlink = C.CL_SOFTLINK /* namespace */
	OpMknod    = C.CL_MKNOD    /* namespace */
	OpUnlink   = C.CL_UNLINK   /* namespace */
	OpRmdir    = C.CL_RMDIR    /* namespace */
	OpRename   = C.CL_RENAME   /* namespace */
	OpExt      = C.CL_EXT      /* namespace extended record (2nd half of rename) */
	OpOpen     = C.CL_OPEN     /* not currently used */
	OpClose    = C.CL_CLOSE    /* may be written to log only with mtime change */
	OpLayout   = C.CL_LAYOUT   /* file layout/striping modified */
	OpTrunc    = C.CL_TRUNC
	OpSetattr  = C.CL_SETATTR
	OpXattr    = C.CL_XATTR
	OpHSM      = C.CL_HSM   /* HSM specific events, see flags */
	OpMtime    = C.CL_MTIME /* Precedence: setattr > mtime > ctime > atime */
	OpCtime    = C.CL_CTIME
	OpAtime    = C.CL_ATIME
	OpLast     = C.CL_LAST
)

Changelog Types

View Source 
const (
	HsmActionNone    = HsmAction(C.HSMA_NONE)
	HsmActionArchive = HsmAction(C.HSMA_ARCHIVE)
	HsmActionRestore = HsmAction(C.HSMA_RESTORE)
	HsmActionRemove  = HsmAction(C.HSMA_REMOVE)
	HsmActionCancel  = HsmAction(C.HSMA_CANCEL)
)

HSM Action constants

View Source 
const (
	// CopytoolDefault set of flags (currently none)
	CopytoolDefault = HsmCopytoolFlags(0)
	// CopytoolNonBlock disables blocking so the poll can be used on the copytoold file descriptor.
	CopytoolNonBlock = HsmCopytoolFlags(C.O_NONBLOCK)
)
View Source 
const (
	// HsmProgressWaiting indicates the HSM action for this file has been
	// submitted but has not yet been sent to an
	// agent for processing.
	HsmProgressWaiting = HsmProgressState(C.HPS_WAITING)

	// HsmProgressRunning indicates the HSM action for this file is currently
	// be processed by an agent.
	HsmProgressRunning = HsmProgressState(C.HPS_RUNNING)

	// HsmProgressDone indicates the HSM action has completed.
	HsmProgressDone = HsmProgressState(C.HPS_DONE)
)
View Source 
const (
	// HsmUserNone: Noop
	HsmUserNone = HsmUserAction(C.HUA_NONE)
	// HsmUserArchive: Archive file to specified archive id
	HsmUserArchive = HsmUserAction(C.HUA_ARCHIVE)
	// HsmUserRelease: Remove file's data from the filesystem. (Must have been achived first.)
	HsmUserRelease = HsmUserAction(C.HUA_RELEASE)
	// HsmUserRestore: Restore a released file
	HsmUserRestore = HsmUserAction(C.HUA_RESTORE)
	// HsmUserRemove: Remove data from the archive. (File must not be released.)
	HsmUserRemove = HsmUserAction(C.HUA_REMOVE)
	// HsmUserCancel: Cancels current in progress rqeuest for the file.
	HsmUserCancel = HsmUserAction(C.HUA_CANCEL)
)

HSM Action Types Each of these actions are applied to up to MaxBatchSize files in a single request. Each file in the request is processed indepentently and potentially in any order.

View Source 
const (
	HsmFileExists    = HsmStateFlag(C.HS_EXISTS)
	HsmFileArchived  = HsmStateFlag(C.HS_ARCHIVED)
	HsmFileReleased  = HsmStateFlag(C.HS_RELEASED)
	HsmFileDirty     = HsmStateFlag(C.HS_DIRTY)
	HsmFileNoRelease = HsmStateFlag(C.HS_NORELEASE)
	HsmFileNoArchive = HsmStateFlag(C.HS_NOARCHIVE)
	HsmFileLost      = HsmStateFlag(C.HS_LOST)
)

HSM State flags

View Source 
const (
	// LovDelayCreate is file creation flag that inhibits creation of the file's layout
	LovDelayCreate = int(C.O_LOV_DELAY_CREATE)
)

Variables

View Source 
var HsmStateFlags = map[HsmStateFlag]string{
	HsmFileExists:    "exists",
	HsmFileArchived:  "archived",
	HsmFileReleased:  "released",
	HsmFileDirty:     "dirty",
	HsmFileNoRelease: "no_release",
	HsmFileNoArchive: "no_archive",
	HsmFileLost:      "lost",
}

HsmStateFlags is a map of HsmStateFlag -> string NB: There's no llapi.hsm_state2name(), so we have to do it ourselves...

View Source 
var MaxBatchSize = 50

MaxBatchSize is a limit imposed by liblustreapi, somewhere.

Functions

func ChangelogClear 

func ChangelogClear(device string, token string, endRec int64) error

ChangelogClear deletes all changelog records up to endRec.

func ChangelogFini 

func ChangelogFini(cl *Changelog) error

ChangelogFini closes the Changelog.

func Fid2Path 

func Fid2Path(device string, f *lustre.Fid, recno *int64, linkno *int) (string, error)

Fid2Path returns next path for given fid. This returns relative paths from the root of the filesystem.

func FileOpenPool 

func FileOpenPool(name string, flags int, mode uint32, layout *DataLayout) (int, error)

FileOpenPool creates a new file with provided layout

func GetMdtIndex 

func GetMdtIndex(f *os.File, fid *lustre.Fid) (int, error)

GetMdtIndex returns the MDT the file resides on.

func GetMdtIndexByFid 

func GetMdtIndexByFid(mountFd int, f *lustre.Fid) (int, error)

GetMdtIndexByFid returns the MDT index for a given Fid

func GetName 

func GetName(mountPath string) (string, error)

GetName returns the name-id of the client filesystem at mountPath

func GetVersion 

func GetVersion() (string, error)

GetVersion returns the version of lustre installed on the host.

func HsmActionEnd 

func HsmActionEnd(hcap **HsmCopyActionPrivate, offset, length int64, flags, errVal int) error

HsmActionEnd ends the HSM transaction. If this is was a successful restore, then the the layout of the temporary data file is swapped with the actual file, and the group lock is dropped so applications can read the file. Ensure all data being written to the data file has been flushed before call End.

func HsmActionGetDataFid 

func HsmActionGetDataFid(hcap *HsmCopyActionPrivate) (*lustre.Fid, error)

HsmActionGetDataFid returns the fid that shoudl be used to restore data to. It can also be used to copy data from for archiving.

func HsmActionGetFd 

func HsmActionGetFd(hcap *HsmCopyActionPrivate) (int, error)

HsmActionGetFd returns filedescriptor of the data fid. The data fid can also be opened directly, so this isn't strictly necessary.

func HsmActionProgress 

func HsmActionProgress(hcap *HsmCopyActionPrivate, offset, length, totalLength int64, flags int) error

HsmActionProgress informs the coordinator of how much progress has been made to file. This also serves as heartbeat, and if the CDT hasn't received progress on the file before the timeout window ends (defaults to 1 hour), the the action will be cancelled and reassigned to a different agent. (TODO: confirm actual CDT behavior on timeout)

func HsmCopytoolGetFd 

func HsmCopytoolGetFd(hcp *HsmCopytoolPrivate) int

HsmCopytoolGetFd returns the descriptor for the coordinator. Useful when non-blocking mode is being used.

func HsmCopytoolUnregister 

func HsmCopytoolUnregister(hcp **HsmCopytoolPrivate)

HsmCopytoolUnregister closes the connection to the coordinator.

func HsmImport 

func HsmImport(name string, archive uint, fi os.FileInfo, layout *DataLayout) (*lustre.Fid, error)

HsmImport creates a placeholder file in Lustre that refers to the file contents stored in an HSM backend. The file is created in the "released" state, and the contents will be retrieved when the file is opened or an explicit restore is requested.

TODO: using an os.FileInfo to pass the file metadata doesn't work for all cases. This should be simple struct the caller can populate. (Though just using syscall.Stat_t is also tempting.)

func HsmRequest 

func HsmRequest(r string, cmd HsmUserAction, archiveID uint, fidsToSend []*lustre.Fid) (int, error)

HsmRequest submits an HSM request for list of files

func Path2Fid 

func Path2Fid(path string) (*lustre.Fid, error)

Path2Fid returns the Fid for the given path or an error.

func SetFileLayout 

func SetFileLayout(fd int, layout *DataLayout) error

SetFileLayout sets the data striping layout on a file that has been created with O_LOV_DELAY_CREATE

func SetHsmFileStatus 

func SetHsmFileStatus(filePath string, setMask, clearMask uint64, archiveID uint32) error

SetHsmFileStatus updates the file's HSM flags and/or archive ID

Types

type Changelog 

type Changelog struct {
	// contains filtered or unexported fields
}

Changelog is opaque data representing an open changelog.

func ChangelogStart 

func ChangelogStart(device string, startRec int64, follow bool) (*Changelog, error)

ChangelogStart opens the changelog. The firsst record read will be the idnex specified by startRec.

type ChangelogRecord 

type ChangelogRecord struct {
	// contains filtered or unexported fields
}

ChangelogRecord is a record in a Changelog

func ChangelogRecv 

func ChangelogRecv(cl *Changelog) (*ChangelogRecord, error)

ChangelogRecv returns the next record in the changelog.

func (*ChangelogRecord) Index 

func (r *ChangelogRecord) Index() int64

Index returns the changelog record's index in the log

func (*ChangelogRecord) IsLastRename 

func (r *ChangelogRecord) IsLastRename() (last, exists bool)

IsLastRename returns a tuple of boolean values to indicate: 1) Whether or not the rename was for the the last hardlink 2) Whether or not there may still be an archive of the file in HSM 

func (r *ChangelogRecord) IsLastUnlink() (last, exists bool)

IsLastUnlink returns a tuple of boolean values to indicate: 1) Whether or not the unlink was for the the last hardlink 2) Whether or not there may still be an archive of the file in HSM

func (*ChangelogRecord) IsRename 

func (r *ChangelogRecord) IsRename() bool

IsRename is true if this record is a rename.

func (*ChangelogRecord) JobID 

func (r *ChangelogRecord) JobID() string

JobID returns the changelog record's Job ID information (if available)

func (*ChangelogRecord) Name 

func (r *ChangelogRecord) Name() string

Name returns the filename associated with the record (if available)

func (*ChangelogRecord) ParentFid 

func (r *ChangelogRecord) ParentFid() *lustre.Fid

ParentFid returns the parent Fid for the changelog record's action

func (*ChangelogRecord) SourceFid 

func (r *ChangelogRecord) SourceFid() *lustre.Fid

SourceFid returns the source Fid when a file is renamed

func (*ChangelogRecord) SourceName 

func (r *ChangelogRecord) SourceName() string

SourceName returns the source filename when a file is renamed

func (*ChangelogRecord) SourceParentFid 

func (r *ChangelogRecord) SourceParentFid() *lustre.Fid

SourceParentFid returns the source Fid's parent Fid when a file is renamed

func (*ChangelogRecord) String 

func (r *ChangelogRecord) String() string

func (*ChangelogRecord) TargetFid 

func (r *ChangelogRecord) TargetFid() *lustre.Fid

TargetFid returns the recipient Fid for the changelog record's action

func (*ChangelogRecord) Time 

func (r *ChangelogRecord) Time() time.Time

Time returns the changelog record's time

func (*ChangelogRecord) Type 

func (r *ChangelogRecord) Type() string

Type returns the changelog record's type as a string

func (*ChangelogRecord) TypeCode 

func (r *ChangelogRecord) TypeCode() uint

TypeCode returns the changelog record's type as a string

type DataLayout 

type DataLayout struct {
	StripePattern int
	StripeSize    int
	StripeCount   int
	StripeOffset  int
	Generation    int
	PoolName      string
	Objects       []OstData
}

DataLayout is the structure for file data.

func DefaultDataLayout 

func DefaultDataLayout() *DataLayout

DefaultDataLayout returns default layout.

func DirDataLayout 

func DirDataLayout(name string) (*DataLayout, error)

DirDataLayout returns the default DataLayout on a directory.

func FileDataLayout 

func FileDataLayout(name string) (*DataLayout, error)

FileDataLayout retrieves the file's data layout

func FileDataLayoutEA 

func FileDataLayoutEA(name string) (*DataLayout, error)

FileDataLayoutEA retrieves the file's data layout from the extended attribute. This would be my preferred method, but with released files this always returns 0 for stripe_count instead of the original value. Instead we're using llapi_file_get_stripe() above which does include stripe_count for released files. Also, this appears to return layout_gen as 0. Leaving this here in case this is fixed someday.

type FidPathError 

type FidPathError struct {
	Fid *lustre.Fid
	Rc  int
	Err error
}

FidPathError is an error that occurs while retrieving the pathname for a fid.

func (*FidPathError) Error 

func (e *FidPathError) Error() string

type HsmAction 

type HsmAction uint32

HsmAction indentifies which action to perform.

func (HsmAction) String 

func (action HsmAction) String() (s string)

type HsmActionItem 

type HsmActionItem struct {
	Action HsmAction
	Fid    *lustre.Fid
	Extent *HsmExtent
	Cookie uint64
	Data   []byte
	// contains filtered or unexported fields
}

HsmActionItem is specifc HSM action to perform on a single file

func (HsmActionItem) String 

func (ai HsmActionItem) String() string

type HsmActionList 

type HsmActionList struct {
	Version    uint32
	CompoundID uint64
	Flags      uint64
	ArchiveID  uint
	FsName     string
	Items      []HsmActionItem
}

HsmActionList is a list of actions received as a compound request from the coordinator.

func HsmCopytoolRecv 

func HsmCopytoolRecv(hcp *HsmCopytoolPrivate) (*HsmActionList, error)

HsmCopytoolRecv retuns a list of actions received from the coordinator.

type HsmCopyActionPrivate 

type HsmCopyActionPrivate C.struct_hsm_copyaction_private

HsmCopyActionPrivate is an opaque that represents an action item thas has been started.

func HsmActionBegin 

func HsmActionBegin(hcp *HsmCopytoolPrivate, hai *HsmActionItem, mdtIndex int, openFlags int, isErr bool) (*HsmCopyActionPrivate, error)

HsmActionBegin initializes the action so it can be processed by the copytool.

type HsmCopytoolFlags 

type HsmCopytoolFlags int

HsmCopytoolFlags are options for initializing the connectino to the coordinator

type HsmCopytoolPrivate 

type HsmCopytoolPrivate C.struct_hsm_copytool_private

HsmCopytoolPrivate is an opaque value representing the connection to the coordinator.

func HsmCopytoolRegister 

func HsmCopytoolRegister(path string, archiveCount int, archives []int, flags HsmCopytoolFlags) (*HsmCopytoolPrivate, error)

HsmCopytoolRegister connects the agent to the HSM Coordinators on all the MDTs. if CopytooLNonBLock flag is passed, then the HsmCopytoolRecv() will not block and poll() could used on the coordinator's descriptor.

type HsmCurrentAction 

type HsmCurrentAction struct {
	Action   HsmUserAction
	State    HsmProgressState
	Location *HsmExtent
}

HsmCurrentAction describes the current in-progress action for a file

func GetHsmCurrentAction 

func GetHsmCurrentAction(filePath string) (*HsmCurrentAction, error)

GetHsmCurrentAction returns the current HSM action for the given file.

func (*HsmCurrentAction) String 

func (hca *HsmCurrentAction) String() string

type HsmEvent 

type HsmEvent int32

HsmEvent is a convenience type to represent an HSM event reported in a changelog record's flags.

func (*HsmEvent) String 

func (he *HsmEvent) String() string

type HsmExtent 

type HsmExtent struct {
	Offset int64
	Length int64
}

HsmExtent is range of data in a file.

func (HsmExtent) String 

func (e HsmExtent) String() string

type HsmFileState 

type HsmFileState uint32

HsmFileState is a bitmask containing the hsm state(s) for a file.

func GetHsmFileStatus 

func GetHsmFileStatus(filePath string) (HsmFileState, uint32, error)

GetHsmFileStatus returns the HSM state and archive number for the given file.

func (HsmFileState) Flags 

func (s HsmFileState) Flags() []string

Flags returns a list of flag strings

func (HsmFileState) HasFlag 

func (s HsmFileState) HasFlag(flag HsmStateFlag) bool

HasFlag checks to see if the supplied flag matches

func (HsmFileState) String 

func (s HsmFileState) String() string

type HsmProgressState 

type HsmProgressState uint32

HsmProgressState describes the HSM request progress state

func (HsmProgressState) String 

func (hps HsmProgressState) String() string

type HsmStateFlag 

type HsmStateFlag uint32

HsmStateFlag represents a given HSM state flag

func (HsmStateFlag) String 

func (f HsmStateFlag) String() string

type HsmUserAction 

type HsmUserAction uint

HsmUserAction specifies an action for HsmRequest().

func (HsmUserAction) String 

func (action HsmUserAction) String() string

type OstData 

type OstData struct {
	Object lustre.Fid
	Gen    int
	Index  int
}

OstData is an element of a stripe layout

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.