fsstate

Documentation

Index

• Constants
• type AccessedFileState
• type ActivityCounter
  • func NewActivityCounter() *ActivityCounter
  • func (c *ActivityCounter) AddToWantedWrite(path string)
  • func (c *ActivityCounter) AllowWrites()
  • func (c *ActivityCounter) ClearWantedWrite()
  • func (c *ActivityCounter) ForEachWantedWrite(fn func(path string) bool)
  • func (c *ActivityCounter) GetActivityCount() uint64
  • func (c *ActivityCounter) GetLastChangedAt() time.Time
  • func (c *ActivityCounter) GetLastSeenActivityCount() uint64
  • func (c *ActivityCounter) IncrementActivityCount()
  • func (c *ActivityCounter) PreventWrites()
  • func (c *ActivityCounter) SetLastChangedAt(t time.Time)
  • func (c *ActivityCounter) SetLastSeenActivityCount(val uint64)
  • func (c *ActivityCounter) WritesNotAllowed() bool
• type ActivityTracker
  • func NewActivityTracker() *ActivityTracker
  • func (m *ActivityTracker) ForEach(fn func(key string, ac *ActivityCounter) error)
  • func (m *ActivityTracker) GetActivityCounter(key string) *ActivityCounter
  • func (m *ActivityTracker) GetOrCreateActivityCounter(key string) *ActivityCounter
  • func (m *ActivityTracker) RemoveActivityFromTracking(key string)
• type ExpiredActivityHandlerFN
• type FSState
  • func NewFSState(tstateTracker *TransferStateTracker, trCache *TransferRequestCache, ...) *FSState
  • func (s *FSState) GetCache() *TransferRequestCache
  • func (s *FSState) RemoveTransferRequestState(uuid string)
• type GlobusTask
• type LoadOrStoreFN
• type TransferRequestCache
  • func NewTransferRequestCache(trStor stor.TransferRequestStor) *TransferRequestCache
  • func (c *TransferRequestCache) GetTransferRequestByUUID(uuid string) (*mcmodel.TransferRequest, error)
  • func (c *TransferRequestCache) RemoveTransferRequestByUUID(uuid string)
• type TransferRequestState
• type TransferStateTracker
  • func NewTransferStateTracker() *TransferStateTracker
  • func (tracker *TransferStateTracker) DeleteBase(transferRequestKey string)
  • func (tracker *TransferStateTracker) DeletePath(transferRequestKey, path string)
  • func (tracker *TransferStateTracker) Get(transferRequestKey, path string) *AccessedFileState
  • func (tracker *TransferStateTracker) GetFile(transferRequestKey, path string) *mcmodel.File
  • func (tracker *TransferStateTracker) GetFileWithHashReset(transferRequestKey, path string) *mcmodel.File
  • func (tracker *TransferStateTracker) LoadOrStore(transferRequestKey, path string, fn LoadOrStoreFN) error
  • func (tracker *TransferStateTracker) Store(transferRequestKey, path string, file *mcmodel.File, state string) (bool, *AccessedFileState)
  • func (tracker *TransferStateTracker) WithLockHeld(transferRequestKey, path string, fn func(fileState *AccessedFileState)) bool

Constants

const FileStateClosed = "closed"

const FileStateOpen = "open"

Types

type AccessedFileState

type AccessedFileState struct {
	// File is the database file the file system file is associated with
	File *mcmodel.File

	// Hasher is used to create the checksum. As writes are done to a file the
	// checksum is updated.
	Hasher hash.Hash

	// HashInvalid is set to true when a user seeks or truncates a file. When
	// that happens the hash state is invalid.
	HashInvalid bool

	// Last time the file was closed
	LastClosedAt time.Time

	// Current state of the file - Open or Closed
	FileState string
}

AccessedFileState tracks the state of a file that was opened for write. This tracks the underlying database file entry and the hasher used to construct the checksum.

type ActivityCounter

type ActivityCounter struct {
	// contains filtered or unexported fields
}

ActivityCounter tracks activity against an object. It atomically updates activityCount. The LastSeenActivityCount and LastChanged are meant to be used by a monitor for tracking how activity has changed.

func NewActivityCounter

func NewActivityCounter() *ActivityCounter

NewActivityCounter creates a new ActivityCounter with LastChanged set to the current time.

func (*ActivityCounter) AddToWantedWrite

func (c *ActivityCounter) AddToWantedWrite(path string)

func (*ActivityCounter) AllowWrites

func (c *ActivityCounter) AllowWrites()

func (*ActivityCounter) ClearWantedWrite

func (c *ActivityCounter) ClearWantedWrite()

func (*ActivityCounter) ForEachWantedWrite

func (c *ActivityCounter) ForEachWantedWrite(fn func(path string) bool)

func (*ActivityCounter) GetActivityCount

func (c *ActivityCounter) GetActivityCount() uint64

func (*ActivityCounter) GetLastChangedAt

func (c *ActivityCounter) GetLastChangedAt() time.Time

func (*ActivityCounter) GetLastSeenActivityCount

func (c *ActivityCounter) GetLastSeenActivityCount() uint64

func (*ActivityCounter) IncrementActivityCount

func (c *ActivityCounter) IncrementActivityCount()

IncrementActivityCount atomically updates the activityCount

func (*ActivityCounter) PreventWrites

func (c *ActivityCounter) PreventWrites()

func (*ActivityCounter) SetLastChangedAt

func (c *ActivityCounter) SetLastChangedAt(t time.Time)

func (*ActivityCounter) SetLastSeenActivityCount

func (c *ActivityCounter) SetLastSeenActivityCount(val uint64)

func (*ActivityCounter) WritesNotAllowed

func (c *ActivityCounter) WritesNotAllowed() bool

type ActivityTracker

type ActivityTracker struct {
	// contains filtered or unexported fields
}

func NewActivityTracker

func NewActivityTracker() *ActivityTracker

func (*ActivityTracker) ForEach

func (m *ActivityTracker) ForEach(fn func(key string, ac *ActivityCounter) error)

func (*ActivityTracker) GetActivityCounter

func (m *ActivityTracker) GetActivityCounter(key string) *ActivityCounter

func (*ActivityTracker) GetOrCreateActivityCounter

func (m *ActivityTracker) GetOrCreateActivityCounter(key string) *ActivityCounter

func (*ActivityTracker) RemoveActivityFromTracking

func (m *ActivityTracker) RemoveActivityFromTracking(key string)

type ExpiredActivityHandlerFN

type ExpiredActivityHandlerFN func(activityKey string)

type FSState

type FSState struct {
	ActivityTracker      *ActivityTracker
	TransferStateTracker *TransferStateTracker
	TransferRequestCache *TransferRequestCache
}

func NewFSState

func NewFSState(tstateTracker *TransferStateTracker, trCache *TransferRequestCache, activityTracker *ActivityTracker) *FSState

func (*FSState) GetCache

func (s *FSState) GetCache() *TransferRequestCache

func (*FSState) RemoveTransferRequestState

func (s *FSState) RemoveTransferRequestState(uuid string)

type GlobusTask

type GlobusTask struct {
	Task      globus.Task
	Transfers []globus.Transfer
}

type LoadOrStoreFN

type LoadOrStoreFN func(fileState *AccessedFileState) (*mcmodel.File, error)

type TransferRequestCache

type TransferRequestCache struct {
	// contains filtered or unexported fields
}

func NewTransferRequestCache

func NewTransferRequestCache(trStor stor.TransferRequestStor) *TransferRequestCache

func (*TransferRequestCache) GetTransferRequestByUUID

func (c *TransferRequestCache) GetTransferRequestByUUID(uuid string) (*mcmodel.TransferRequest, error)

func (*TransferRequestCache) RemoveTransferRequestByUUID

func (c *TransferRequestCache) RemoveTransferRequestByUUID(uuid string)

type TransferRequestState

type TransferRequestState struct {
	GlobusTasks        []GlobusTask
	AccessedFileStates map[string]*AccessedFileState
}

type TransferStateTracker

type TransferStateTracker struct {

	// A map of known files. This is a two level map. The first key is a unique tied to an entire
	// upload instance. For example a transfer request uuid or a project/user combination. The
	// second map is keyed by the path for the project.
	TransferRequestStates map[string]*TransferRequestState
	// contains filtered or unexported fields
}

The TransferStateTracker is used to track files the filesystem has already seen. The paths that are stored include the user/project combination. This means that there could be multiple instances of the same project file path, but different for each user. This is ok, because each these will represent a different file version because each user will get their own version when opened for write.

Note: Files are only stored if opened for write. That means if a file is opened for write, then later opened for read, it will refer to the version opened for write. All calls on a TransferStateTracker are thread safe.

func NewTransferStateTracker

func NewTransferStateTracker() *TransferStateTracker

func (*TransferStateTracker) DeleteBase

func (tracker *TransferStateTracker) DeleteBase(transferRequestKey string)

func (*TransferStateTracker) DeletePath

func (tracker *TransferStateTracker) DeletePath(transferRequestKey, path string)

DeletePath will delete path from the tracker. It doesn't check for existence.

func (*TransferStateTracker) Get

func (tracker *TransferStateTracker) Get(transferRequestKey, path string) *AccessedFileState

Get will return the AccessedFileState entry if path exists. Otherwise, it will return nil.

func (*TransferStateTracker) GetFile

func (tracker *TransferStateTracker) GetFile(transferRequestKey, path string) *mcmodel.File

GetFile will return the &mcmodel.File entry in the AccessedFileState if path exists. Otherwise, it will return nil.

func (*TransferStateTracker) GetFileWithHashReset

func (tracker *TransferStateTracker) GetFileWithHashReset(transferRequestKey, path string) *mcmodel.File

func (*TransferStateTracker) LoadOrStore

func (tracker *TransferStateTracker) LoadOrStore(transferRequestKey, path string, fn LoadOrStoreFN) error

LoadOrStore calls fn within the context of the mutex lock. It passes the entry found at path, or nil if there wasn't an entry at that path. The function can then conduct any operations within the context of the lock. If it encounters an error then an error is returned from LoadOrStore. If no error is returned from the function, but a *mcmodel.File is returned, then a new AccessedFileState entry will be created. If there is no error and no *mcmodel.File is returned then nothing will happen.

This function exists so that a set of operations can be completed before loading a known file. It prevents a race condition where other file system calls made don't complete before the known files tracker is loaded.

func (*TransferStateTracker) Store

func (tracker *TransferStateTracker) Store(transferRequestKey, path string, file *mcmodel.File, state string) (bool, *AccessedFileState)

Store grabs the mutex and stores the entry. It returns false if an entry already existed (and doesn't update it), otherwise it returns true and adds the path and a new AccessedFileState to the tracker.

func (*TransferStateTracker) WithLockHeld

func (tracker *TransferStateTracker) WithLockHeld(transferRequestKey, path string, fn func(fileState *AccessedFileState)) bool

WithLockHeld finds the AccessedFileState associated with path and calls fn. The call to fn happens while the tracker mutex is held. This way fn knows that the AccessedFileState entry cannot change while fn is running. WithLockHeld returns true if it found a matching AccessedFileState. It returns false if path didn't match a AccessedFileState. When no match is found, fn is **not** called.