inode

package
v2.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2024 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Index

Constants

View Source 
const ConflictingFileNameSuffix = "\n"

A suffix that can be used to unambiguously tag a file system name. (Unambiguous because U+000A is not allowed in GCS object names.) This is used to refer to the file/symlink in a (file/symlink, directory) pair with conflicting object names.

See also the notes on DirInode.LookUpChild.

View Source 
const FileMtimeMetadataKey = gcsx.MtimeMetadataKey

A GCS object metadata key for file mtimes. mtimes are UTC, and are stored in the format defined by time.RFC3339Nano.

View Source 
const MaxResultsForListObjectsCall = 5000

ListObjects call supports fetching upto 5000 results when projection is noAcl via maxResults param in one call. When projection is set to full, it returns 2000 results max. In GcsFuse flows we will be setting projection as noAcl. By default 1000 results are returned if maxResults is not set. Defining a constant to set maxResults param.

View Source 
const SymlinkMetadataKey = "gcsfuse_symlink_target"

When this custom metadata key is present in an object record, it is to be treated as a symlink. For use in testing only; other users should detect this with IsSymlink.

Variables

This section is empty.

Functions 

func IsSymlink(m *gcs.MinObject) bool

IsSymlink Does the supplied object represent a symlink inode?

Types

type BucketOwnedDirInode 

type BucketOwnedDirInode interface {
	DirInode
	BucketOwnedInode
}

An inode that represents a directory from a GCS bucket.

type BucketOwnedInode 

type BucketOwnedInode interface {
	Inode

	// Return the gcs.Bucket which the dir or file belongs to.
	Bucket() *gcsx.SyncerBucket
}

An inode owned by a gcs bucket.

type Core 

type Core struct {
	// The full name of the file or directory. Required for all inodes.
	FullName Name

	// The bucket that backs up the inode. Required for all inodes except the
	// base directory that holds all the buckets mounted.
	Bucket *gcsx.SyncerBucket

	// The GCS object in the bucket above that backs up the inode. Can be empty
	// if the inode is the base directory or an implicit directory.
	MinObject *gcs.MinObject

	// Specifies a local object which is not yet synced to GCS.
	Local bool
}

Core contains critical information about an inode before its creation.

func (*Core) Exists 

func (c *Core) Exists() bool

Exists returns true iff the back object exists implicitly or explicitly.

func (Core) SanityCheck 

func (c Core) SanityCheck() error

SanityCheck returns an error if the object is conflicting with itself, which means the metadata of the file system is broken.

func (*Core) Type 

func (c *Core) Type() metadata.Type

type DirInode 

type DirInode interface {
	Inode

	// Look up the direct child with the given relative name, returning
	// information about the object backing the child or whether it exists as an
	// implicit directory. If a file/symlink and a directory with the given name
	// both exist, the directory is preferred. Return nil result and a nil error
	// if neither is found.
	//
	// Special case: if the name ends in ConflictingFileNameSuffix, we strip the
	// suffix, confirm that a conflicting directory exists, then return a result
	// for the file/symlink.
	//
	// If this inode was created with implicitDirs is set, this method will use
	// ListObjects to find child directories that are "implicitly" defined by the
	// existence of their own descendents. For example, if there is an object
	// named "foo/bar/baz" and this is the directory "foo", a child directory
	// named "bar" will be implied. In this case, result.ImplicitDir will be
	// true.
	LookUpChild(ctx context.Context, name string) (*Core, error)

	// Read the children objects of this dir, recursively. The result count
	// is capped at the given limit. Internal caches are not refreshed from this
	// call.
	ReadDescendants(ctx context.Context, limit int) (map[Name]*Core, error)

	// Read some number of entries from the directory, returning a continuation
	// token that can be used to pick up the read operation where it left off.
	// Supply the empty token on the first call.
	//
	// At the end of the directory, the returned continuation token will be
	// empty. Otherwise it will be non-empty. There is no guarantee about the
	// number of entries returned; it may be zero even with a non-empty
	// continuation token.
	//
	// The contents of the Offset and Inode fields for returned entries is
	// undefined.
	ReadEntries(
		ctx context.Context,
		tok string) (entries []fuseutil.Dirent, newTok string, err error)

	// Create an empty child file with the supplied (relative) name, failing with
	// *gcs.PreconditionError if a backing object already exists in GCS.
	// Return the full name of the child and the GCS object it backs up.
	CreateChildFile(ctx context.Context, name string) (*Core, error)

	// Create an empty local child file with the supplied (relative) name. Local
	// file means the object is not yet created in GCS.
	CreateLocalChildFile(name string) (*Core, error)

	// Like CreateChildFile, except clone the supplied source object instead of
	// creating an empty object.
	// Return the full name of the child and the GCS object it backs up.
	CloneToChildFile(ctx context.Context, name string, src *gcs.MinObject) (*Core, error)

	// Create a symlink object with the supplied (relative) name and the supplied
	// target, failing with *gcs.PreconditionError if a backing object already
	// exists in GCS.
	// Return the full name of the child and the GCS object it backs up.
	CreateChildSymlink(ctx context.Context, name string, target string) (*Core, error)

	// Create a backing object for a child directory with the supplied (relative)
	// name, failing with *gcs.PreconditionError if a backing object already
	// exists in GCS.
	// Return the full name of the child and the GCS object it backs up.
	CreateChildDir(ctx context.Context, name string) (*Core, error)

	// Delete the backing object for the child file or symlink with the given
	// (relative) name and generation number, where zero means the latest
	// generation. If the object/generation doesn't exist, no error is returned.
	//
	// metaGeneration may be set to a non-nil pointer giving a meta-generation
	// precondition, but need not be.
	DeleteChildFile(
		ctx context.Context,
		name string,
		generation int64,
		metaGeneration *int64) (err error)

	// Delete the backing object for the child directory with the given
	// (relative) name if it is not an Implicit Directory.
	DeleteChildDir(
		ctx context.Context,
		name string,
		isImplicitDir bool) (err error)

	// LocalFileEntries lists the local files present in the directory.
	// Local means that the file is not yet present on GCS.
	LocalFileEntries(localFileInodes map[Name]Inode) (localEntries []fuseutil.Dirent)
}

An inode representing a directory, with facilities for listing entries, looking up children, and creating and deleting children. Must be locked for any method additional to the Inode interface.

func NewBaseDirInode 

func NewBaseDirInode(
	id fuseops.InodeID,
	name Name,
	attrs fuseops.InodeAttributes,
	bm gcsx.BucketManager) (d DirInode)

NewBaseDirInode returns a baseDirInode that acts as the directory of buckets.

func NewDirInode 

func NewDirInode(
	id fuseops.InodeID,
	name Name,
	attrs fuseops.InodeAttributes,
	implicitDirs bool,
	enableManagedFoldersListing bool,
	enableNonexistentTypeCache bool,
	typeCacheTTL time.Duration,
	bucket *gcsx.SyncerBucket,
	mtimeClock timeutil.Clock,
	cacheClock timeutil.Clock,
	typeCacheMaxSizeMB int) (d DirInode)

Create a directory inode for the name, representing the directory containing the objects for which it is an immediate prefix. For the root directory, this is the empty string.

If implicitDirs is set, LookUpChild will use ListObjects to find child directories that are "implicitly" defined by the existence of their own descendents. For example, if there is an object named "foo/bar/baz" and this is the directory "foo", a child directory named "bar" will be implied.

If typeCacheTTL is non-zero, a cache from child name to information about whether that name exists as a file/symlink and/or directory will be maintained. This may speed up calls to LookUpChild, especially when combined with a stat-caching GCS bucket, but comes at the cost of consistency: if the child is removed and recreated with a different type before the expiration, we may fail to find it.

The initial lookup count is zero.

REQUIRES: name.IsDir()

type ExplicitDirInode 

type ExplicitDirInode interface {
	DirInode
	SourceGeneration() Generation
}

An inode representing a directory backed by an object in GCS with a specific generation.

func NewExplicitDirInode 

func NewExplicitDirInode(
	id fuseops.InodeID,
	name Name,
	m *gcs.MinObject,
	attrs fuseops.InodeAttributes,
	implicitDirs bool,
	enableManagedFoldersListing bool,
	enableNonexistentTypeCache bool,
	typeCacheTTL time.Duration,
	bucket *gcsx.SyncerBucket,
	mtimeClock timeutil.Clock,
	cacheClock timeutil.Clock,
	typeCacheMaxSizeMB int) (d ExplicitDirInode)

Create an explicit dir inode backed by the supplied object. See notes on NewDirInode for more.

type FileInode 

type FileInode struct {
	// contains filtered or unexported fields
}

func NewFileInode 

func NewFileInode(
	id fuseops.InodeID,
	name Name,
	m *gcs.MinObject,
	attrs fuseops.InodeAttributes,
	bucket *gcsx.SyncerBucket,
	localFileCache bool,
	contentCache *contentcache.ContentCache,
	mtimeClock timeutil.Clock,
	localFile bool) (f *FileInode)

Create a file inode for the given min object in GCS. The initial lookup count is zero.

REQUIRES: m != nil REQUIRES: m.Generation > 0 REQUIRES: m.MetaGeneration > 0 REQUIRES: len(m.Name) > 0 REQUIRES: m.Name[len(m.Name)-1] != '/'

func (*FileInode) Attributes 

func (f *FileInode) Attributes(
	ctx context.Context) (attrs fuseops.InodeAttributes, err error)

LOCKS_REQUIRED(f.mu)

func (*FileInode) Bucket 

func (f *FileInode) Bucket() *gcsx.SyncerBucket

func (*FileInode) CacheEnsureContent 

func (f *FileInode) CacheEnsureContent(ctx context.Context) (err error)

Ensures cache content on read if content cache enabled

func (*FileInode) CreateEmptyTempFile 

func (f *FileInode) CreateEmptyTempFile() (err error)

func (*FileInode) DecrementLookupCount 

func (f *FileInode) DecrementLookupCount(n uint64) (destroy bool)

LOCKS_REQUIRED(f.mu)

func (*FileInode) Destroy 

func (f *FileInode) Destroy() (err error)

LOCKS_REQUIRED(f.mu)

func (*FileInode) ID 

func (f *FileInode) ID() fuseops.InodeID

func (*FileInode) IncrementLookupCount 

func (f *FileInode) IncrementLookupCount()

LOCKS_REQUIRED(f.mu)

func (*FileInode) IsLocal 

func (f *FileInode) IsLocal() bool

func (*FileInode) IsUnlinked 

func (f *FileInode) IsUnlinked() bool

func (*FileInode) Lock 

func (f *FileInode) Lock()

func (*FileInode) Name 

func (f *FileInode) Name() Name

func (*FileInode) Read 

func (f *FileInode) Read(
	ctx context.Context,
	dst []byte,
	offset int64) (n int, err error)

Serve a read for this file with semantics matching io.ReaderAt.

The caller may be better off reading directly from GCS when f.SourceGenerationIsAuthoritative() is true.

LOCKS_REQUIRED(f.mu)

func (*FileInode) SetMtime 

func (f *FileInode) SetMtime(
	ctx context.Context,
	mtime time.Time) (err error)

Set the mtime for this file. May involve a round trip to GCS.

LOCKS_REQUIRED(f.mu)

func (*FileInode) Source 

func (f *FileInode) Source() *gcs.MinObject

Source returns a record for the GCS object from which this inode is branched. The record is guaranteed not to be modified, and users must not modify it.

LOCKS_REQUIRED(f.mu)

func (*FileInode) SourceGeneration 

func (f *FileInode) SourceGeneration() (g Generation)

Equivalent to the generation returned by f.Source().

LOCKS_REQUIRED(f)

func (*FileInode) SourceGenerationIsAuthoritative 

func (f *FileInode) SourceGenerationIsAuthoritative() bool

If true, it is safe to serve reads directly from the object given by f.Source(), rather than calling f.ReadAt. Doing so may be more efficient, because f.ReadAt may cause the entire object to be faulted in and requires the inode to be locked during the read.

LOCKS_REQUIRED(f.mu)

func (*FileInode) Sync 

func (f *FileInode) Sync(ctx context.Context) (err error)

Sync writes out contents to GCS. If this fails due to the generation having been clobbered, treat it as a non-error (simulating the inode having been unlinked).

After this method succeeds, SourceGeneration will return the new generation by which this inode should be known (which may be the same as before). If it fails, the generation will not change.

LOCKS_REQUIRED(f.mu)

func (*FileInode) Truncate 

func (f *FileInode) Truncate(
	ctx context.Context,
	size int64) (err error)

Truncate the file to the specified size.

LOCKS_REQUIRED(f.mu) 

func (f *FileInode) Unlink()

func (*FileInode) Unlock 

func (f *FileInode) Unlock()

func (*FileInode) Write 

func (f *FileInode) Write(
	ctx context.Context,
	data []byte,
	offset int64) (err error)

Serve a write for this file with semantics matching fuseops.WriteFileOp.

LOCKS_REQUIRED(f.mu)

type Generation 

type Generation struct {
	Object   int64
	Metadata int64
}

A particular generation of a GCS object, consisting of both a GCS object generation number and meta-generation number. Lexicographically ordered on the two.

Cf. https://cloud.google.com/storage/docs/generations-preconditions

func (Generation) Compare 

func (g Generation) Compare(other Generation) int

Compare returns -1, 0, or 1 according to whether g is less than, equal to, or greater than other.

type GenerationBackedInode 

type GenerationBackedInode interface {
	Inode

	// Requires the inode lock.
	SourceGeneration() Generation
}

An inode that is backed by a particular generation of a GCS object.

type Inode 

type Inode interface {
	// All methods below require the lock to be held unless otherwise documented.
	sync.Locker

	// Return the ID assigned to the inode.
	//
	// Does not require the lock to be held.
	ID() fuseops.InodeID

	// Return the name of the GCS object backing the inode.
	//
	// Does not require the lock to be held.
	Name() Name

	// Increment the lookup count for the inode. For use in fuse operations where
	// the kernel expects us to remember the inode.
	IncrementLookupCount()

	// Return up to date attributes for this inode.
	Attributes(ctx context.Context) (fuseops.InodeAttributes, error)

	// Decrement the lookup count for the inode by the given amount.
	//
	// If this method returns true, the lookup count has hit zero and the
	// Destroy() method should be called to release any local resources, perhaps
	// after releasing locks that should not be held while blocking.
	DecrementLookupCount(n uint64) (destroy bool)

	// Clean up any local resources used by the inode, putting it into an
	// indeterminate state where no method should be called except Unlock.
	//
	// This method may block. Errors are for logging purposes only.
	Destroy() (err error)
}

type Name 

type Name struct {
	// contains filtered or unexported fields
}

Name is the inode's name that can be interpreted in 2 ways: 

(1) LocalName: the name of the inode in the local file system.
(2) GcsObjectName: the name of its gcs object backed by the inode.

func NewDescendantName 

func NewDescendantName(ancestor Name, descendantObjectName string) Name

NewDescendant creates a new inode name for an object as a descendant of another inode.

func NewDirName 

func NewDirName(parentName Name, dirName string) Name

NewDirName creates a new inode name for a directory.

func NewFileName 

func NewFileName(parentName Name, fileName string) Name

NewFileName creates a new inode name for a file.

func NewRootName 

func NewRootName(bucketName string) Name

NewRootName creates a Name for the root directory of a gcs bucket

func (Name) GcsObjectName 

func (name Name) GcsObjectName() string

GcsObjectName returns the name of the gcs object backed by the inode.

func (Name) IsBucketRoot 

func (name Name) IsBucketRoot() bool

IsBucketRoot returns true if the name represents of a root directory of a GCS bucket.

func (Name) IsDir 

func (name Name) IsDir() bool

IsDir returns true if the name represents a directory.

func (Name) IsDirectChildOf 

func (name Name) IsDirectChildOf(parent Name) bool

IsDirectChildOf returns true if the name is a direct child file or directory of another directory.

func (Name) IsFile 

func (name Name) IsFile() bool

IsFile returns true if the name represents a file.

func (Name) LocalName 

func (name Name) LocalName() string

LocalName returns the name of the directory or file in the local file system.

func (Name) String 

func (name Name) String() string

String returns LocalName.

type SymlinkInode 

type SymlinkInode struct {
	// contains filtered or unexported fields
}

func NewSymlinkInode 

func NewSymlinkInode(
	id fuseops.InodeID,
	name Name,
	m *gcs.MinObject,
	attrs fuseops.InodeAttributes) (s *SymlinkInode)

Create a symlink inode for the supplied object record.

REQUIRES: IsSymlink(o)

func (*SymlinkInode) Attributes 

func (s *SymlinkInode) Attributes(
	ctx context.Context) (attrs fuseops.InodeAttributes, err error)

func (*SymlinkInode) DecrementLookupCount 

func (s *SymlinkInode) DecrementLookupCount(n uint64) (destroy bool)

LOCKS_REQUIRED(s.mu)

func (*SymlinkInode) Destroy 

func (s *SymlinkInode) Destroy() (err error)

LOCKS_REQUIRED(s.mu)

func (*SymlinkInode) ID 

func (s *SymlinkInode) ID() fuseops.InodeID

func (*SymlinkInode) IncrementLookupCount 

func (s *SymlinkInode) IncrementLookupCount()

LOCKS_REQUIRED(s.mu)

func (*SymlinkInode) Lock 

func (s *SymlinkInode) Lock()

func (*SymlinkInode) Name 

func (s *SymlinkInode) Name() Name

func (*SymlinkInode) SourceGeneration 

func (s *SymlinkInode) SourceGeneration() Generation

SourceGeneration returns the object generation from which this inode was branched.

LOCKS_REQUIRED(s)

func (*SymlinkInode) Target 

func (s *SymlinkInode) Target() (target string)

Target returns the target of the symlink.

func (*SymlinkInode) Unlock 

func (s *SymlinkInode) Unlock()

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.