vfs

package
v1.27.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 2, 2024 License: Apache-2.0 Imports: 48 Imported by: 1,770

Documentation

Index

Constants

This section is empty.

Variables

View Source
var Context = NewVFSContext()

Context holds the global VFS state. Deprecated: prefer FromContext.

Functions

func AWSErrorCode

func AWSErrorCode(err error) string

AWSErrorCode returns the aws error code, if it is an awserr.Error, otherwise ""

func IsClusterReadable

func IsClusterReadable(p Path) bool

func IsDirectory

func IsDirectory(p Path) bool

func NewSwiftClient added in v1.10.0

func NewSwiftClient(ctx context.Context) (*gophercloud.ServiceClient, error)

func RelativePath

func RelativePath(base Path, child Path) (string, error)

func RetryWithBackoff added in v1.10.0

func RetryWithBackoff(backoff wait.Backoff, condition func() (bool, error)) (bool, error)

RetryWithBackoff runs until a condition function returns true, or until Steps attempts have been taken As compared to wait.ExponentialBackoff, this function returns the results from the function on the final attempt

func VFSPath added in v1.10.0

func VFSPath(url string) (string, error)

func WithContext added in v1.26.0

func WithContext(parent context.Context, vfsContext *VFSContext) context.Context

Types

type ACL added in v1.10.0

type ACL interface{}

type ACLOracle added in v1.10.0

type ACLOracle func(Path) (ACL, error)

type AzureBlobPath added in v1.20.0

type AzureBlobPath struct {
	// contains filtered or unexported fields
}

AzureBlobPath is a path in the VFS space backed by Azure Blob.

func NewAzureBlobPath added in v1.20.0

func NewAzureBlobPath(vfsContext *VFSContext, container string, key string) *AzureBlobPath

NewAzureBlobPath returns a new AzureBlobPath.

func (*AzureBlobPath) Base added in v1.20.0

func (p *AzureBlobPath) Base() string

Base returns the base name (last element).

func (*AzureBlobPath) CreateFile added in v1.20.0

func (p *AzureBlobPath) CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

CreateFile writes the file contents only if the file does not already exist.

func (*AzureBlobPath) Hash added in v1.20.0

Hash gets the hash, or nil if the hash cannot be (easily) computed.

func (*AzureBlobPath) Join added in v1.20.0

func (p *AzureBlobPath) Join(relativePath ...string) Path

Join returns a new path that joins the current path and given relative paths.

func (*AzureBlobPath) Path added in v1.20.0

func (p *AzureBlobPath) Path() string

Path returns a string representing the full path.

func (*AzureBlobPath) PreferredHash added in v1.20.0

func (p *AzureBlobPath) PreferredHash() (*hashing.Hash, error)

PreferredHash returns the hash of the file contents, with the preferred hash algorithm.

func (*AzureBlobPath) ReadDir added in v1.20.0

func (p *AzureBlobPath) ReadDir() ([]Path, error)

ReadDir lists the blobs under the current Path.

func (*AzureBlobPath) ReadFile added in v1.20.0

func (p *AzureBlobPath) ReadFile(ctx context.Context) ([]byte, error)

ReadFile returns the content of the blob.

func (*AzureBlobPath) ReadTree added in v1.20.0

func (p *AzureBlobPath) ReadTree() ([]Path, error)

ReadTree lists all blobs (recursively) in the subtree rooted at the current Path.

func (*AzureBlobPath) Remove added in v1.20.0

func (p *AzureBlobPath) Remove() error

Remove deletes the blob.

func (*AzureBlobPath) RemoveAllVersions added in v1.20.0

func (p *AzureBlobPath) RemoveAllVersions() error

func (*AzureBlobPath) WriteFile added in v1.20.0

func (p *AzureBlobPath) WriteFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

WriteFile writes the blob to the reader.

TODO(kenji): Support ACL.

func (*AzureBlobPath) WriteTo added in v1.20.0

func (p *AzureBlobPath) WriteTo(w io.Writer) (n int64, err error)

WriteTo writes the content of the blob to the writer.

type Cache added in v1.16.0

type Cache struct {
	// contains filtered or unexported fields
}

Cache is a simple cache for vfs files.

Currently we never expire the cache, so this is only safe for a relatively bounded set of files - but it would not be too hard to fix this.

func NewCache added in v1.16.0

func NewCache() *Cache

NewCache is a constructor for a Cache

func (*Cache) Read added in v1.16.0

func (c *Cache) Read(p Path, ttl time.Duration) ([]byte, error)

type FSPath

type FSPath struct {
	// contains filtered or unexported fields
}

func NewFSPath

func NewFSPath(location string) *FSPath

func (*FSPath) Base

func (p *FSPath) Base() string

func (*FSPath) CreateFile

func (p *FSPath) CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*FSPath) Hash

func (p *FSPath) Hash(a hashing.HashAlgorithm) (*hashing.Hash, error)

func (*FSPath) Join

func (p *FSPath) Join(relativePath ...string) Path

func (*FSPath) Path

func (p *FSPath) Path() string

func (*FSPath) PreferredHash

func (p *FSPath) PreferredHash() (*hashing.Hash, error)

func (*FSPath) ReadDir

func (p *FSPath) ReadDir() ([]Path, error)

func (*FSPath) ReadFile

func (p *FSPath) ReadFile(ctx context.Context) ([]byte, error)

ReadFile implements Path::ReadFile

func (*FSPath) ReadTree

func (p *FSPath) ReadTree() ([]Path, error)

func (*FSPath) Remove

func (p *FSPath) Remove() error

func (*FSPath) RemoveAllVersions added in v1.16.3

func (p *FSPath) RemoveAllVersions() error

func (*FSPath) String

func (p *FSPath) String() string

func (*FSPath) WriteFile

func (p *FSPath) WriteFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*FSPath) WriteTo added in v1.10.0

func (p *FSPath) WriteTo(out io.Writer) (int64, error)

WriteTo implements io.WriterTo

type GSAcl added in v1.10.0

type GSAcl struct {
	Acl []*storage.ObjectAccessControl
}

GSAcl is an ACL implementation for objects on Google Cloud Storage

func (*GSAcl) String added in v1.18.0

func (a *GSAcl) String() string

type GSPath added in v1.10.0

type GSPath struct {
	// contains filtered or unexported fields
}

GSPath is a vfs path for Google Cloud Storage

func NewGSPath added in v1.10.0

func NewGSPath(c *VFSContext, bucket string, key string) *GSPath

func (*GSPath) Base added in v1.10.0

func (p *GSPath) Base() string

func (*GSPath) Bucket added in v1.10.0

func (p *GSPath) Bucket() string

func (*GSPath) Client added in v1.10.0

func (p *GSPath) Client(ctx context.Context) (*storage.Service, error)

Client returns the storage.Service bound to this path

func (*GSPath) CreateFile added in v1.10.0

func (p *GSPath) CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*GSPath) Hash added in v1.10.0

func (p *GSPath) Hash(a hashing.HashAlgorithm) (*hashing.Hash, error)

func (*GSPath) Join added in v1.10.0

func (p *GSPath) Join(relativePath ...string) Path

func (*GSPath) Object added in v1.10.0

func (p *GSPath) Object() string

func (*GSPath) Path added in v1.10.0

func (p *GSPath) Path() string

func (*GSPath) PreferredHash added in v1.10.0

func (p *GSPath) PreferredHash() (*hashing.Hash, error)

func (*GSPath) ReadDir added in v1.10.0

func (p *GSPath) ReadDir() ([]Path, error)

ReadDir implements Path::ReadDir

func (*GSPath) ReadFile added in v1.10.0

func (p *GSPath) ReadFile(ctx context.Context) ([]byte, error)

ReadFile implements Path::ReadFile

func (*GSPath) ReadTree added in v1.10.0

func (p *GSPath) ReadTree() ([]Path, error)

ReadTree implements Path::ReadTree

func (*GSPath) Remove added in v1.10.0

func (p *GSPath) Remove() error

func (*GSPath) RemoveAllVersions added in v1.16.3

func (p *GSPath) RemoveAllVersions() error

func (*GSPath) RenderTerraform added in v1.23.0

func (p *GSPath) RenderTerraform(w *terraformWriter.TerraformWriter, name string, data io.Reader, acl ACL) error

func (*GSPath) String added in v1.10.0

func (p *GSPath) String() string
func (s *GSPath) TerraformLink(name string) *terraformWriter.Literal

func (*GSPath) WriteFile added in v1.10.0

func (p *GSPath) WriteFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*GSPath) WriteTo added in v1.10.0

func (p *GSPath) WriteTo(out io.Writer) (int64, error)

WriteTo implements io.WriterTo::WriteTo

type HasClusterReadable

type HasClusterReadable interface {
	IsClusterReadable() bool
}

type HasHash

type HasHash interface {
	// Returns the hash of the file contents, with the preferred hash algorithm
	PreferredHash() (*hashing.Hash, error)

	// Gets the hash, or nil if the hash cannot be (easily) computed
	Hash(algorithm hashing.HashAlgorithm) (*hashing.Hash, error)
}

type KubernetesContext added in v1.10.0

type KubernetesContext struct{}

KubernetesContext is the context for a Kubernetes VFS implementation

func NewKubernetesContext added in v1.10.0

func NewKubernetesContext() *KubernetesContext

NewKubernetesContext builds a KubernetesContext This will likely take a kubernetes rest client object (or similar) once the implementation is more complete

type KubernetesPath added in v1.10.0

type KubernetesPath struct {
	// contains filtered or unexported fields
}

KubernetesPath is a path for a VFS backed by the kubernetes API Currently all operations are no-ops

func (*KubernetesPath) Base added in v1.10.0

func (p *KubernetesPath) Base() string

func (*KubernetesPath) CreateFile added in v1.10.0

func (p *KubernetesPath) CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*KubernetesPath) Hash added in v1.10.0

func (*KubernetesPath) Host added in v1.10.0

func (p *KubernetesPath) Host() string

func (*KubernetesPath) Join added in v1.10.0

func (p *KubernetesPath) Join(relativePath ...string) Path

func (*KubernetesPath) Key added in v1.10.0

func (p *KubernetesPath) Key() string

func (*KubernetesPath) Path added in v1.10.0

func (p *KubernetesPath) Path() string

func (*KubernetesPath) PreferredHash added in v1.10.0

func (p *KubernetesPath) PreferredHash() (*hashing.Hash, error)

func (*KubernetesPath) ReadDir added in v1.10.0

func (p *KubernetesPath) ReadDir() ([]Path, error)

func (*KubernetesPath) ReadFile added in v1.10.0

func (p *KubernetesPath) ReadFile(ctx context.Context) ([]byte, error)

ReadFile implements Path::ReadFile

func (*KubernetesPath) ReadTree added in v1.10.0

func (p *KubernetesPath) ReadTree() ([]Path, error)

func (*KubernetesPath) Remove added in v1.10.0

func (p *KubernetesPath) Remove() error

func (*KubernetesPath) RemoveAllVersions added in v1.16.3

func (p *KubernetesPath) RemoveAllVersions() error

func (*KubernetesPath) String added in v1.10.0

func (p *KubernetesPath) String() string

func (*KubernetesPath) WriteFile added in v1.10.0

func (p *KubernetesPath) WriteFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*KubernetesPath) WriteTo added in v1.10.0

func (p *KubernetesPath) WriteTo(out io.Writer) (int64, error)

WriteTo implements io.WriterTo

type MemFSContext

type MemFSContext struct {
	// contains filtered or unexported fields
}

func NewMemFSContext

func NewMemFSContext() *MemFSContext

func (*MemFSContext) MarkClusterReadable

func (c *MemFSContext) MarkClusterReadable()

MarkClusterReadable pretends the current memfscontext is cluster readable; this is useful for tests

type MemFSPath

type MemFSPath struct {
	// contains filtered or unexported fields
}

func NewMemFSPath

func NewMemFSPath(context *MemFSContext, location string) *MemFSPath

func (*MemFSPath) Base

func (p *MemFSPath) Base() string

func (*MemFSPath) CreateFile

func (p *MemFSPath) CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*MemFSPath) HasChildren added in v1.10.0

func (c *MemFSPath) HasChildren() bool

func (*MemFSPath) IsClusterReadable

func (c *MemFSPath) IsClusterReadable() bool

func (*MemFSPath) IsPublic added in v1.24.0

func (p *MemFSPath) IsPublic() (bool, error)

func (*MemFSPath) Join

func (p *MemFSPath) Join(relativePath ...string) Path

func (*MemFSPath) Location added in v1.21.3

func (p *MemFSPath) Location() string

func (*MemFSPath) Path

func (p *MemFSPath) Path() string

func (*MemFSPath) ReadDir

func (p *MemFSPath) ReadDir() ([]Path, error)

func (*MemFSPath) ReadFile

func (p *MemFSPath) ReadFile(ctx context.Context) ([]byte, error)

ReadFile implements Path::ReadFile

func (*MemFSPath) ReadTree

func (p *MemFSPath) ReadTree() ([]Path, error)

func (*MemFSPath) Remove

func (p *MemFSPath) Remove() error

func (*MemFSPath) RemoveAllVersions added in v1.16.3

func (p *MemFSPath) RemoveAllVersions() error

func (*MemFSPath) RenderTerraform added in v1.21.3

func (p *MemFSPath) RenderTerraform(w *terraformWriter.TerraformWriter, name string, data io.Reader, acl ACL) error

func (*MemFSPath) String

func (p *MemFSPath) String() string

func (*MemFSPath) WriteFile

func (p *MemFSPath) WriteFile(ctx context.Context, r io.ReadSeeker, acl ACL) error

func (*MemFSPath) WriteTo added in v1.10.0

func (p *MemFSPath) WriteTo(out io.Writer) (int64, error)

WriteTo implements io.WriterTo

type OpenstackConfig added in v1.10.0

type OpenstackConfig struct{}

func (OpenstackConfig) GetCredential added in v1.10.0

func (oc OpenstackConfig) GetCredential() (gophercloud.AuthOptions, error)

func (OpenstackConfig) GetRegion added in v1.15.1

func (oc OpenstackConfig) GetRegion() (string, error)

func (OpenstackConfig) GetServiceConfig added in v1.10.0

func (oc OpenstackConfig) GetServiceConfig(name string) (gophercloud.EndpointOpts, error)

type Path

type Path interface {
	io.WriterTo
	Join(relativePath ...string) Path

	// ReadFile returns the contents of the file, or an error if the file could not be read.
	// If the file did not exist, err = os.ErrNotExist
	// As this reads the entire file into memory, consider using WriteTo for bigger files
	ReadFile(ctx context.Context) ([]byte, error)
	WriteFile(ctx context.Context, data io.ReadSeeker, acl ACL) error
	// CreateFile writes the file contents, but only if the file does not already exist
	CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

	// Remove deletes the file
	Remove() error

	// RemoveAllVersions completely deletes the file (with all its versions and markers).
	RemoveAllVersions() error

	// Base returns the base name (last element)
	Base() string

	// Path returns a string representing the full path
	Path() string

	// ReadDir lists the files in a particular Path
	ReadDir() ([]Path, error)

	// ReadTree lists all files (recursively) in the subtree rooted at the current Path
	/// Note: returns only files, not directories
	ReadTree() ([]Path, error)
}

Path is a path in the VFS space, which we can read, write, list etc

type S3Acl added in v1.10.0

type S3Acl struct {
	RequestACL *string
}

S3Acl is an ACL implementation for objects on S3

type S3BucketDetails added in v1.10.0

type S3BucketDetails struct {
	// contains filtered or unexported fields
}

type S3Context added in v1.4.1

type S3Context struct {
	// contains filtered or unexported fields
}

func NewS3Context added in v1.4.1

func NewS3Context() *S3Context

type S3Path

type S3Path struct {
	// contains filtered or unexported fields
}

func (*S3Path) Base

func (p *S3Path) Base() string

func (*S3Path) Bucket

func (p *S3Path) Bucket() string

func (*S3Path) CreateFile

func (p *S3Path) CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*S3Path) GetHTTPsUrl added in v1.21.0

func (p *S3Path) GetHTTPsUrl(dualstack bool) (string, error)

func (*S3Path) Hash

func (p *S3Path) Hash(a hashing.HashAlgorithm) (*hashing.Hash, error)

func (*S3Path) IsBucketPublic added in v1.26.0

func (p *S3Path) IsBucketPublic(ctx context.Context) (bool, error)

func (*S3Path) IsPublic added in v1.21.1

func (p *S3Path) IsPublic() (bool, error)

func (*S3Path) Join

func (p *S3Path) Join(relativePath ...string) Path

func (*S3Path) Key

func (p *S3Path) Key() string

func (*S3Path) Path

func (p *S3Path) Path() string

func (*S3Path) PreferredHash

func (p *S3Path) PreferredHash() (*hashing.Hash, error)

func (*S3Path) ReadDir

func (p *S3Path) ReadDir() ([]Path, error)

func (*S3Path) ReadFile

func (p *S3Path) ReadFile(ctx context.Context) ([]byte, error)

ReadFile implements Path::ReadFile

func (*S3Path) ReadTree

func (p *S3Path) ReadTree() ([]Path, error)

func (*S3Path) Remove

func (p *S3Path) Remove() error

func (*S3Path) RemoveAllVersions added in v1.16.3

func (p *S3Path) RemoveAllVersions() error

func (*S3Path) RenderTerraform added in v1.21.3

func (p *S3Path) RenderTerraform(w *terraformWriter.TerraformWriter, name string, data io.Reader, acl ACL) error

func (*S3Path) String

func (p *S3Path) String() string

func (*S3Path) WriteFile

func (p *S3Path) WriteFile(ctx context.Context, data io.ReadSeeker, aclObj ACL) error

func (*S3Path) WriteTo added in v1.10.0

func (p *S3Path) WriteTo(out io.Writer) (int64, error)

WriteTo implements io.WriterTo

type SSHAcl added in v1.10.0

type SSHAcl struct {
	Mode os.FileMode
}

type SSHPath

type SSHPath struct {
	// contains filtered or unexported fields
}

func NewSSHPath

func NewSSHPath(client *ssh.Client, server string, path string, sudo bool) *SSHPath

func (*SSHPath) Base

func (p *SSHPath) Base() string

func (*SSHPath) CreateFile

func (p *SSHPath) CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*SSHPath) Join

func (p *SSHPath) Join(relativePath ...string) Path

func (*SSHPath) Path

func (p *SSHPath) Path() string

func (*SSHPath) ReadDir

func (p *SSHPath) ReadDir() ([]Path, error)

func (*SSHPath) ReadFile

func (p *SSHPath) ReadFile(ctx context.Context) ([]byte, error)

ReadFile implements Path::ReadFile

func (*SSHPath) ReadTree

func (p *SSHPath) ReadTree() ([]Path, error)

func (*SSHPath) Remove

func (p *SSHPath) Remove() error

func (*SSHPath) RemoveAllVersions added in v1.16.3

func (p *SSHPath) RemoveAllVersions() error

func (*SSHPath) String

func (p *SSHPath) String() string

func (*SSHPath) WriteFile

func (p *SSHPath) WriteFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*SSHPath) WriteTo added in v1.10.0

func (p *SSHPath) WriteTo(out io.Writer) (int64, error)

WriteTo reads the file (in a streaming way) This implements io.WriterTo

type SwiftPath added in v1.10.0

type SwiftPath struct {
	// contains filtered or unexported fields
}

SwiftPath is a vfs path for Openstack Cloud Storage.

func NewSwiftPath added in v1.10.0

func NewSwiftPath(vfsContext *VFSContext, bucket string, key string) (*SwiftPath, error)

func (*SwiftPath) Base added in v1.10.0

func (p *SwiftPath) Base() string

func (*SwiftPath) Bucket added in v1.10.0

func (p *SwiftPath) Bucket() string

func (*SwiftPath) CreateFile added in v1.10.0

func (p *SwiftPath) CreateFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*SwiftPath) Hash added in v1.10.0

func (*SwiftPath) Join added in v1.10.0

func (p *SwiftPath) Join(relativePath ...string) Path

func (*SwiftPath) Path added in v1.10.0

func (p *SwiftPath) Path() string

func (*SwiftPath) PreferredHash added in v1.10.0

func (p *SwiftPath) PreferredHash() (*hashing.Hash, error)

func (*SwiftPath) ReadDir added in v1.10.0

func (p *SwiftPath) ReadDir() ([]Path, error)

ReadDir implements Path::ReadDir.

func (*SwiftPath) ReadFile added in v1.10.0

func (p *SwiftPath) ReadFile(ctx context.Context) ([]byte, error)

ReadFile implements Path::ReadFile

func (*SwiftPath) ReadTree added in v1.10.0

func (p *SwiftPath) ReadTree() ([]Path, error)

ReadTree implements Path::ReadTree.

func (*SwiftPath) Remove added in v1.10.0

func (p *SwiftPath) Remove() error

func (*SwiftPath) RemoveAllVersions added in v1.16.3

func (p *SwiftPath) RemoveAllVersions() error

func (*SwiftPath) String added in v1.10.0

func (p *SwiftPath) String() string

func (*SwiftPath) WriteFile added in v1.10.0

func (p *SwiftPath) WriteFile(ctx context.Context, data io.ReadSeeker, acl ACL) error

func (*SwiftPath) WriteTo added in v1.10.0

func (p *SwiftPath) WriteTo(out io.Writer) (int64, error)

WriteTo implements io.WriterTo

type TerraformPath added in v1.21.3

type TerraformPath interface {
	Path
	// RenderTerraform renders the file to a TerraformWriter.
	RenderTerraform(writer *terraformWriter.TerraformWriter, name string, data io.Reader, acl ACL) error
}

TerraformPath is a Path that can render to Terraform.

type VFS

type VFS interface{}

type VFSContext

type VFSContext struct {
	// contains filtered or unexported fields
}

VFSContext is a 'context' for VFS, that is normally a singleton but allows us to configure S3 credentials, for example

func FromContext added in v1.26.0

func FromContext(ctx context.Context) *VFSContext

func NewVFSContext added in v1.26.0

func NewVFSContext() *VFSContext

NewVFSContext builds a new VFSContext

func (*VFSContext) BuildVfsPath

func (c *VFSContext) BuildVfsPath(p string) (Path, error)

func (*VFSContext) ReadFile

func (c *VFSContext) ReadFile(location string, options ...VFSOption) ([]byte, error)

ReadFile reads a file from a vfs URL It supports additional schemes which don't (yet) have full VFS implementations:

metadata: reads from instance metadata on GCE/AWS
http / https: reads from HTTP

func (*VFSContext) ResetMemfsContext added in v1.10.0

func (c *VFSContext) ResetMemfsContext(clusterReadable bool)

func (*VFSContext) WithGCSClient added in v1.26.0

func (v *VFSContext) WithGCSClient(gcsClient *storage.Service) *VFSContext

type VFSOption added in v1.15.1

type VFSOption func(options *vfsOptions)

func WithBackoff added in v1.15.1

func WithBackoff(backoff wait.Backoff) VFSOption

WithBackoff specifies a custom VFS backoff policy

type WriteOption added in v1.4.1

type WriteOption string
const (
	WriteOptionCreate       WriteOption = "Create"
	WriteOptionOnlyIfExists WriteOption = "IfExists"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL