gcs

package
v1.0.13 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 20, 2024 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Index

Constants

View Source 
const ContentEncodingGzip = "gzip"
View Source 
const MaxComponentCount = 1024

MaxComponentCount is the maximum number of components that a composite object may have. The sum of the component counts of the sources in a ComposeObjectsRequest must be no more than this value.

Cf. https://cloud.google.com/storage/docs/composite-objects#_Count

View Source 
const MaxSourcesPerComposeRequest = 32

MaxSourcesPerComposeRequest is the maximum number of sources that a ComposeObjectsRequest may contain.

Cf. https://cloud.google.com/storage/docs/composite-objects#_Compose

Variables

This section is empty.

Functions

This section is empty.

Types

type Bucket 

type Bucket interface {
	Name() string

	// Create a reader for the contents of a particular generation of an object.
	// On a nil error, the caller must arrange for the reader to be closed when
	// it is no longer needed.
	//
	// Non-existent objects cause either this method or the first read from the
	// resulting reader to return an error of type *NotFoundError.
	//
	// Official documentation:
	//     https://cloud.google.com/storage/docs/json_api/v1/objects/get
	NewReader(
		ctx context.Context,
		req *ReadObjectRequest) (io.ReadCloser, error)

	// Create or overwrite an object according to the supplied request. The new
	// object is guaranteed to exist immediately for the purposes of reading (and
	// eventually for listing) after this method returns a nil error. It is
	// guaranteed not to exist before req.Contents returns io.EOF.
	//
	// Official documentation:
	//     https://cloud.google.com/storage/docs/json_api/v1/objects/insert
	//     https://cloud.google.com/storage/docs/json_api/v1/how-tos/upload
	CreateObject(
		ctx context.Context,
		req *CreateObjectRequest) (*Object, error)

	// Copy an object to a new name, preserving all metadata. Any existing
	// generation of the destination name will be overwritten.
	//
	// Returns a record for the new object.
	//
	// Official documentation:
	//     https://cloud.google.com/storage/docs/json_api/v1/objects/copy
	CopyObject(
		ctx context.Context,
		req *CopyObjectRequest) (*Object, error)

	// Compose one or more source objects into a single destination object by
	// concatenating. Any existing generation of the destination name will be
	// overwritten.
	//
	// Returns a record for the new object.
	//
	// Official documentation:
	//     https://cloud.google.com/storage/docs/json_api/v1/objects/compose
	ComposeObjects(
		ctx context.Context,
		req *ComposeObjectsRequest) (*Object, error)

	// Return current information about the object with the given name.
	//
	// Official documentation:
	//     https://cloud.google.com/storage/docs/json_api/v1/objects/get
	StatObject(
		ctx context.Context,
		req *StatObjectRequest) (*MinObject, *ExtendedObjectAttributes, error)

	// List the objects in the bucket that meet the criteria defined by the
	// request, returning a result object that contains the results and
	// potentially a cursor for retrieving the next portion of the larger set of
	// results.
	//
	// Official documentation:
	//     https://cloud.google.com/storage/docs/json_api/v1/objects/list
	ListObjects(
		ctx context.Context,
		req *ListObjectsRequest) (*Listing, error)

	// Update the object specified by newAttrs.Name, patching using the non-zero
	// fields of newAttrs.
	//
	// Official documentation:
	//     https://cloud.google.com/storage/docs/json_api/v1/objects/patch
	UpdateObject(
		ctx context.Context,
		req *UpdateObjectRequest) (*Object, error)

	// Delete an object. Non-existence of the object is not treated as an error.
	//
	// Official documentation:
	//     https://cloud.google.com/storage/docs/json_api/v1/objects/delete
	DeleteObject(
		ctx context.Context,
		req *DeleteObjectRequest) error
}

Bucket represents a GCS bucket, pre-bound with a bucket name and necessary authorization information.

Each method that may block accepts a context object that is used for deadlines and cancellation. Users need not package authorization information into the context object (using cloud.WithContext or similar).

All methods are safe for concurrent access.

type ByteRange 

type ByteRange struct {
	Start uint64
	Limit uint64
}

ByteRange is a [start, limit) range of bytes within an object.

Its semantics are as follows:

  • If Limit is less than or equal to Start, the range is treated as empty.

  • The effective range is [start, limit) intersected with [0, L), where L is the length of the object.

    For example, a read for [L-1, L+10) returns the last byte of the object, and [L+2, L+10) is legal but returns nothing.

func (ByteRange) String 

func (br ByteRange) String() string

type ComposeObjectsRequest 

type ComposeObjectsRequest struct {
	// The name of the destination composite object.
	DstName string

	// If non-nil, the destination object will be created/overwritten only if the
	// current generation for its name is equal to the given value. Zero means
	// the object does not exist.
	DstGenerationPrecondition *int64

	// If non-nil, the destination object will be created/overwritten only if the
	// current meta-generation for its name is equal to the given value.
	//
	// This is only meaningful if DstGenerationPrecondition is also specified.
	DstMetaGenerationPrecondition *int64

	// The source objects from which to compose. This must be non-empty.
	//
	// Make sure to see the notes on MaxSourcesPerComposeRequest and
	// MaxComponentCount.
	Sources []ComposeSource

	// Optional information with which to create the object. See here for more
	// information:
	//
	//     https://cloud.google.com/storage/docs/json_api/v1/objects#resource
	//
	ContentType        string
	Metadata           map[string]string
	ContentLanguage    string
	ContentEncoding    string
	CacheControl       string
	ContentDisposition string
	CustomTime         string
	EventBasedHold     bool
	StorageClass       string
	Acl                []*storagev1.ObjectAccessControl
}

A request to compose one or more objects into a single composite object.

type ComposeSource 

type ComposeSource struct {
	// The name of the source object.
	Name string

	// The generation of the source object to compose from. Zero means the latest
	// generation.
	Generation int64
}

type CopyObjectRequest 

type CopyObjectRequest struct {
	SrcName string
	DstName string

	// The generation of the source object to copy, or zero for the latest
	// generation.
	SrcGeneration int64

	// If non-nil, the destination object will be created/overwritten only if the
	// current meta-generation for the source object is equal to the given value.
	//
	// This is probably only meaningful in conjunction with SrcGeneration.
	SrcMetaGenerationPrecondition *int64

	// Destination object will be overwritten only if the current
	// generation is equal to the given value. Zero means the object does not
	// exist.
	DstGenerationPrecondition *int64
}

A request to copy an object to a new name, preserving all metadata.

type CreateObjectRequest 

type CreateObjectRequest struct {
	// The name with which to create the object. This field must be set.
	//
	// Object names must:
	//
	// *  be non-empty.
	// *  be no longer than 1024 bytes.
	// *  be valid UTF-8.
	// *  not contain the code point U+000A (line feed).
	// *  not contain the code point U+000D (carriage return).
	//
	// See here for authoritative documentation:
	//     https://cloud.google.com/storage/docs/bucket-naming#objectnames
	Name string

	// Optional information with which to create the object. See here for more
	// information:
	//
	//     https://cloud.google.com/storage/docs/json_api/v1/objects#resource
	//
	ContentType        string
	ContentLanguage    string
	ContentEncoding    string
	CacheControl       string
	Metadata           map[string]string
	ContentDisposition string
	CustomTime         string
	EventBasedHold     bool
	StorageClass       string
	Acl                []*storagev1.ObjectAccessControl

	// A reader from which to obtain the contents of the object. Must be non-nil.
	Contents io.Reader

	// If non-nil, the object will not be created if the checksum of the received
	// contents does not match the supplied value.
	CRC32C *uint32

	// If non-nil, the object will not be created if the MD5 sum of the received
	// contents does not match the supplied value.
	MD5 *[md5.Size]byte

	// If non-nil, the object will be created/overwritten only if the current
	// generation for the object name is equal to the given value. Zero means the
	// object does not exist.
	GenerationPrecondition *int64

	// If non-nil, the object will be created/overwritten only if the current
	// meta-generation for the object name is equal to the given value. This is
	// only meaningful in conjunction with GenerationPrecondition.
	MetaGenerationPrecondition *int64
}

A request to create an object, accepted by Bucket.CreateObject.

type DeleteObjectRequest 

type DeleteObjectRequest struct {
	// The name of the object to delete. Must be specified.
	Name string

	// The generation of the object to delete. Zero means the latest generation.
	Generation int64

	// If non-nil, the request will fail without effect if there is an object
	// with the given name (and optionally generation), and its meta-generation
	// is not equal to this value.
	MetaGenerationPrecondition *int64
}

A request to delete an object by name. Non-existence is not treated as an error.

type ExtendedObjectAttributes 

type ExtendedObjectAttributes struct {
	ContentType        string
	ContentLanguage    string
	CacheControl       string
	Owner              string
	MD5                *[md5.Size]byte // Missing for composite objects
	CRC32C             *uint32         // Missing for CMEK buckets
	MediaLink          string
	StorageClass       string
	Deleted            time.Time
	ComponentCount     int64
	ContentDisposition string
	CustomTime         string
	EventBasedHold     bool
	Acl                []*storagev1.ObjectAccessControl
}

ExtendedObjectAttributes contains the missing attributes of Object which are not present in MinObject.

type ListObjectsRequest 

type ListObjectsRequest struct {
	// List only objects whose names begin with this prefix.
	Prefix string

	// Collapse results based on a delimiter.
	//
	// If non-empty, enable the following behavior. For each run of one or more
	// objects whose names are of the form:
	//
	//     <Prefix><S><Delimiter><...>
	//
	// where <S> is a string that doesn't itself contain Delimiter and <...> is
	// anything, return a single Collaped entry in the listing consisting of
	//
	//     <Prefix><S><Delimiter>
	//
	// instead of one Object record per object. If a collapsed entry consists of
	// a large number of objects, this may be more efficient.
	Delimiter string

	// Only applicable when Delimiter is set nonempty. Default to false.
	//
	// The objects in the result listing would contain those objects that end
	// with the first delimiter if this is true. Otherwise, those objects are
	// not included.
	//
	// Example:
	//  Assume there is an object "foo/bar/".
	//  1. Prefix: "foo/", Delimiter: "/", IncludeTrailingDelimiter: true
	//     -> "foo/bar/" exists in both listing.CollapsedRuns and
	//     listing.Objects.
	//  2. Prefix: "foo/", Delimiter: "/", IncludeTrailingDelimiter: false
	//     -> "foo/bar/" exists in only listing.CollapsedRuns but not
	//     listing.Objects.
	IncludeTrailingDelimiter bool

	// IncludeFoldersAsPrefixes includes Folders and Managed Folders in the set of
	// prefixes returned by the query.
	IncludeFoldersAsPrefixes bool

	// Used to continue a listing where a previous one left off. See
	// Listing.ContinuationToken for more information.
	ContinuationToken string

	// The maximum number of objects and collapsed runs to return. Fewer than
	// this number may actually be returned. If this is zero, a sensible default
	// is used.
	MaxResults int

	// Set of properties to return. Acceptable values- full & noAcl.
	//    1. full  - returns all properties
	//    2. noAcl - omit owner, acl properties
	//
	// Currently projection value is hardcoded to full. To keep it aligned with
	// the current flow, default value will be full and callers can override it
	// using this param.
	ProjectionVal Projection
}

type Listing 

type Listing struct {
	// Records for objects matching the listing criteria.
	//
	// Guaranteed to be strictly increasing under a lexicographical comparison on
	// (name, generation) pairs.
	Objects []*Object

	// Collapsed entries for runs of names sharing a prefix followed by a
	// delimiter. See notes on ListObjectsRequest.Delimiter.
	//
	// Guaranteed to be strictly increasing.
	CollapsedRuns []string

	// A continuation token, for fetching more results.
	//
	// If non-empty, this listing does not represent the full set of matching
	// objects in the bucket. Call ListObjects again with the request's
	// ContinuationToken field set to this value to continue where you left off.
	//
	// Guarantees, for replies R1 and R2, with R2 continuing from R1:
	//
	//  *  All of R1's object names are strictly less than all object names and
	//     collapsed runs in R2.
	//
	//  *  All of R1's collapsed runs are strictly less than all object names and
	//     prefixes in R2.
	//
	// (Cf. Google-internal bug 19286144)
	//
	// Note that there is no guarantee of atomicity of listings. Objects written
	// and deleted concurrently with a single or multiple listing requests may or
	// may not be returned.
	ContinuationToken string
}

Listing contains a set of objects and delimter-based collapsed runs returned by a call to ListObjects. See also ListObjectsRequest.

type MinObject 

type MinObject struct {
	Name            string
	Size            uint64
	Generation      int64
	MetaGeneration  int64
	Updated         time.Time
	Metadata        map[string]string
	ContentEncoding string
}

MinObject is a record representing subset of properties of a particular generation object in GCS.

See here for more information about its fields: 

https://cloud.google.com/storage/docs/json_api/v1/objects#resource

func (MinObject) HasContentEncodingGzip 

func (mo MinObject) HasContentEncodingGzip() bool

type NotFoundError 

type NotFoundError struct {
	Err error
}

A *NotFoundError value is an error that indicates an object name or a particular generation for that name were not found.

func (*NotFoundError) Error 

func (nfe *NotFoundError) Error() string

type Object 

type Object struct {
	Name            string
	ContentType     string
	ContentLanguage string
	CacheControl    string
	Owner           string
	Size            uint64
	ContentEncoding string
	MD5             *[md5.Size]byte // Missing for composite objects
	CRC32C          *uint32         //Missing for CMEK buckets
	MediaLink       string
	Metadata        map[string]string
	Generation      int64
	MetaGeneration  int64
	StorageClass    string
	Deleted         time.Time
	Updated         time.Time

	// As of 2015-06-03, the official GCS documentation for this
	// property (https://goo.gl/GwD5Dq) says this:
	//
	//     Newly uploaded objects have a component count of 1, and composing a
	//     sequence of objects creates an object whose component count is equal
	//     to the sum of component counts in the sequence.
	//
	// However, in Google-internal bug 21572928 it was clarified that this
	// doesn't match the actual implementation, which can be documented as:
	//
	//     Newly uploaded objects do not have a component count. Composing a
	//     sequence of objects creates an object whose component count is equal
	//     to the sum of the component counts of the objects in the sequence,
	//     where objects that do not have a component count are treated as having
	//     a component count of 1.
	//
	// This is a much less elegant and convenient rule, so this package emulates
	// the officially documented behavior above. That is, it synthesizes a
	// component count of 1 for objects that do not have a component count.
	ComponentCount int64

	ContentDisposition string
	CustomTime         string
	EventBasedHold     bool
	Acl                []*storagev1.ObjectAccessControl
}

Object is a record representing a particular generation of a particular object name in GCS.

See here for more information about its fields: 

https://cloud.google.com/storage/docs/json_api/v1/objects#resource

type PreconditionError 

type PreconditionError struct {
	Err error
}

A *PreconditionError value is an error that indicates a precondition failed.

func (*PreconditionError) Error 

func (pe *PreconditionError) Error() string

Returns pe.Err.Error().

type Projection 

type Projection int64
const (
	Full Projection = iota
	NoAcl
)

func (Projection) String 

func (p Projection) String() string

Returning the string values based on the values accepted by projection param. https://cloud.google.com/storage/docs/json_api/v1/objects/list#parameters

type ReadObjectRequest 

type ReadObjectRequest struct {
	// The name of the object to read.
	Name string

	// The generation of the object to read. Zero means the latest generation.
	Generation int64

	// If present, limit the contents returned to a range within the object.
	Range *ByteRange

	// If present, read the contents of the GCS object as it is on GCS.
	// This might not be honoured by all the implementations.
	ReadCompressed bool
}

A request to read the contents of an object at a particular generation.

type StatObjectRequest 

type StatObjectRequest struct {
	// The name of the object in question.
	Name string

	// Relevant only when fast_stat_bucket is used. This field controls whether
	// to fetch from gcs or from cache.
	ForceFetchFromGcs bool

	// Controls whether StatObject response includes GCS ExtendedObjectAttributes.
	ReturnExtendedObjectAttributes bool
}

type UpdateObjectRequest 

type UpdateObjectRequest struct {
	// The name of the object to update. Must be specified.
	Name string

	// The generation of the object to update. Zero means the latest generation.
	Generation int64

	// If non-nil, the request will fail without effect if there is an object
	// with the given name (and optionally generation), and its meta-generation
	// is not equal to this value.
	MetaGenerationPrecondition *int64

	// String fields in the object to update (or not). The semantics are as
	// follows, for a given field F:
	//
	//  *  If F is set to nil, the corresponding GCS object field is untouched.
	//
	//  *  If *F is the empty string, then the corresponding GCS object field is
	//     removed.
	//
	//  *  Otherwise, the corresponding GCS object field is set to *F.
	//
	//  *  There is no facility for setting a GCS object field to the empty
	//     string, since many of the fields do not actually allow that as a legal
	//     value.
	//
	// Note that the GCS object's content type field cannot be removed.
	ContentType     *string
	ContentEncoding *string
	ContentLanguage *string
	CacheControl    *string

	// User-provided metadata updates. Keys that are not mentioned are untouched.
	// Keys whose values are nil are deleted, and others are updated to the
	// supplied string. There is no facility for completely removing user
	// metadata.
	Metadata map[string]*string
}

A request to update the metadata of an object, accepted by Bucket.UpdateObject.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.