storage

package
v1.7.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 28, 2020 License: MPL-2.0 Imports: 16 Imported by: 52

Documentation

Index

Constants

View Source 
const (
	JobStateDone    = "done"
	JobStateRunning = "running"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type AbortMpuInput 

type AbortMpuInput struct {
	PartsDirectoryPath string
}

AbortMpuInput represents parameters to an AbortMpu operation

type AddJobInputsInput 

type AddJobInputsInput struct {
	JobID       string
	ObjectPaths []string
}

AddJobInputs represents parameters to a AddJobInputs operation.

type CancelJobInput 

type CancelJobInput struct {
	JobID string
}

CancelJobInput represents parameters to a CancelJob operation.

type CommitMpuBody 

type CommitMpuBody struct {
	Parts []string `json:"parts"`
}

CommitMpuBody represents the body of a CommitMpu request

type CommitMpuInput 

type CommitMpuInput struct {
	Id      string
	Headers map[string]string
	Body    CommitMpuBody
}

CommitMpuInput represents parameters to a CommitMpu operation

type CreateJobInput 

type CreateJobInput struct {
	Name   string      `json:"name"`
	Phases []*JobPhase `json:"phases"`
}

CreateJobInput represents parameters to a CreateJob operation.

type CreateJobOutput 

type CreateJobOutput struct {
	JobID string
}

CreateJobOutput contains the outputs of a CreateJob operation.

type CreateMpuBody 

type CreateMpuBody struct {
	ObjectPath string            `json:"objectPath"`
	Headers    map[string]string `json:"headers,omitempty"`
}

CreateMpuBody represents the body of a CreateMpu request.

type CreateMpuInput 

type CreateMpuInput struct {
	Body            CreateMpuBody
	ContentLength   uint64
	ContentMD5      string
	DurabilityLevel uint64
	ForceInsert     bool //Force the creation of the directory tree
}

CreateMpuInput represents parameters to a CreateMpu operation.

type CreateMpuOutput 

type CreateMpuOutput struct {
	Id             string `json:"id"`
	PartsDirectory string `json:"partsDirectory"`
}

CreateMpuOutput represents the response from a CreateMpu operation

type DeleteDirectoryInput 

type DeleteDirectoryInput struct {
	DirectoryName string
	ForceDelete   bool //Will recursively delete all child directories and objects
}

DeleteDirectoryInput represents parameters to a Delete operation.

type DeleteObjectInput 

type DeleteObjectInput struct {
	ObjectPath string
	Headers    map[string]string
}

DeleteObjectInput represents parameters to a DeleteObject operation.

type DirectoryClient 

type DirectoryClient struct {
	// contains filtered or unexported fields
}

func (*DirectoryClient) Delete 

func (s *DirectoryClient) Delete(ctx context.Context, input *DeleteDirectoryInput) error

Delete deletes a directory on the Triton Object Storage. The directory must be empty.

func (*DirectoryClient) List 

func (s *DirectoryClient) List(ctx context.Context, input *ListDirectoryInput) (*ListDirectoryOutput, error)

List lists the contents of a directory on the Triton Object Store service.

func (*DirectoryClient) Put 

func (s *DirectoryClient) Put(ctx context.Context, input *PutDirectoryInput) error

Put puts a director into the Triton Object Storage service is an idempotent create-or-update operation. Your private namespace starts at /:login, and you can create any nested set of directories or objects within it.

type DirectoryEntry 

type DirectoryEntry struct {
	ETag         string    `json:"etag"`
	ModifiedTime time.Time `json:"mtime"`
	Name         string    `json:"name"`
	Size         uint64    `json:"size"`
	Type         string    `json:"type"`
}

DirectoryEntry represents an object or directory in Manta.

type EndJobInputInput 

type EndJobInputInput struct {
	JobID string
}

EndJobInputInput represents parameters to a EndJobInput operation.

type GetInfoInput 

type GetInfoInput struct {
	ObjectPath string
	Headers    map[string]string
}

GetObjectInput represents parameters to a GetObject operation.

type GetInfoOutput 

type GetInfoOutput struct {
	ContentLength uint64
	ContentType   string
	LastModified  time.Time
	ContentMD5    string
	ETag          string
	Metadata      map[string]string
}

GetObjectOutput contains the outputs for a GetObject operation. It is your responsibility to ensure that the io.ReadCloser ObjectReader is closed.

type GetJobFailuresInput 

type GetJobFailuresInput struct {
	JobID string
}

GetJobFailuresInput represents parameters to a GetJobFailures operation.

type GetJobFailuresOutput 

type GetJobFailuresOutput struct {
	ResultSetSize uint64
	Items         io.ReadCloser
}

GetJobFailuresOutput contains the outputs for a GetJobFailures operation. It is your responsibility to ensure that the io.ReadCloser Items is closed.

type GetJobInput 

type GetJobInput struct {
	JobID string
}

GetJobInput represents parameters to a GetJob operation.

type GetJobInputInput 

type GetJobInputInput struct {
	JobID string
}

GetJobInputInput represents parameters to a GetJobOutput operation.

type GetJobInputOutput 

type GetJobInputOutput struct {
	ResultSetSize uint64
	Items         io.ReadCloser
}

GetJobInputOutput contains the outputs for a GetJobOutput operation. It is your responsibility to ensure that the io.ReadCloser Items is closed.

type GetJobOutput 

type GetJobOutput struct {
	Job *Job
}

GetJobOutput contains the outputs of a GetJob operation.

type GetJobOutputInput 

type GetJobOutputInput struct {
	JobID string
}

GetJobOutputInput represents parameters to a GetJobOutput operation.

type GetJobOutputOutput 

type GetJobOutputOutput struct {
	ResultSetSize uint64
	Items         io.ReadCloser
}

GetJobOutputOutput contains the outputs for a GetJobOutput operation. It is your responsibility to ensure that the io.ReadCloser Items is closed.

type GetMpuHeaders 

type GetMpuHeaders struct {
	ContentLength int64  `json:"content-length"`
	ContentMd5    string `json:"content-md5"`
}

type GetMpuInput 

type GetMpuInput struct {
	PartsDirectoryPath string
}

GetMpuInput represents parameters to a GetMpu operation

type GetMpuOutput 

type GetMpuOutput struct {
	Id             string        `json:"id"`
	State          string        `json:"state"`
	PartsDirectory string        `json:"partsDirectory"`
	TargetObject   string        `json:"targetObject"`
	Headers        GetMpuHeaders `json:"headers"`
	NumCopies      int64         `json:"numCopies"`
	CreationTimeMs int64         `json:"creationTimeMs"`
}

type GetObjectInput 

type GetObjectInput struct {
	ObjectPath string
	Headers    map[string]string
}

GetObjectInput represents parameters to a GetObject operation.

type GetObjectOutput 

type GetObjectOutput struct {
	ContentLength uint64
	ContentType   string
	LastModified  time.Time
	ContentMD5    string
	ETag          string
	Metadata      map[string]string
	ObjectReader  io.ReadCloser
}

GetObjectOutput contains the outputs for a GetObject operation. It is your responsibility to ensure that the io.ReadCloser ObjectReader is closed.

type Job 

type Job struct {
	ID          string      `json:"id"`
	Name        string      `json:"name"`
	Phases      []*JobPhase `json:"phases"`
	State       string      `json:"state"`
	Cancelled   bool        `json:"cancelled"`
	InputDone   bool        `json:"inputDone"`
	CreatedTime time.Time   `json:"timeCreated"`
	DoneTime    time.Time   `json:"timeDone"`
	Transient   bool        `json:"transient"`
	Stats       *JobStats   `json:"stats"`
}

Job represents a compute job in Manta.

type JobClient 

type JobClient struct {
	// contains filtered or unexported fields
}

func (*JobClient) AddInputs 

func (s *JobClient) AddInputs(ctx context.Context, input *AddJobInputsInput) error

AddJobInputs submits inputs to an already created job.

func (*JobClient) Cancel 

func (s *JobClient) Cancel(ctx context.Context, input *CancelJobInput) error

CancelJob cancels a job from doing any further work. Cancellation is asynchronous and "best effort"; there is no guarantee the job will actually stop. For example, short jobs where input is already closed will likely still run to completion.

This is however useful when:

  • input is still open
  • you have a long-running job

func (*JobClient) Create 

func (s *JobClient) Create(ctx context.Context, input *CreateJobInput) (*CreateJobOutput, error)

CreateJob submits a new job to be executed. This call is not idempotent, so calling it twice will create two jobs.

func (*JobClient) EndInput 

func (s *JobClient) EndInput(ctx context.Context, input *EndJobInputInput) error

EndJobInput submits inputs to an already created job.

func (*JobClient) Get 

func (s *JobClient) Get(ctx context.Context, input *GetJobInput) (*GetJobOutput, error)

GetJob returns the list of jobs you currently have.

func (*JobClient) GetFailures 

func (s *JobClient) GetFailures(ctx context.Context, input *GetJobFailuresInput) (*GetJobFailuresOutput, error)

GetJobFailures returns the current "live" set of outputs from a job. Think of this like `tail -f`. If error is nil (i.e. the operation is successful), it is your responsibility to close the io.ReadCloser named Items in the output.

func (*JobClient) GetInput 

func (s *JobClient) GetInput(ctx context.Context, input *GetJobInputInput) (*GetJobInputOutput, error)

GetJobInput returns the current "live" set of inputs from a job. Think of this like `tail -f`. If error is nil (i.e. the operation is successful), it is your responsibility to close the io.ReadCloser named Items in the output.

func (*JobClient) GetOutput 

func (s *JobClient) GetOutput(ctx context.Context, input *GetJobOutputInput) (*GetJobOutputOutput, error)

GetJobOutput returns the current "live" set of outputs from a job. Think of this like `tail -f`. If error is nil (i.e. the operation is successful), it is your responsibility to close the io.ReadCloser named Items in the output.

func (*JobClient) List 

func (s *JobClient) List(ctx context.Context, input *ListJobsInput) (*ListJobsOutput, error)

ListJobs returns the list of jobs you currently have.

type JobPhase 

type JobPhase struct {
	// Type is the type of phase. Must be `map` or `reduce`.
	Type string `json:"type,omitempty"`

	// Assets is an array of objects to be placed in your compute zones.
	Assets []string `json:"assets,omitempty"`

	// Exec is the shell statement to execute. It may be any valid shell
	// command, including pipelines and other shell syntax. You can also
	// execute programs stored in the service by including them in "assets"
	// and referencing them as /assets/$manta_path.
	Exec string `json:"exec"`

	// Init is a shell statement to execute in each compute zone before
	// any tasks are executed. The same constraints apply as to Exec.
	Init string `json:"init"`

	// ReducerCount is an optional number of reducers for this phase. The
	// default value if not specified is 1. The maximum value is 1024.
	ReducerCount uint `json:"count,omitempty"`

	// Memory is the amount of DRAM in MB to be allocated to the compute
	// zone. Valid values are 256, 512, 1024, 2048, 4096 or 8192.
	Memory uint64 `json:"memory,omitempty"`

	// Disk is the amount of disk space in GB to be allocated to the compute
	// zone. Valid values are 2, 4, 8, 16, 32, 64, 128, 256, 512 or 1024.
	Disk uint64 `json:"disk,omitempty"`
}

JobPhase represents the specification for a map or reduce phase of a Manta job.

type JobStats 

type JobStats struct {
	Errors    uint64 `json:"errors"`
	Outputs   uint64 `json:"outputs"`
	Retries   uint64 `json:"retries"`
	Tasks     uint64 `json:"tasks"`
	TasksDone uint64 `json:"tasksDone"`
}

JobStats represents statistics for a compute job in Manta.

type JobSummary 

type JobSummary struct {
	ModifiedTime time.Time `json:"mtime"`
	ID           string    `json:"name"`
}

JobSummary represents the summary of a compute job in Manta.

type ListDirectoryInput 

type ListDirectoryInput struct {
	DirectoryName string
	Limit         uint64
	Marker        string
}

ListDirectoryInput represents parameters to a List operation.

type ListDirectoryOutput 

type ListDirectoryOutput struct {
	Entries       []*DirectoryEntry
	ResultSetSize uint64
}

ListDirectoryOutput contains the outputs of a List operation.

type ListJobsInput 

type ListJobsInput struct {
	RunningOnly bool
	Limit       uint64
	Marker      string
}

ListJobsInput represents parameters to a ListJobs operation.

type ListJobsOutput 

type ListJobsOutput struct {
	Jobs          []*JobSummary
	ResultSetSize uint64
}

ListJobsOutput contains the outputs of a ListJobs operation.

type ListMpuPart 

type ListMpuPart struct {
	ETag       string
	PartNumber int
	Size       int64
}

type ListMpuPartsInput 

type ListMpuPartsInput struct {
	Id string
}

type ListMpuPartsOutput 

type ListMpuPartsOutput struct {
	Parts []ListMpuPart
}

type ObjectsClient 

type ObjectsClient struct {
	// contains filtered or unexported fields
}

func (*ObjectsClient) AbortMultipartUpload 

func (s *ObjectsClient) AbortMultipartUpload(ctx context.Context, input *AbortMpuInput) error

func (*ObjectsClient) CommitMultipartUpload 

func (s *ObjectsClient) CommitMultipartUpload(ctx context.Context, input *CommitMpuInput) error

func (*ObjectsClient) CreateMultipartUpload 

func (s *ObjectsClient) CreateMultipartUpload(ctx context.Context, input *CreateMpuInput) (*CreateMpuOutput, error)

func (*ObjectsClient) Delete 

func (s *ObjectsClient) Delete(ctx context.Context, input *DeleteObjectInput) error

DeleteObject deletes an object.

func (*ObjectsClient) Get 

func (s *ObjectsClient) Get(ctx context.Context, input *GetObjectInput) (*GetObjectOutput, error)

Get retrieves an object from the Manta service. If error is nil (i.e. the call returns successfully), it is your responsibility to close the io.ReadCloser named ObjectReader in the operation output.

func (*ObjectsClient) GetInfo 

func (s *ObjectsClient) GetInfo(ctx context.Context, input *GetInfoInput) (*GetInfoOutput, error)

GetInfo sends a HEAD request to an object in the Manta service. This function does not return a response body.

func (*ObjectsClient) GetMultipartUpload 

func (s *ObjectsClient) GetMultipartUpload(ctx context.Context, input *GetMpuInput) (*GetMpuOutput, error)

func (*ObjectsClient) IsDir 

func (s *ObjectsClient) IsDir(ctx context.Context, objectPath string) (bool, error)

IsDir is a convenience wrapper around the GetInfo function which takes an ObjectPath and returns a boolean whether or not the object is a directory type in Manta. Returns an error if GetInfo failed upstream for some reason.

func (*ObjectsClient) ListMultipartUploadParts 

func (s *ObjectsClient) ListMultipartUploadParts(ctx context.Context, input *ListMpuPartsInput) (*ListMpuPartsOutput, error)

func (*ObjectsClient) Put 

func (s *ObjectsClient) Put(ctx context.Context, input *PutObjectInput) error

func (*ObjectsClient) PutMetadata 

func (s *ObjectsClient) PutMetadata(ctx context.Context, input *PutObjectMetadataInput) error

PutObjectMetadata allows you to overwrite the HTTP headers for an already existing object, without changing the data. Note this is an idempotent "replace" operation, so you must specify the complete set of HTTP headers you want stored on each request.

You cannot change "critical" headers:

  • Content-Length
  • Content-MD5
  • Durability-Level

func (*ObjectsClient) UploadPart 

func (s *ObjectsClient) UploadPart(ctx context.Context, input *UploadPartInput) (*UploadPartOutput, error)

type PutDirectoryInput 

type PutDirectoryInput struct {
	DirectoryName string
}

PutDirectoryInput represents parameters to a Put operation.

type PutObjectInput 

type PutObjectInput struct {
	ObjectPath       string
	DurabilityLevel  uint64
	ContentType      string
	ContentMD5       string
	IfMatch          string
	IfModifiedSince  *time.Time
	ContentLength    uint64
	MaxContentLength uint64
	ObjectReader     io.Reader
	Headers          map[string]string
	ForceInsert      bool //Force the creation of the directory tree
}

PutObjectInput represents parameters to a PutObject operation.

type PutObjectMetadataInput 

type PutObjectMetadataInput struct {
	ObjectPath  string
	ContentType string
	Metadata    map[string]string
}

PutObjectMetadataInput represents parameters to a PutObjectMetadata operation.

type PutSnapLinkInput 

type PutSnapLinkInput struct {
	LinkPath   string
	SourcePath string
}

PutSnapLinkInput represents parameters to a PutSnapLink operation.

type SignURLInput 

type SignURLInput struct {
	ValidityPeriod time.Duration
	Method         string
	ObjectPath     string
}

SignURLInput represents parameters to a SignURL operation.

type SignURLOutput 

type SignURLOutput struct {
	Method    string
	Algorithm string
	Signature string
	Expires   string
	KeyID     string
	// contains filtered or unexported fields
}

SignURLOutput contains the outputs of a SignURL operation. To simply access the signed URL, use the SignedURL method.

func (*SignURLOutput) SignedURL 

func (output *SignURLOutput) SignedURL(scheme string) string

SignedURL returns a signed URL for the given scheme. Valid schemes are `http` and `https`.

type SnapLinksClient 

type SnapLinksClient struct {
	// contains filtered or unexported fields
}

func (*SnapLinksClient) Put 

func (s *SnapLinksClient) Put(ctx context.Context, input *PutSnapLinkInput) error

PutSnapLink creates a SnapLink to an object.

type StorageClient 

type StorageClient struct {
	Client *client.Client
}

func NewClient 

func NewClient(config *triton.ClientConfig) (*StorageClient, error)

NewClient returns a new client for working with Storage endpoints and resources within CloudAPI

func (*StorageClient) Dir 

func (c *StorageClient) Dir() *DirectoryClient

Dir returns a DirectoryClient used for accessing functions pertaining to Directories functionality of the Manta API.

func (*StorageClient) Jobs 

func (c *StorageClient) Jobs() *JobClient

Jobs returns a JobClient used for accessing functions pertaining to Jobs functionality of the Triton Object Storage API.

func (*StorageClient) Objects 

func (c *StorageClient) Objects() *ObjectsClient

Objects returns an ObjectsClient used for accessing functions pertaining to Objects functionality of the Triton Object Storage API.

func (*StorageClient) SetHeader 

func (c *StorageClient) SetHeader(header *http.Header)

SetHeader allows a consumer of the current client to set a custom header for the next backend HTTP request sent to CloudAPI.

func (*StorageClient) SignURL 

func (s *StorageClient) SignURL(input *SignURLInput) (*SignURLOutput, error)

SignURL creates a time-expiring URL that can be shared with others. This is useful to generate HTML links, for example. 

func (c *StorageClient) SnapLinks() *SnapLinksClient

SnapLinks returns an SnapLinksClient used for accessing functions pertaining to SnapLinks functionality of the Triton Object Storage API.

type UploadPartInput 

type UploadPartInput struct {
	Id           string
	PartNum      uint64
	ContentMD5   string
	Headers      map[string]string
	ObjectReader io.Reader
}

UploadPartInput represents parameters to a UploadPart operation.

type UploadPartOutput 

type UploadPartOutput struct {
	Part string `json:"part"`
}

UploadPartOutput represents the response from a

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.