gcsproxy

Documentation

Overview

Package gcsproxy contains artifact staging and retrieval servers backed by GCS.

Index

• func ReadProxyManifest(ctx context.Context, object string) (*jobpb.ProxyManifest, error)
• type RetrievalServer
  • func NewRetrievalServer(md *jobpb.ProxyManifest) (*RetrievalServer, error)
  • func (s *RetrievalServer) GetArtifact(req *jobpb.LegacyGetArtifactRequest, ...) error
  • func (s *RetrievalServer) GetManifest(ctx context.Context, req *jobpb.GetManifestRequest) (*jobpb.GetManifestResponse, error)
• type StagingServer
  • func NewStagingServer(manifest string) (*StagingServer, error)
  • func (s *StagingServer) CommitManifest(ctx context.Context, req *jobpb.CommitManifestRequest) (*jobpb.CommitManifestResponse, error)
  • func (s *StagingServer) PutArtifact(ps jobpb.LegacyArtifactStagingService_PutArtifactServer) error

Functions

func ReadProxyManifest

func ReadProxyManifest(ctx context.Context, object string) (*jobpb.ProxyManifest, error)

ReadProxyManifest reads and parses the proxy manifest from GCS.

Types

type RetrievalServer

type RetrievalServer struct {
	// contains filtered or unexported fields
}

RetrievalServer is a artifact retrieval server backed by Google Cloud Storage (GCS). It serves a single manifest and ignores the worker id. The server performs no caching or pre-fetching.

func NewRetrievalServer

func NewRetrievalServer(md *jobpb.ProxyManifest) (*RetrievalServer, error)

NewRetrievalServer creates a artifact retrieval server for the given manifest. It requires that the locations are in GCS.

func (*RetrievalServer) GetArtifact

func (s *RetrievalServer) GetArtifact(req *jobpb.LegacyGetArtifactRequest, stream jobpb.LegacyArtifactRetrievalService_GetArtifactServer) error

GetArtifact returns a given artifact.

func (*RetrievalServer) GetManifest

func (s *RetrievalServer) GetManifest(ctx context.Context, req *jobpb.GetManifestRequest) (*jobpb.GetManifestResponse, error)

GetManifest returns the manifest for all artifacts.

type StagingServer

type StagingServer struct {
	// contains filtered or unexported fields
}

StagingServer is a artifact staging server backed by Google Cloud Storage (GCS). It commits a single manifest and ignores the staging id.

func NewStagingServer

func NewStagingServer(manifest string) (*StagingServer, error)

NewStagingServer creates a artifact staging server for the given manifest. It requires that the manifest is in GCS and will stage the supplied artifacts next to it.

func (*StagingServer) CommitManifest

func (s *StagingServer) CommitManifest(ctx context.Context, req *jobpb.CommitManifestRequest) (*jobpb.CommitManifestResponse, error)

CommitManifest commits the given artifact manifest to GCS.

func (*StagingServer) PutArtifact

func (s *StagingServer) PutArtifact(ps jobpb.LegacyArtifactStagingService_PutArtifactServer) error

PutArtifact stores the given artifact in GCS.