gcs

Documentation

Overview

Package gcs handles uploading files and raw data to GCS and determines where in the GCS bucket data should go given a specific job specification

Index

• func AliasForSpec(spec *downwardapi.JobSpec) string
• func LatestBuildForSpec(spec *downwardapi.JobSpec, pathSegment RepoPathBuilder) []string
• func PathForSpec(spec *downwardapi.JobSpec, pathSegment RepoPathBuilder) string
• func RootForSpec(spec *downwardapi.JobSpec) string
• func Upload(bucket *storage.BucketHandle, uploadTargets map[string]UploadFunc) error
• type Finished
• type RepoPathBuilder
  • func NewExplicitRepoPathBuilder() RepoPathBuilder
  • func NewLegacyRepoPathBuilder(defaultOrg, defaultRepo string) RepoPathBuilder
  • func NewSingleDefaultRepoPathBuilder(defaultOrg, defaultRepo string) RepoPathBuilder
• type Started
• type UploadFunc
  • func DataUpload(src io.Reader) UploadFunc
  • func DataUploadWithMetadata(src io.Reader, metadata map[string]string) UploadFunc
  • func FileUpload(file string) UploadFunc

Functions

func AliasForSpec

func AliasForSpec(spec *downwardapi.JobSpec) string

AliasForSpec determines the GCS path aliases for a job spec

func LatestBuildForSpec

func LatestBuildForSpec(spec *downwardapi.JobSpec, pathSegment RepoPathBuilder) []string

LatestBuildForSpec determines the GCS path for storing the latest build id for a job. pathSegment can be nil so callers of this helper are not required to choose a path strategy but can still get back a result.

func PathForSpec

func PathForSpec(spec *downwardapi.JobSpec, pathSegment RepoPathBuilder) string

PathForSpec determines the GCS path prefix for files uploaded for a specific job spec

func RootForSpec

func RootForSpec(spec *downwardapi.JobSpec) string

RootForSpec determines the root GCS path for storing artifacts about the provided job.

func Upload

func Upload(bucket *storage.BucketHandle, uploadTargets map[string]UploadFunc) error

Upload uploads all of the data in the uploadTargets map to GCS in parallel. The map is keyed on GCS path under the bucket

Types

type Finished

type Finished struct {
	// Timestamp is the time in epoch seconds recorded when finished.json is created.
	Timestamp int64 `json:"timestamp"`
	// Version is deprecated.
	// TODO: Version should be removed in favor of Revision.
	Version string `json:"version"`
	// JobVersion is deprecated.
	// TODO: JobVersion should be removed in favor of Revision.
	JobVersion string `json:"job-version"`
	// Revision identifies the revision of the code the build tested.
	// Revision can be either a SHA or a ref.
	// TODO: resolve http://github.com/kubernetes/test-infra/issues/10359
	Revision string `json:"revision,omitempty"`
	// Passed is true when the job has passed, else false.
	Passed bool `json:"passed"`
	// Result is the result of the job ("SUCCESS", "ABORTED", or "FAILURE").
	Result string `json:"result"`
	// Metadata is any additional metadata uploaded by the job.
	// Keys typically include infra-commit, repos, repo, repo-commit.
	// Values typically have type string or map[string]string.
	Metadata map[string]interface{} `json:"metadata,omitempty"`
}

Finished is used to mirror the finished.json artifact.

type RepoPathBuilder

type RepoPathBuilder func(org, repo string) string

RepoPathBuilder builds GCS path segments and embeds defaulting behavior

func NewExplicitRepoPathBuilder

func NewExplicitRepoPathBuilder() RepoPathBuilder

NewExplicitRepoPathBuilder returns a builder that handles the path encoding where a path will always have an explicit "org_repo" path segment

func NewLegacyRepoPathBuilder

func NewLegacyRepoPathBuilder(defaultOrg, defaultRepo string) RepoPathBuilder

NewLegacyRepoPathBuilder returns a builder that handles the legacy path encoding where a path will only contain an org or repo if they are non-default

func NewSingleDefaultRepoPathBuilder

func NewSingleDefaultRepoPathBuilder(defaultOrg, defaultRepo string) RepoPathBuilder

NewSingleDefaultRepoPathBuilder returns a builder that handles the legacy path encoding where a path will contain org and repo for all but one default repo

type Started

type Started struct {
	// Timestamp is the time in epoch seconds recorded when started.json is created.
	Timestamp int64 `json:"timestamp"`
	// Node is deprecated.
	Node string `json:"node"`
	// Repos is deprecated.
	Repos map[string]string `json:"repos"`
	// Pull is deprecated.
	Pull string `json:"pull"`
}

Started is used to mirror the started.json artifact. All fields in Started beside Timestamp are deprecated.

type UploadFunc

type UploadFunc func(obj *storage.ObjectHandle) error

UploadFunc knows how to upload into an object

func DataUpload

func DataUpload(src io.Reader) UploadFunc

DataUpload returns an UploadFunc which copies all data from src reader into GCS

func DataUploadWithMetadata

func DataUploadWithMetadata(src io.Reader, metadata map[string]string) UploadFunc

DataUploadWithMetadata returns an UploadFunc which copies all data from src reader into GCS and also sets the provided metadata fields onto the object.

func FileUpload

func FileUpload(file string) UploadFunc

FileUpload returns an UploadFunc which copies all data from the file on disk to the GCS object