bucketupload

README

bucketupload

Uploads a local file to a configured gocloud.dev/blob bucket. Allows setting the object key with the Key parameter, using the filename if not set. The BufferSize parameter changes the default size in bytes of the chuncks that will be uploaded in a single request. If BufferSize is 0, the driver will chose a reasonable default. Buffersize may be ignored by some drivers. For example, MinIO supports a maximum of 10,000 chuncks per upload, which may require increasing the buffer size to upload big files.

This activity will heartbeat each one-third of the configured timeout, if set in the activity options.

Registration

The Name constant is used as example, use any name to register and execute the activity that meets your needs. See the Go CDK guide and go.artefactual.dev/tools/bucket for options to open a bucket. An example registration:

import (
	"go.artefactual.dev/tools/bucket"
	"go.temporal.io/sdk/activity"
	"go.temporal.io/sdk/worker"

	"github.com/artefactual-sdps/temporal-activities/bucketupload"
)

tw := worker.New(...)

b, err := bucket.NewWithConfig(ctx, &bucket.Config{URL: "<driver-url>"})
if err != nil {
    // Handle error.
}
defer b.Close()

tw.RegisterActivityWithOptions(
    bucketupload.New(b).Execute,
    activity.RegisterOptions{Name: bucketupload.Name},
)

Execution

An example execution:

import (
    "time"

    "go.temporal.io/sdk/temporal"
    "go.temporal.io/sdk/workflow"

    "github.com/artefactual-sdps/temporal-activities/bucketupload"
)

opts := workflow.WithActivityOptions(ctx, workflow.ActivityOptions{
    StartToCloseTimeout: time.Hour * 2,
    HeartbeatTimeout:    time.Second * 5,
    RetryPolicy: &temporal.RetryPolicy{
        MaximumAttempts: 3,
        NonRetryableErrorTypes: []string{
            "TemporalTimeout:StartToClose",
        },
    },
})

var re bucketupload.Result
err := workflow.ExecuteActivity(
    opts,
    bucketupload.Name,
    &bucketupload.Params{
        Path:       "/path/to/file.zip",
        Key:        "file.zip",
        BufferSize: 100_000_000,
    },
).Get(opts, &re)

err may contain any system error. re.Key contains the object key used in the upload.

Documentation

Index

• Constants
• type Activity
  • func New(bucket *blob.Bucket) *Activity
  • func (a *Activity) Execute(ctx context.Context, params *Params) (*Result, error)
• type Params
• type Result

Constants

const Name = "bucket-upload"

Types

type Activity

type Activity struct {
	// contains filtered or unexported fields
}

func New

func New(bucket *blob.Bucket) *Activity

func (*Activity) Execute

func (a *Activity) Execute(ctx context.Context, params *Params) (*Result, error)

type Params

type Params struct {
	Path       string
	Key        string
	BufferSize int
}

type Result

type Result struct {
	Key string
}