dataproc

package
v0.34.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 5, 2018 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Overview

Package dataproc is an auto-generated package for the Google Cloud Dataproc API. 

NOTE: This package is in alpha. It is not stable, and is likely to change.

Manages Hadoop-based clusters and jobs on Google Cloud Platform.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultAuthScopes 

func DefaultAuthScopes() []string

DefaultAuthScopes reports the default set of authentication scopes to use with this package.

Types

type ClusterControllerCallOptions 

type ClusterControllerCallOptions struct {
	CreateCluster   []gax.CallOption
	UpdateCluster   []gax.CallOption
	DeleteCluster   []gax.CallOption
	GetCluster      []gax.CallOption
	ListClusters    []gax.CallOption
	DiagnoseCluster []gax.CallOption
}

ClusterControllerCallOptions contains the retry settings for each method of ClusterControllerClient.

type ClusterControllerClient 

type ClusterControllerClient struct {

	// LROClient is used internally to handle longrunning operations.
	// It is exposed so that its CallOptions can be modified if required.
	// Users should not Close this client.
	LROClient *lroauto.OperationsClient

	// The call options for this service.
	CallOptions *ClusterControllerCallOptions
	// contains filtered or unexported fields
}

ClusterControllerClient is a client for interacting with Google Cloud Dataproc API.

Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

func NewClusterControllerClient 

func NewClusterControllerClient(ctx context.Context, opts ...option.ClientOption) (*ClusterControllerClient, error)

NewClusterControllerClient creates a new cluster controller client.

The ClusterControllerService provides methods to manage clusters of Compute Engine instances.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewClusterControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use client.
	_ = c
}

Output:

func (*ClusterControllerClient) Close 

func (c *ClusterControllerClient) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*ClusterControllerClient) Connection 

func (c *ClusterControllerClient) Connection() *grpc.ClientConn

Connection returns the client's connection to the API service.

func (*ClusterControllerClient) CreateCluster 

func (c *ClusterControllerClient) CreateCluster(ctx context.Context, req *dataprocpb.CreateClusterRequest, opts ...gax.CallOption) (*CreateClusterOperation, error)

CreateCluster creates a cluster in a project.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewClusterControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.CreateClusterRequest{
		// TODO: Fill request struct fields.
	}
	op, err := c.CreateCluster(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}

	resp, err := op.Wait(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

func (*ClusterControllerClient) CreateClusterOperation 

func (c *ClusterControllerClient) CreateClusterOperation(name string) *CreateClusterOperation

CreateClusterOperation returns a new CreateClusterOperation from a given name. The name must be that of a previously created CreateClusterOperation, possibly from a different process.

func (*ClusterControllerClient) DeleteCluster 

func (c *ClusterControllerClient) DeleteCluster(ctx context.Context, req *dataprocpb.DeleteClusterRequest, opts ...gax.CallOption) (*DeleteClusterOperation, error)

DeleteCluster deletes a cluster in a project.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewClusterControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.DeleteClusterRequest{
		// TODO: Fill request struct fields.
	}
	op, err := c.DeleteCluster(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}

	err = op.Wait(ctx)
	// TODO: Handle error.
}

Output:

func (*ClusterControllerClient) DeleteClusterOperation 

func (c *ClusterControllerClient) DeleteClusterOperation(name string) *DeleteClusterOperation

DeleteClusterOperation returns a new DeleteClusterOperation from a given name. The name must be that of a previously created DeleteClusterOperation, possibly from a different process.

func (*ClusterControllerClient) DiagnoseCluster 

func (c *ClusterControllerClient) DiagnoseCluster(ctx context.Context, req *dataprocpb.DiagnoseClusterRequest, opts ...gax.CallOption) (*DiagnoseClusterOperation, error)

DiagnoseCluster gets cluster diagnostic information. After the operation completes, the Operation.response field contains DiagnoseClusterOutputLocation.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewClusterControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.DiagnoseClusterRequest{
		// TODO: Fill request struct fields.
	}
	op, err := c.DiagnoseCluster(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}

	err = op.Wait(ctx)
	// TODO: Handle error.
}

Output:

func (*ClusterControllerClient) DiagnoseClusterOperation 

func (c *ClusterControllerClient) DiagnoseClusterOperation(name string) *DiagnoseClusterOperation

DiagnoseClusterOperation returns a new DiagnoseClusterOperation from a given name. The name must be that of a previously created DiagnoseClusterOperation, possibly from a different process.

func (*ClusterControllerClient) GetCluster 

func (c *ClusterControllerClient) GetCluster(ctx context.Context, req *dataprocpb.GetClusterRequest, opts ...gax.CallOption) (*dataprocpb.Cluster, error)

GetCluster gets the resource representation for a cluster in a project.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewClusterControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.GetClusterRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.GetCluster(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

func (*ClusterControllerClient) ListClusters 

func (c *ClusterControllerClient) ListClusters(ctx context.Context, req *dataprocpb.ListClustersRequest, opts ...gax.CallOption) *ClusterIterator

ListClusters lists all regions/{region}/clusters in a project.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"
	"google.golang.org/api/iterator"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewClusterControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.ListClustersRequest{
		// TODO: Fill request struct fields.
	}
	it := c.ListClusters(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		// TODO: Use resp.
		_ = resp
	}
}

Output:

func (*ClusterControllerClient) UpdateCluster 

func (c *ClusterControllerClient) UpdateCluster(ctx context.Context, req *dataprocpb.UpdateClusterRequest, opts ...gax.CallOption) (*UpdateClusterOperation, error)

UpdateCluster updates a cluster in a project.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewClusterControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.UpdateClusterRequest{
		// TODO: Fill request struct fields.
	}
	op, err := c.UpdateCluster(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}

	resp, err := op.Wait(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

func (*ClusterControllerClient) UpdateClusterOperation 

func (c *ClusterControllerClient) UpdateClusterOperation(name string) *UpdateClusterOperation

UpdateClusterOperation returns a new UpdateClusterOperation from a given name. The name must be that of a previously created UpdateClusterOperation, possibly from a different process.

type ClusterIterator 

type ClusterIterator struct {

	// InternalFetch is for use by the Google Cloud Libraries only.
	// It is not part of the stable interface of this package.
	//
	// InternalFetch returns results from a single call to the underlying RPC.
	// The number of results is no greater than pageSize.
	// If there are no more results, nextPageToken is empty and err is nil.
	InternalFetch func(pageSize int, pageToken string) (results []*dataprocpb.Cluster, nextPageToken string, err error)
	// contains filtered or unexported fields
}

ClusterIterator manages a stream of *dataprocpb.Cluster.

func (*ClusterIterator) Next 

func (it *ClusterIterator) Next() (*dataprocpb.Cluster, error)

Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

func (*ClusterIterator) PageInfo 

func (it *ClusterIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

type CreateClusterOperation 

type CreateClusterOperation struct {
	// contains filtered or unexported fields
}

CreateClusterOperation manages a long-running operation from CreateCluster.

func (*CreateClusterOperation) Delete 

func (op *CreateClusterOperation) Delete(ctx context.Context, opts ...gax.CallOption) error

Delete deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation.

func (*CreateClusterOperation) Done 

func (op *CreateClusterOperation) Done() bool

Done reports whether the long-running operation has completed.

func (*CreateClusterOperation) Metadata 

func (op *CreateClusterOperation) Metadata() (*dataprocpb.ClusterOperationMetadata, error)

Metadata returns metadata associated with the long-running operation. Metadata itself does not contact the server, but Poll does. To get the latest metadata, call this method after a successful call to Poll. If the metadata is not available, the returned metadata and error are both nil.

func (*CreateClusterOperation) Name 

func (op *CreateClusterOperation) Name() string

Name returns the name of the long-running operation. The name is assigned by the server and is unique within the service from which the operation is created.

func (*CreateClusterOperation) Poll 

func (op *CreateClusterOperation) Poll(ctx context.Context, opts ...gax.CallOption) (*dataprocpb.Cluster, error)

Poll fetches the latest state of the long-running operation.

Poll also fetches the latest metadata, which can be retrieved by Metadata.

If Poll fails, the error is returned and op is unmodified. If Poll succeeds and the operation has completed with failure, the error is returned and op.Done will return true. If Poll succeeds and the operation has completed successfully, op.Done will return true, and the response of the operation is returned. If Poll succeeds and the operation has not completed, the returned response and error are both nil.

func (*CreateClusterOperation) Wait 

func (op *CreateClusterOperation) Wait(ctx context.Context, opts ...gax.CallOption) (*dataprocpb.Cluster, error)

Wait blocks until the long-running operation is completed, returning the response and any errors encountered.

See documentation of Poll for error-handling information.

type DeleteClusterOperation 

type DeleteClusterOperation struct {
	// contains filtered or unexported fields
}

DeleteClusterOperation manages a long-running operation from DeleteCluster.

func (*DeleteClusterOperation) Delete 

func (op *DeleteClusterOperation) Delete(ctx context.Context, opts ...gax.CallOption) error

Delete deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation.

func (*DeleteClusterOperation) Done 

func (op *DeleteClusterOperation) Done() bool

Done reports whether the long-running operation has completed.

func (*DeleteClusterOperation) Metadata 

func (op *DeleteClusterOperation) Metadata() (*dataprocpb.ClusterOperationMetadata, error)

Metadata returns metadata associated with the long-running operation. Metadata itself does not contact the server, but Poll does. To get the latest metadata, call this method after a successful call to Poll. If the metadata is not available, the returned metadata and error are both nil.

func (*DeleteClusterOperation) Name 

func (op *DeleteClusterOperation) Name() string

Name returns the name of the long-running operation. The name is assigned by the server and is unique within the service from which the operation is created.

func (*DeleteClusterOperation) Poll 

func (op *DeleteClusterOperation) Poll(ctx context.Context, opts ...gax.CallOption) error

Poll fetches the latest state of the long-running operation.

Poll also fetches the latest metadata, which can be retrieved by Metadata.

If Poll fails, the error is returned and op is unmodified. If Poll succeeds and the operation has completed with failure, the error is returned and op.Done will return true. If Poll succeeds and the operation has completed successfully, op.Done will return true.

func (*DeleteClusterOperation) Wait 

func (op *DeleteClusterOperation) Wait(ctx context.Context, opts ...gax.CallOption) error

Wait blocks until the long-running operation is completed, returning any error encountered.

See documentation of Poll for error-handling information.

type DiagnoseClusterOperation 

type DiagnoseClusterOperation struct {
	// contains filtered or unexported fields
}

DiagnoseClusterOperation manages a long-running operation from DiagnoseCluster.

func (*DiagnoseClusterOperation) Delete 

func (op *DiagnoseClusterOperation) Delete(ctx context.Context, opts ...gax.CallOption) error

Delete deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation.

func (*DiagnoseClusterOperation) Done 

func (op *DiagnoseClusterOperation) Done() bool

Done reports whether the long-running operation has completed.

func (*DiagnoseClusterOperation) Metadata 

func (op *DiagnoseClusterOperation) Metadata() (*dataprocpb.DiagnoseClusterResults, error)

Metadata returns metadata associated with the long-running operation. Metadata itself does not contact the server, but Poll does. To get the latest metadata, call this method after a successful call to Poll. If the metadata is not available, the returned metadata and error are both nil.

func (*DiagnoseClusterOperation) Name 

func (op *DiagnoseClusterOperation) Name() string

Name returns the name of the long-running operation. The name is assigned by the server and is unique within the service from which the operation is created.

func (*DiagnoseClusterOperation) Poll 

func (op *DiagnoseClusterOperation) Poll(ctx context.Context, opts ...gax.CallOption) error

Poll fetches the latest state of the long-running operation.

Poll also fetches the latest metadata, which can be retrieved by Metadata.

If Poll fails, the error is returned and op is unmodified. If Poll succeeds and the operation has completed with failure, the error is returned and op.Done will return true. If Poll succeeds and the operation has completed successfully, op.Done will return true.

func (*DiagnoseClusterOperation) Wait 

func (op *DiagnoseClusterOperation) Wait(ctx context.Context, opts ...gax.CallOption) error

Wait blocks until the long-running operation is completed, returning any error encountered.

See documentation of Poll for error-handling information.

type InstantiateInlineWorkflowTemplateOperation added in v0.34.0 

type InstantiateInlineWorkflowTemplateOperation struct {
	// contains filtered or unexported fields
}

InstantiateInlineWorkflowTemplateOperation manages a long-running operation from InstantiateInlineWorkflowTemplate.

func (*InstantiateInlineWorkflowTemplateOperation) Cancel added in v0.34.0 

func (op *InstantiateInlineWorkflowTemplateOperation) Cancel(ctx context.Context, opts ...gax.CallOption) error

Cancel starts asynchronous cancellation on a long-running operation. The server makes a best effort to cancel the operation, but success is not guaranteed. Clients can use Poll or other methods to check whether the cancellation succeeded or whether the operation completed despite cancellation. On successful cancellation, the operation is not deleted; instead, op.Poll returns an error with code Canceled.

func (*InstantiateInlineWorkflowTemplateOperation) Delete added in v0.34.0 

func (op *InstantiateInlineWorkflowTemplateOperation) Delete(ctx context.Context, opts ...gax.CallOption) error

Delete deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation.

func (*InstantiateInlineWorkflowTemplateOperation) Done added in v0.34.0 

func (op *InstantiateInlineWorkflowTemplateOperation) Done() bool

Done reports whether the long-running operation has completed.

func (*InstantiateInlineWorkflowTemplateOperation) Metadata added in v0.34.0 

func (op *InstantiateInlineWorkflowTemplateOperation) Metadata() (*dataprocpb.WorkflowMetadata, error)

Metadata returns metadata associated with the long-running operation. Metadata itself does not contact the server, but Poll does. To get the latest metadata, call this method after a successful call to Poll. If the metadata is not available, the returned metadata and error are both nil.

func (*InstantiateInlineWorkflowTemplateOperation) Name added in v0.34.0 

func (op *InstantiateInlineWorkflowTemplateOperation) Name() string

Name returns the name of the long-running operation. The name is assigned by the server and is unique within the service from which the operation is created.

func (*InstantiateInlineWorkflowTemplateOperation) Poll added in v0.34.0 

func (op *InstantiateInlineWorkflowTemplateOperation) Poll(ctx context.Context, opts ...gax.CallOption) error

Poll fetches the latest state of the long-running operation.

Poll also fetches the latest metadata, which can be retrieved by Metadata.

If Poll fails, the error is returned and op is unmodified. If Poll succeeds and the operation has completed with failure, the error is returned and op.Done will return true. If Poll succeeds and the operation has completed successfully, op.Done will return true.

func (*InstantiateInlineWorkflowTemplateOperation) Wait added in v0.34.0 

func (op *InstantiateInlineWorkflowTemplateOperation) Wait(ctx context.Context, opts ...gax.CallOption) error

Wait blocks until the long-running operation is completed, returning any error encountered.

See documentation of Poll for error-handling information.

type InstantiateWorkflowTemplateOperation 

type InstantiateWorkflowTemplateOperation struct {
	// contains filtered or unexported fields
}

InstantiateWorkflowTemplateOperation manages a long-running operation from InstantiateWorkflowTemplate.

func (*InstantiateWorkflowTemplateOperation) Cancel 

func (op *InstantiateWorkflowTemplateOperation) Cancel(ctx context.Context, opts ...gax.CallOption) error

Cancel starts asynchronous cancellation on a long-running operation. The server makes a best effort to cancel the operation, but success is not guaranteed. Clients can use Poll or other methods to check whether the cancellation succeeded or whether the operation completed despite cancellation. On successful cancellation, the operation is not deleted; instead, op.Poll returns an error with code Canceled.

func (*InstantiateWorkflowTemplateOperation) Delete 

func (op *InstantiateWorkflowTemplateOperation) Delete(ctx context.Context, opts ...gax.CallOption) error

Delete deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation.

func (*InstantiateWorkflowTemplateOperation) Done 

func (op *InstantiateWorkflowTemplateOperation) Done() bool

Done reports whether the long-running operation has completed.

func (*InstantiateWorkflowTemplateOperation) Metadata 

func (op *InstantiateWorkflowTemplateOperation) Metadata() (*dataprocpb.WorkflowMetadata, error)

Metadata returns metadata associated with the long-running operation. Metadata itself does not contact the server, but Poll does. To get the latest metadata, call this method after a successful call to Poll. If the metadata is not available, the returned metadata and error are both nil.

func (*InstantiateWorkflowTemplateOperation) Name 

func (op *InstantiateWorkflowTemplateOperation) Name() string

Name returns the name of the long-running operation. The name is assigned by the server and is unique within the service from which the operation is created.

func (*InstantiateWorkflowTemplateOperation) Poll 

func (op *InstantiateWorkflowTemplateOperation) Poll(ctx context.Context, opts ...gax.CallOption) error

Poll fetches the latest state of the long-running operation.

Poll also fetches the latest metadata, which can be retrieved by Metadata.

If Poll fails, the error is returned and op is unmodified. If Poll succeeds and the operation has completed with failure, the error is returned and op.Done will return true. If Poll succeeds and the operation has completed successfully, op.Done will return true.

func (*InstantiateWorkflowTemplateOperation) Wait 

func (op *InstantiateWorkflowTemplateOperation) Wait(ctx context.Context, opts ...gax.CallOption) error

Wait blocks until the long-running operation is completed, returning any error encountered.

See documentation of Poll for error-handling information.

type JobControllerCallOptions 

type JobControllerCallOptions struct {
	SubmitJob []gax.CallOption
	GetJob    []gax.CallOption
	ListJobs  []gax.CallOption
	UpdateJob []gax.CallOption
	CancelJob []gax.CallOption
	DeleteJob []gax.CallOption
}

JobControllerCallOptions contains the retry settings for each method of JobControllerClient.

type JobControllerClient 

type JobControllerClient struct {

	// The call options for this service.
	CallOptions *JobControllerCallOptions
	// contains filtered or unexported fields
}

JobControllerClient is a client for interacting with Google Cloud Dataproc API.

Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

func NewJobControllerClient 

func NewJobControllerClient(ctx context.Context, opts ...option.ClientOption) (*JobControllerClient, error)

NewJobControllerClient creates a new job controller client.

The JobController provides methods to manage jobs.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewJobControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use client.
	_ = c
}

Output:

func (*JobControllerClient) CancelJob 

func (c *JobControllerClient) CancelJob(ctx context.Context, req *dataprocpb.CancelJobRequest, opts ...gax.CallOption) (*dataprocpb.Job, error)

CancelJob starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list (at /dataproc/docs/reference/rest/v1beta2/projects.regions.jobs/list) or regions/{region}/jobs.get (at /dataproc/docs/reference/rest/v1beta2/projects.regions.jobs/get).

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewJobControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.CancelJobRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.CancelJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

func (*JobControllerClient) Close 

func (c *JobControllerClient) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*JobControllerClient) Connection 

func (c *JobControllerClient) Connection() *grpc.ClientConn

Connection returns the client's connection to the API service.

func (*JobControllerClient) DeleteJob 

func (c *JobControllerClient) DeleteJob(ctx context.Context, req *dataprocpb.DeleteJobRequest, opts ...gax.CallOption) error

DeleteJob deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewJobControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.DeleteJobRequest{
		// TODO: Fill request struct fields.
	}
	err = c.DeleteJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
}

Output:

func (*JobControllerClient) GetJob 

func (c *JobControllerClient) GetJob(ctx context.Context, req *dataprocpb.GetJobRequest, opts ...gax.CallOption) (*dataprocpb.Job, error)

GetJob gets the resource representation for a job in a project.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewJobControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.GetJobRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.GetJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

func (*JobControllerClient) ListJobs 

func (c *JobControllerClient) ListJobs(ctx context.Context, req *dataprocpb.ListJobsRequest, opts ...gax.CallOption) *JobIterator

ListJobs lists regions/{region}/jobs in a project.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"
	"google.golang.org/api/iterator"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewJobControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.ListJobsRequest{
		// TODO: Fill request struct fields.
	}
	it := c.ListJobs(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		// TODO: Use resp.
		_ = resp
	}
}

Output:

func (*JobControllerClient) SubmitJob 

func (c *JobControllerClient) SubmitJob(ctx context.Context, req *dataprocpb.SubmitJobRequest, opts ...gax.CallOption) (*dataprocpb.Job, error)

SubmitJob submits a job to a cluster.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewJobControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.SubmitJobRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.SubmitJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

func (*JobControllerClient) UpdateJob 

func (c *JobControllerClient) UpdateJob(ctx context.Context, req *dataprocpb.UpdateJobRequest, opts ...gax.CallOption) (*dataprocpb.Job, error)

UpdateJob updates a job in a project.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewJobControllerClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.UpdateJobRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.UpdateJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

type JobIterator 

type JobIterator struct {

	// InternalFetch is for use by the Google Cloud Libraries only.
	// It is not part of the stable interface of this package.
	//
	// InternalFetch returns results from a single call to the underlying RPC.
	// The number of results is no greater than pageSize.
	// If there are no more results, nextPageToken is empty and err is nil.
	InternalFetch func(pageSize int, pageToken string) (results []*dataprocpb.Job, nextPageToken string, err error)
	// contains filtered or unexported fields
}

JobIterator manages a stream of *dataprocpb.Job.

func (*JobIterator) Next 

func (it *JobIterator) Next() (*dataprocpb.Job, error)

Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

func (*JobIterator) PageInfo 

func (it *JobIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

type UpdateClusterOperation 

type UpdateClusterOperation struct {
	// contains filtered or unexported fields
}

UpdateClusterOperation manages a long-running operation from UpdateCluster.

func (*UpdateClusterOperation) Delete 

func (op *UpdateClusterOperation) Delete(ctx context.Context, opts ...gax.CallOption) error

Delete deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation.

func (*UpdateClusterOperation) Done 

func (op *UpdateClusterOperation) Done() bool

Done reports whether the long-running operation has completed.

func (*UpdateClusterOperation) Metadata 

func (op *UpdateClusterOperation) Metadata() (*dataprocpb.ClusterOperationMetadata, error)

Metadata returns metadata associated with the long-running operation. Metadata itself does not contact the server, but Poll does. To get the latest metadata, call this method after a successful call to Poll. If the metadata is not available, the returned metadata and error are both nil.

func (*UpdateClusterOperation) Name 

func (op *UpdateClusterOperation) Name() string

Name returns the name of the long-running operation. The name is assigned by the server and is unique within the service from which the operation is created.

func (*UpdateClusterOperation) Poll 

func (op *UpdateClusterOperation) Poll(ctx context.Context, opts ...gax.CallOption) (*dataprocpb.Cluster, error)

Poll fetches the latest state of the long-running operation.

Poll also fetches the latest metadata, which can be retrieved by Metadata.

If Poll fails, the error is returned and op is unmodified. If Poll succeeds and the operation has completed with failure, the error is returned and op.Done will return true. If Poll succeeds and the operation has completed successfully, op.Done will return true, and the response of the operation is returned. If Poll succeeds and the operation has not completed, the returned response and error are both nil.

func (*UpdateClusterOperation) Wait 

func (op *UpdateClusterOperation) Wait(ctx context.Context, opts ...gax.CallOption) (*dataprocpb.Cluster, error)

Wait blocks until the long-running operation is completed, returning the response and any errors encountered.

See documentation of Poll for error-handling information.

type WorkflowTemplateCallOptions 

type WorkflowTemplateCallOptions struct {
	CreateWorkflowTemplate            []gax.CallOption
	GetWorkflowTemplate               []gax.CallOption
	InstantiateWorkflowTemplate       []gax.CallOption
	InstantiateInlineWorkflowTemplate []gax.CallOption
	UpdateWorkflowTemplate            []gax.CallOption
	ListWorkflowTemplates             []gax.CallOption
	DeleteWorkflowTemplate            []gax.CallOption
}

WorkflowTemplateCallOptions contains the retry settings for each method of WorkflowTemplateClient.

type WorkflowTemplateClient 

type WorkflowTemplateClient struct {

	// LROClient is used internally to handle longrunning operations.
	// It is exposed so that its CallOptions can be modified if required.
	// Users should not Close this client.
	LROClient *lroauto.OperationsClient

	// The call options for this service.
	CallOptions *WorkflowTemplateCallOptions
	// contains filtered or unexported fields
}

WorkflowTemplateClient is a client for interacting with Google Cloud Dataproc API.

Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

func NewWorkflowTemplateClient 

func NewWorkflowTemplateClient(ctx context.Context, opts ...option.ClientOption) (*WorkflowTemplateClient, error)

NewWorkflowTemplateClient creates a new workflow template service client.

The API interface for managing Workflow Templates in the Cloud Dataproc API.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewWorkflowTemplateClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use client.
	_ = c
}

Output:

func (*WorkflowTemplateClient) Close 

func (c *WorkflowTemplateClient) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*WorkflowTemplateClient) Connection 

func (c *WorkflowTemplateClient) Connection() *grpc.ClientConn

Connection returns the client's connection to the API service.

func (*WorkflowTemplateClient) CreateWorkflowTemplate 

func (c *WorkflowTemplateClient) CreateWorkflowTemplate(ctx context.Context, req *dataprocpb.CreateWorkflowTemplateRequest, opts ...gax.CallOption) (*dataprocpb.WorkflowTemplate, error)

CreateWorkflowTemplate creates new workflow template.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewWorkflowTemplateClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.CreateWorkflowTemplateRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.CreateWorkflowTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

func (*WorkflowTemplateClient) DeleteWorkflowTemplate 

func (c *WorkflowTemplateClient) DeleteWorkflowTemplate(ctx context.Context, req *dataprocpb.DeleteWorkflowTemplateRequest, opts ...gax.CallOption) error

DeleteWorkflowTemplate deletes a workflow template. It does not cancel in-progress workflows.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewWorkflowTemplateClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.DeleteWorkflowTemplateRequest{
		// TODO: Fill request struct fields.
	}
	err = c.DeleteWorkflowTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
}

Output:

func (*WorkflowTemplateClient) GetWorkflowTemplate 

func (c *WorkflowTemplateClient) GetWorkflowTemplate(ctx context.Context, req *dataprocpb.GetWorkflowTemplateRequest, opts ...gax.CallOption) (*dataprocpb.WorkflowTemplate, error)

GetWorkflowTemplate retrieves the latest workflow template.

Can retrieve previously instantiated template by specifying optional version parameter.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewWorkflowTemplateClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.GetWorkflowTemplateRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.GetWorkflowTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

func (*WorkflowTemplateClient) InstantiateInlineWorkflowTemplate added in v0.34.0 

func (c *WorkflowTemplateClient) InstantiateInlineWorkflowTemplate(ctx context.Context, req *dataprocpb.InstantiateInlineWorkflowTemplateRequest, opts ...gax.CallOption) (*InstantiateInlineWorkflowTemplateOperation, error)

InstantiateInlineWorkflowTemplate instantiates a template and begins execution.

This method is equivalent to executing the sequence [CreateWorkflowTemplate][google.cloud.dataproc.v1beta2.WorkflowTemplateService.CreateWorkflowTemplate], [InstantiateWorkflowTemplate][google.cloud.dataproc.v1beta2.WorkflowTemplateService.InstantiateWorkflowTemplate], [DeleteWorkflowTemplate][google.cloud.dataproc.v1beta2.WorkflowTemplateService.DeleteWorkflowTemplate].

The returned Operation can be used to track execution of workflow by polling [operations.get][google.longrunning.Operations.GetOperation]. The Operation will complete when entire workflow is finished.

The running workflow can be aborted via [operations.cancel][google.longrunning.Operations.CancelOperation]. This will cause any inflight jobs to be cancelled and workflow-owned clusters to be deleted.

The [Operation.metadata][google.longrunning.Operation.metadata] will be [WorkflowMetadata][google.cloud.dataproc.v1beta2.WorkflowMetadata].

On successful completion, [Operation.response][google.longrunning.Operation.response] will be [Empty][google.protobuf.Empty].

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewWorkflowTemplateClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.InstantiateInlineWorkflowTemplateRequest{
		// TODO: Fill request struct fields.
	}
	op, err := c.InstantiateInlineWorkflowTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}

	err = op.Wait(ctx)
	// TODO: Handle error.
}

Output:

func (*WorkflowTemplateClient) InstantiateInlineWorkflowTemplateOperation added in v0.34.0 

func (c *WorkflowTemplateClient) InstantiateInlineWorkflowTemplateOperation(name string) *InstantiateInlineWorkflowTemplateOperation

InstantiateInlineWorkflowTemplateOperation returns a new InstantiateInlineWorkflowTemplateOperation from a given name. The name must be that of a previously created InstantiateInlineWorkflowTemplateOperation, possibly from a different process.

func (*WorkflowTemplateClient) InstantiateWorkflowTemplate 

func (c *WorkflowTemplateClient) InstantiateWorkflowTemplate(ctx context.Context, req *dataprocpb.InstantiateWorkflowTemplateRequest, opts ...gax.CallOption) (*InstantiateWorkflowTemplateOperation, error)

InstantiateWorkflowTemplate instantiates a template and begins execution.

The returned Operation can be used to track execution of workflow by polling [operations.get][google.longrunning.Operations.GetOperation]. The Operation will complete when entire workflow is finished.

The running workflow can be aborted via [operations.cancel][google.longrunning.Operations.CancelOperation]. This will cause any inflight jobs to be cancelled and workflow-owned clusters to be deleted.

The [Operation.metadata][google.longrunning.Operation.metadata] will be [WorkflowMetadata][google.cloud.dataproc.v1beta2.WorkflowMetadata].

On successful completion, [Operation.response][google.longrunning.Operation.response] will be [Empty][google.protobuf.Empty].

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewWorkflowTemplateClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.InstantiateWorkflowTemplateRequest{
		// TODO: Fill request struct fields.
	}
	op, err := c.InstantiateWorkflowTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}

	err = op.Wait(ctx)
	// TODO: Handle error.
}

Output:

func (*WorkflowTemplateClient) InstantiateWorkflowTemplateOperation 

func (c *WorkflowTemplateClient) InstantiateWorkflowTemplateOperation(name string) *InstantiateWorkflowTemplateOperation

InstantiateWorkflowTemplateOperation returns a new InstantiateWorkflowTemplateOperation from a given name. The name must be that of a previously created InstantiateWorkflowTemplateOperation, possibly from a different process.

func (*WorkflowTemplateClient) ListWorkflowTemplates 

func (c *WorkflowTemplateClient) ListWorkflowTemplates(ctx context.Context, req *dataprocpb.ListWorkflowTemplatesRequest, opts ...gax.CallOption) *WorkflowTemplateIterator

ListWorkflowTemplates lists workflows that match the specified filter in the request.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"
	"google.golang.org/api/iterator"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewWorkflowTemplateClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.ListWorkflowTemplatesRequest{
		// TODO: Fill request struct fields.
	}
	it := c.ListWorkflowTemplates(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		// TODO: Use resp.
		_ = resp
	}
}

Output:

func (*WorkflowTemplateClient) UpdateWorkflowTemplate 

func (c *WorkflowTemplateClient) UpdateWorkflowTemplate(ctx context.Context, req *dataprocpb.UpdateWorkflowTemplateRequest, opts ...gax.CallOption) (*dataprocpb.WorkflowTemplate, error)

UpdateWorkflowTemplate updates (replaces) workflow template. The updated template must contain version that matches the current server version.

Example 
package main

import (
	"context"

	dataproc "cloud.google.com/go/dataproc/apiv1beta2"

	dataprocpb "google.golang.org/genproto/googleapis/cloud/dataproc/v1beta2"
)

func main() {
	ctx := context.Background()
	c, err := dataproc.NewWorkflowTemplateClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}

	req := &dataprocpb.UpdateWorkflowTemplateRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.UpdateWorkflowTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

Output:

type WorkflowTemplateIterator 

type WorkflowTemplateIterator struct {

	// InternalFetch is for use by the Google Cloud Libraries only.
	// It is not part of the stable interface of this package.
	//
	// InternalFetch returns results from a single call to the underlying RPC.
	// The number of results is no greater than pageSize.
	// If there are no more results, nextPageToken is empty and err is nil.
	InternalFetch func(pageSize int, pageToken string) (results []*dataprocpb.WorkflowTemplate, nextPageToken string, err error)
	// contains filtered or unexported fields
}

WorkflowTemplateIterator manages a stream of *dataprocpb.WorkflowTemplate.

func (*WorkflowTemplateIterator) Next 

func (it *WorkflowTemplateIterator) Next() (*dataprocpb.WorkflowTemplate, error)

Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

func (*WorkflowTemplateIterator) PageInfo 

func (it *WorkflowTemplateIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.