job

package
v11.2.1-beta+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 7, 2017 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package job implements the Azure ARM Job service API version 2016-03-20-preview.

Creates an Azure Data Lake Analytics job client.

Index

Constants

View Source 
const (
	// DefaultAdlaJobDNSSuffix is the default value for adla job dns suffix
	DefaultAdlaJobDNSSuffix = "azuredatalakeanalytics.net"
)

Variables

This section is empty.

Functions

func UserAgent 

func UserAgent() string

UserAgent returns the UserAgent string to use when sending http.Requests.

func Version 

func Version() string

Version returns the semantic version (see http://semver.org) of the client.

Types

type CompileMode 

type CompileMode string

CompileMode enumerates the values for compile mode. 

const (
	// Full specifies the full state for compile mode.
	Full CompileMode = "Full"
	// Semantic specifies the semantic state for compile mode.
	Semantic CompileMode = "Semantic"
	// SingleBox specifies the single box state for compile mode.
	SingleBox CompileMode = "SingleBox"
)

type DataPath 

type DataPath struct {
	autorest.Response `json:"-"`
	JobID             *uuid.UUID `json:"jobId,omitempty"`
	Command           *string    `json:"command,omitempty"`
	Paths             *[]string  `json:"paths,omitempty"`
}

DataPath is a Data Lake Analytics job data path item.

type Diagnostics 

type Diagnostics struct {
	ColumnNumber *int32        `json:"columnNumber,omitempty"`
	End          *int32        `json:"end,omitempty"`
	LineNumber   *int32        `json:"lineNumber,omitempty"`
	Message      *string       `json:"message,omitempty"`
	Severity     SeverityTypes `json:"severity,omitempty"`
	Start        *int32        `json:"start,omitempty"`
}

Diagnostics is

type ErrorDetails 

type ErrorDetails struct {
	Description         *string       `json:"description,omitempty"`
	Details             *string       `json:"details,omitempty"`
	EndOffset           *int32        `json:"endOffset,omitempty"`
	ErrorID             *string       `json:"errorId,omitempty"`
	FilePath            *string       `json:"filePath,omitempty"`
	HelpLink            *string       `json:"helpLink,omitempty"`
	InternalDiagnostics *string       `json:"internalDiagnostics,omitempty"`
	LineNumber          *int32        `json:"lineNumber,omitempty"`
	Message             *string       `json:"message,omitempty"`
	Resolution          *string       `json:"resolution,omitempty"`
	InnerError          *InnerError   `json:"innerError,omitempty"`
	Severity            SeverityTypes `json:"severity,omitempty"`
	Source              *string       `json:"source,omitempty"`
	StartOffset         *int32        `json:"startOffset,omitempty"`
}

ErrorDetails is the Data Lake Analytics job error details.

type GroupClient 

type GroupClient struct {
	ManagementClient
}

GroupClient is the creates an Azure Data Lake Analytics job client.

func NewGroupClient 

func NewGroupClient() GroupClient

NewGroupClient creates an instance of the GroupClient client.

func (GroupClient) Build 

func (client GroupClient) Build(accountName string, parameters Information) (result Information, err error)

Build builds (compiles) the specified job in the specified Data Lake Analytics account for job correctness and validation.

accountName is the Azure Data Lake Analytics account to execute job operations on. parameters is the parameters to build a job.

func (GroupClient) BuildPreparer 

func (client GroupClient) BuildPreparer(accountName string, parameters Information) (*http.Request, error)

BuildPreparer prepares the Build request.

func (GroupClient) BuildResponder 

func (client GroupClient) BuildResponder(resp *http.Response) (result Information, err error)

BuildResponder handles the response to the Build request. The method always closes the http.Response Body.

func (GroupClient) BuildSender 

func (client GroupClient) BuildSender(req *http.Request) (*http.Response, error)

BuildSender sends the Build request. The method will close the http.Response Body if it receives an error.

func (GroupClient) Cancel 

func (client GroupClient) Cancel(accountName string, jobIdentity uuid.UUID) (result autorest.Response, err error)

Cancel cancels the running job specified by the job ID.

accountName is the Azure Data Lake Analytics account to execute job operations on. jobIdentity is jobInfo ID to cancel.

func (GroupClient) CancelPreparer 

func (client GroupClient) CancelPreparer(accountName string, jobIdentity uuid.UUID) (*http.Request, error)

CancelPreparer prepares the Cancel request.

func (GroupClient) CancelResponder 

func (client GroupClient) CancelResponder(resp *http.Response) (result autorest.Response, err error)

CancelResponder handles the response to the Cancel request. The method always closes the http.Response Body.

func (GroupClient) CancelSender 

func (client GroupClient) CancelSender(req *http.Request) (*http.Response, error)

CancelSender sends the Cancel request. The method will close the http.Response Body if it receives an error.

func (GroupClient) Create 

func (client GroupClient) Create(accountName string, jobIdentity uuid.UUID, parameters Information) (result Information, err error)

Create submits a job to the specified Data Lake Analytics account.

accountName is the Azure Data Lake Analytics account to execute job operations on. jobIdentity is the job ID (a GUID) for the job being submitted. parameters is the parameters to submit a job.

func (GroupClient) CreatePreparer 

func (client GroupClient) CreatePreparer(accountName string, jobIdentity uuid.UUID, parameters Information) (*http.Request, error)

CreatePreparer prepares the Create request.

func (GroupClient) CreateResponder 

func (client GroupClient) CreateResponder(resp *http.Response) (result Information, err error)

CreateResponder handles the response to the Create request. The method always closes the http.Response Body.

func (GroupClient) CreateSender 

func (client GroupClient) CreateSender(req *http.Request) (*http.Response, error)

CreateSender sends the Create request. The method will close the http.Response Body if it receives an error.

func (GroupClient) Get 

func (client GroupClient) Get(accountName string, jobIdentity uuid.UUID) (result Information, err error)

Get gets the job information for the specified job ID.

accountName is the Azure Data Lake Analytics account to execute job operations on. jobIdentity is jobInfo ID.

func (GroupClient) GetDebugDataPath 

func (client GroupClient) GetDebugDataPath(accountName string, jobIdentity uuid.UUID) (result DataPath, err error)

GetDebugDataPath gets the job debug data information specified by the job ID.

accountName is the Azure Data Lake Analytics account to execute job operations on. jobIdentity is jobInfo ID.

func (GroupClient) GetDebugDataPathPreparer 

func (client GroupClient) GetDebugDataPathPreparer(accountName string, jobIdentity uuid.UUID) (*http.Request, error)

GetDebugDataPathPreparer prepares the GetDebugDataPath request.

func (GroupClient) GetDebugDataPathResponder 

func (client GroupClient) GetDebugDataPathResponder(resp *http.Response) (result DataPath, err error)

GetDebugDataPathResponder handles the response to the GetDebugDataPath request. The method always closes the http.Response Body.

func (GroupClient) GetDebugDataPathSender 

func (client GroupClient) GetDebugDataPathSender(req *http.Request) (*http.Response, error)

GetDebugDataPathSender sends the GetDebugDataPath request. The method will close the http.Response Body if it receives an error.

func (GroupClient) GetPreparer 

func (client GroupClient) GetPreparer(accountName string, jobIdentity uuid.UUID) (*http.Request, error)

GetPreparer prepares the Get request.

func (GroupClient) GetResponder 

func (client GroupClient) GetResponder(resp *http.Response) (result Information, err error)

GetResponder handles the response to the Get request. The method always closes the http.Response Body.

func (GroupClient) GetSender 

func (client GroupClient) GetSender(req *http.Request) (*http.Response, error)

GetSender sends the Get request. The method will close the http.Response Body if it receives an error.

func (GroupClient) GetStatistics 

func (client GroupClient) GetStatistics(accountName string, jobIdentity uuid.UUID) (result Statistics, err error)

GetStatistics gets statistics of the specified job.

accountName is the Azure Data Lake Analytics account to execute job operations on. jobIdentity is job Information ID.

func (GroupClient) GetStatisticsPreparer 

func (client GroupClient) GetStatisticsPreparer(accountName string, jobIdentity uuid.UUID) (*http.Request, error)

GetStatisticsPreparer prepares the GetStatistics request.

func (GroupClient) GetStatisticsResponder 

func (client GroupClient) GetStatisticsResponder(resp *http.Response) (result Statistics, err error)

GetStatisticsResponder handles the response to the GetStatistics request. The method always closes the http.Response Body.

func (GroupClient) GetStatisticsSender 

func (client GroupClient) GetStatisticsSender(req *http.Request) (*http.Response, error)

GetStatisticsSender sends the GetStatistics request. The method will close the http.Response Body if it receives an error.

func (GroupClient) List 

func (client GroupClient) List(accountName string, filter string, top *int32, skip *int32, expand string, selectParameter string, orderby string, count *bool, search string, formatParameter string) (result InfoListResult, err error)

List lists the jobs, if any, associated with the specified Data Lake Analytics account. The response includes a link to the next page of results, if any.

accountName is the Azure Data Lake Analytics account to execute job operations on. filter is oData filter. Optional. top is the number of items to return. Optional. skip is the number of items to skip over before returning elements. Optional. expand is oData expansion. Expand related resources in line with the retrieved resources, e.g. Categories?$expand=Products would expand Product data in line with each Category entry. Optional. selectParameter is oData Select statement. Limits the properties on each entry to just those requested, e.g. Categories?$select=CategoryName,Description. Optional. orderby is orderBy clause. One or more comma-separated expressions with an optional "asc" (the default) or "desc" depending on the order you'd like the values sorted, e.g. Categories?$orderby=CategoryName desc. Optional. count is the Boolean value of true or false to request a count of the matching resources included with the resources in the response, e.g. Categories?$count=true. Optional. search is a free form search. A free-text search expression to match for whether a particular entry should be included in the feed, e.g. Categories?$search=blue OR green. Optional. formatParameter is the return format. Return the response in particular formatxii without access to request headers for standard content-type negotiation (e.g Orders?$format=json). Optional.

func (GroupClient) ListComplete 

func (client GroupClient) ListComplete(accountName string, filter string, top *int32, skip *int32, expand string, selectParameter string, orderby string, count *bool, search string, formatParameter string, cancel <-chan struct{}) (<-chan Information, <-chan error)

ListComplete gets all elements from the list without paging.

func (GroupClient) ListNextResults 

func (client GroupClient) ListNextResults(lastResults InfoListResult) (result InfoListResult, err error)

ListNextResults retrieves the next set of results, if any.

func (GroupClient) ListPreparer 

func (client GroupClient) ListPreparer(accountName string, filter string, top *int32, skip *int32, expand string, selectParameter string, orderby string, count *bool, search string, formatParameter string) (*http.Request, error)

ListPreparer prepares the List request.

func (GroupClient) ListResponder 

func (client GroupClient) ListResponder(resp *http.Response) (result InfoListResult, err error)

ListResponder handles the response to the List request. The method always closes the http.Response Body.

func (GroupClient) ListSender 

func (client GroupClient) ListSender(req *http.Request) (*http.Response, error)

ListSender sends the List request. The method will close the http.Response Body if it receives an error.

type HiveJobProperties 

type HiveJobProperties struct {
	RuntimeVersion         *string `json:"runtimeVersion,omitempty"`
	Script                 *string `json:"script,omitempty"`
	Type                   Type    `json:"type,omitempty"`
	LogsLocation           *string `json:"logsLocation,omitempty"`
	OutputLocation         *string `json:"outputLocation,omitempty"`
	StatementCount         *int32  `json:"statementCount,omitempty"`
	ExecutedStatementCount *int32  `json:"executedStatementCount,omitempty"`
}

HiveJobProperties is

func (HiveJobProperties) AsHiveJobProperties 

func (hjp HiveJobProperties) AsHiveJobProperties() (*HiveJobProperties, bool)

AsHiveJobProperties is the Properties implementation for HiveJobProperties.

func (HiveJobProperties) AsUSQLJobProperties 

func (hjp HiveJobProperties) AsUSQLJobProperties() (*USQLJobProperties, bool)

AsUSQLJobProperties is the Properties implementation for HiveJobProperties.

func (HiveJobProperties) MarshalJSON 

func (hjp HiveJobProperties) MarshalJSON() ([]byte, error)

MarshalJSON is the custom marshaler for HiveJobProperties.

type InfoListResult 

type InfoListResult struct {
	autorest.Response `json:"-"`
	Value             *[]Information `json:"value,omitempty"`
	NextLink          *string        `json:"nextLink,omitempty"`
	Count             *int64         `json:"count,omitempty"`
}

InfoListResult is list of jobInfo items.

func (InfoListResult) InfoListResultPreparer 

func (client InfoListResult) InfoListResultPreparer() (*http.Request, error)

InfoListResultPreparer prepares a request to retrieve the next set of results. It returns nil if no more results exist.

type Information 

type Information struct {
	autorest.Response   `json:"-"`
	JobID               *uuid.UUID          `json:"jobId,omitempty"`
	Name                *string             `json:"name,omitempty"`
	Type                Type                `json:"type,omitempty"`
	Submitter           *string             `json:"submitter,omitempty"`
	ErrorMessage        *[]ErrorDetails     `json:"errorMessage,omitempty"`
	DegreeOfParallelism *int32              `json:"degreeOfParallelism,omitempty"`
	Priority            *int32              `json:"priority,omitempty"`
	SubmitTime          *date.Time          `json:"submitTime,omitempty"`
	StartTime           *date.Time          `json:"startTime,omitempty"`
	EndTime             *date.Time          `json:"endTime,omitempty"`
	State               State               `json:"state,omitempty"`
	Result              Result              `json:"result,omitempty"`
	LogFolder           *string             `json:"logFolder,omitempty"`
	LogFilePatterns     *[]string           `json:"logFilePatterns,omitempty"`
	StateAuditRecords   *[]StateAuditRecord `json:"stateAuditRecords,omitempty"`
	Properties          Properties          `json:"properties,omitempty"`
}

Information is the common Data Lake Analytics job information properties.

func (*Information) UnmarshalJSON 

func (i *Information) UnmarshalJSON(body []byte) error

UnmarshalJSON is the custom unmarshaler for Information struct.

type InnerError 

type InnerError struct {
	DiagnosticCode      *int32        `json:"diagnosticCode,omitempty"`
	Severity            SeverityTypes `json:"severity,omitempty"`
	Details             *string       `json:"details,omitempty"`
	Component           *string       `json:"component,omitempty"`
	ErrorID             *string       `json:"errorId,omitempty"`
	HelpLink            *string       `json:"helpLink,omitempty"`
	InternalDiagnostics *string       `json:"internalDiagnostics,omitempty"`
	Message             *string       `json:"message,omitempty"`
	Resolution          *string       `json:"resolution,omitempty"`
	Source              *string       `json:"source,omitempty"`
	Description         *string       `json:"description,omitempty"`
}

InnerError is the Data Lake Analytics job error details.

type ManagementClient 

type ManagementClient struct {
	autorest.Client
	AdlaJobDNSSuffix string
}

ManagementClient is the base client for Job.

func New 

func New() ManagementClient

New creates an instance of the ManagementClient client.

func NewWithoutDefaults 

func NewWithoutDefaults(adlaJobDNSSuffix string) ManagementClient

NewWithoutDefaults creates an instance of the ManagementClient client.

type Properties 

type Properties interface {
	AsUSQLJobProperties() (*USQLJobProperties, bool)
	AsHiveJobProperties() (*HiveJobProperties, bool)
}

Properties is the common Data Lake Analytics job properties.

type Resource 

type Resource struct {
	Name         *string      `json:"name,omitempty"`
	ResourcePath *string      `json:"resourcePath,omitempty"`
	Type         ResourceType `json:"type,omitempty"`
}

Resource is the Data Lake Analytics job resources.

type ResourceType 

type ResourceType string

ResourceType enumerates the values for resource type. 

const (
	// JobManagerResource specifies the job manager resource state for resource type.
	JobManagerResource ResourceType = "JobManagerResource"
	// JobManagerResourceInUserFolder specifies the job manager resource in user folder state for resource type.
	JobManagerResourceInUserFolder ResourceType = "JobManagerResourceInUserFolder"
	// StatisticsResource specifies the statistics resource state for resource type.
	StatisticsResource ResourceType = "StatisticsResource"
	// StatisticsResourceInUserFolder specifies the statistics resource in user folder state for resource type.
	StatisticsResourceInUserFolder ResourceType = "StatisticsResourceInUserFolder"
	// VertexResource specifies the vertex resource state for resource type.
	VertexResource ResourceType = "VertexResource"
	// VertexResourceInUserFolder specifies the vertex resource in user folder state for resource type.
	VertexResourceInUserFolder ResourceType = "VertexResourceInUserFolder"
)

type Result 

type Result string

Result enumerates the values for result. 

const (
	// Cancelled specifies the cancelled state for result.
	Cancelled Result = "Cancelled"
	// Failed specifies the failed state for result.
	Failed Result = "Failed"
	// None specifies the none state for result.
	None Result = "None"
	// Succeeded specifies the succeeded state for result.
	Succeeded Result = "Succeeded"
)

type SeverityTypes 

type SeverityTypes string

SeverityTypes enumerates the values for severity types. 

const (
	// Error specifies the error state for severity types.
	Error SeverityTypes = "Error"
	// Info specifies the info state for severity types.
	Info SeverityTypes = "Info"
	// Warning specifies the warning state for severity types.
	Warning SeverityTypes = "Warning"
)

type State 

type State string

State enumerates the values for state. 

const (
	// StateAccepted specifies the state accepted state for state.
	StateAccepted State = "Accepted"
	// StateCompiling specifies the state compiling state for state.
	StateCompiling State = "Compiling"
	// StateEnded specifies the state ended state for state.
	StateEnded State = "Ended"
	// StateNew specifies the state new state for state.
	StateNew State = "New"
	// StatePaused specifies the state paused state for state.
	StatePaused State = "Paused"
	// StateQueued specifies the state queued state for state.
	StateQueued State = "Queued"
	// StateRunning specifies the state running state for state.
	StateRunning State = "Running"
	// StateScheduling specifies the state scheduling state for state.
	StateScheduling State = "Scheduling"
	// StateStarting specifies the state starting state for state.
	StateStarting State = "Starting"
	// StateWaitingForCapacity specifies the state waiting for capacity state for state.
	StateWaitingForCapacity State = "WaitingForCapacity"
)

type StateAuditRecord 

type StateAuditRecord struct {
	NewState        *string    `json:"newState,omitempty"`
	TimeStamp       *date.Time `json:"timeStamp,omitempty"`
	RequestedByUser *string    `json:"requestedByUser,omitempty"`
	Details         *string    `json:"details,omitempty"`
}

StateAuditRecord is the Data Lake Analytics job state audit records for tracking the lifecycle of a job.

type Statistics 

type Statistics struct {
	autorest.Response `json:"-"`
	LastUpdateTimeUtc *date.Time               `json:"lastUpdateTimeUtc,omitempty"`
	FinalizingTimeUtc *date.Time               `json:"finalizingTimeUtc,omitempty"`
	Stages            *[]StatisticsVertexStage `json:"stages,omitempty"`
}

Statistics is the Data Lake Analytics job execution statistics.

type StatisticsVertexStage 

type StatisticsVertexStage struct {
	DataRead              *int64  `json:"dataRead,omitempty"`
	DataReadCrossPod      *int64  `json:"dataReadCrossPod,omitempty"`
	DataReadIntraPod      *int64  `json:"dataReadIntraPod,omitempty"`
	DataToRead            *int64  `json:"dataToRead,omitempty"`
	DataWritten           *int64  `json:"dataWritten,omitempty"`
	DuplicateDiscardCount *int32  `json:"duplicateDiscardCount,omitempty"`
	FailedCount           *int32  `json:"failedCount,omitempty"`
	MaxVertexDataRead     *int64  `json:"maxVertexDataRead,omitempty"`
	MinVertexDataRead     *int64  `json:"minVertexDataRead,omitempty"`
	ReadFailureCount      *int32  `json:"readFailureCount,omitempty"`
	RevocationCount       *int32  `json:"revocationCount,omitempty"`
	RunningCount          *int32  `json:"runningCount,omitempty"`
	ScheduledCount        *int32  `json:"scheduledCount,omitempty"`
	StageName             *string `json:"stageName,omitempty"`
	SucceededCount        *int32  `json:"succeededCount,omitempty"`
	TempDataWritten       *int64  `json:"tempDataWritten,omitempty"`
	TotalCount            *int32  `json:"totalCount,omitempty"`
	TotalFailedTime       *string `json:"totalFailedTime,omitempty"`
	TotalProgress         *int32  `json:"totalProgress,omitempty"`
	TotalSucceededTime    *string `json:"totalSucceededTime,omitempty"`
}

StatisticsVertexStage is the Data Lake Analytics job statistics vertex stage information.

type Type 

type Type string

Type enumerates the values for type. 

const (
	// Hive specifies the hive state for type.
	Hive Type = "Hive"
	// USQL specifies the usql state for type.
	USQL Type = "USql"
)

type USQLJobProperties 

type USQLJobProperties struct {
	RuntimeVersion           *string        `json:"runtimeVersion,omitempty"`
	Script                   *string        `json:"script,omitempty"`
	Type                     Type           `json:"type,omitempty"`
	Resources                *[]Resource    `json:"resources,omitempty"`
	Statistics               *Statistics    `json:"statistics,omitempty"`
	DebugData                *DataPath      `json:"debugData,omitempty"`
	Diagnostics              *[]Diagnostics `json:"diagnostics,omitempty"`
	AlgebraFilePath          *string        `json:"algebraFilePath,omitempty"`
	TotalCompilationTime     *string        `json:"totalCompilationTime,omitempty"`
	TotalPauseTime           *string        `json:"totalPauseTime,omitempty"`
	TotalQueuedTime          *string        `json:"totalQueuedTime,omitempty"`
	TotalRunningTime         *string        `json:"totalRunningTime,omitempty"`
	RootProcessNodeID        *string        `json:"rootProcessNodeId,omitempty"`
	YarnApplicationID        *string        `json:"yarnApplicationId,omitempty"`
	YarnApplicationTimeStamp *int64         `json:"yarnApplicationTimeStamp,omitempty"`
	CompileMode              CompileMode    `json:"compileMode,omitempty"`
}

USQLJobProperties is

func (USQLJobProperties) AsHiveJobProperties 

func (usjp USQLJobProperties) AsHiveJobProperties() (*HiveJobProperties, bool)

AsHiveJobProperties is the Properties implementation for USQLJobProperties.

func (USQLJobProperties) AsUSQLJobProperties 

func (usjp USQLJobProperties) AsUSQLJobProperties() (*USQLJobProperties, bool)

AsUSQLJobProperties is the Properties implementation for USQLJobProperties.

func (USQLJobProperties) MarshalJSON 

func (usjp USQLJobProperties) MarshalJSON() ([]byte, error)

MarshalJSON is the custom marshaler for USQLJobProperties.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.