emr

package
v0.16.2-dev Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 22, 2018 License: Apache-2.0 Imports: 2 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Cluster 

type Cluster struct {
	// contains filtered or unexported fields
}

Provides an Elastic MapReduce Cluster, a web service that makes it easy to process large amounts of data efficiently. See [Amazon Elastic MapReduce Documentation](https://aws.amazon.com/documentation/elastic-mapreduce/) for more information.

func GetCluster 

func GetCluster(ctx *pulumi.Context,
	name string, id pulumi.ID, state *ClusterState, opts ...pulumi.ResourceOpt) (*Cluster, error)

GetCluster gets an existing Cluster resource's state with the given name, ID, and optional state properties that are used to uniquely qualify the lookup (nil if not required).

func NewCluster 

func NewCluster(ctx *pulumi.Context,
	name string, args *ClusterArgs, opts ...pulumi.ResourceOpt) (*Cluster, error)

NewCluster registers a new resource with the given unique name, arguments, and options.

func (*Cluster) AdditionalInfo 

func (r *Cluster) AdditionalInfo() *pulumi.StringOutput

A JSON string for selecting additional features such as adding proxy information. Note: Currently there is no API to retrieve the value of this argument after EMR cluster creation from provider, therefore Terraform cannot detect drift from the actual EMR cluster if its value is changed outside Terraform.

func (*Cluster) Applications 

func (r *Cluster) Applications() *pulumi.ArrayOutput

A list of applications for the cluster. Valid values are: `Flink`, `Hadoop`, `Hive`, `Mahout`, `Pig`, `Spark`, and `JupyterHub` (as of EMR 5.14.0). Case insensitive

func (*Cluster) AutoscalingRole 

func (r *Cluster) AutoscalingRole() *pulumi.StringOutput

An IAM role for automatic scaling policies. The IAM role provides permissions that the automatic scaling feature requires to launch and terminate EC2 instances in an instance group.

func (*Cluster) BootstrapActions 

func (r *Cluster) BootstrapActions() *pulumi.ArrayOutput

List of bootstrap actions that will be run before Hadoop is started on the cluster nodes. Defined below

func (*Cluster) ClusterState 

func (r *Cluster) ClusterState() *pulumi.StringOutput

func (*Cluster) Configurations 

func (r *Cluster) Configurations() *pulumi.StringOutput

List of configurations supplied for the EMR cluster you are creating

func (*Cluster) ConfigurationsJson added in v0.15.1 

func (r *Cluster) ConfigurationsJson() *pulumi.StringOutput

A JSON string for supplying list of configurations for the EMR cluster.

func (*Cluster) CoreInstanceCount 

func (r *Cluster) CoreInstanceCount() *pulumi.IntOutput

Number of Amazon EC2 instances used to execute the job flow. EMR will use one node as the cluster's master node and use the remainder of the nodes (`core_instance_count`-1) as core nodes. Cannot be specified if `instance_groups` is set. Default `1`

func (*Cluster) CoreInstanceType 

func (r *Cluster) CoreInstanceType() *pulumi.StringOutput

The EC2 instance type of the slave nodes. Cannot be specified if `instance_groups` is set

func (*Cluster) CustomAmiId 

func (r *Cluster) CustomAmiId() *pulumi.StringOutput

A custom Amazon Linux AMI for the cluster (instead of an EMR-owned AMI). Available in Amazon EMR version 5.7.0 and later.

func (*Cluster) EbsRootVolumeSize 

func (r *Cluster) EbsRootVolumeSize() *pulumi.IntOutput

Size in GiB of the EBS root device volume of the Linux AMI that is used for each EC2 instance. Available in Amazon EMR version 4.x and later.

func (*Cluster) Ec2Attributes 

func (r *Cluster) Ec2Attributes() *pulumi.Output

Attributes for the EC2 instances running the job flow. Defined below

func (*Cluster) ID 

func (r *Cluster) ID() *pulumi.IDOutput

ID is this resource's unique identifier assigned by its provider.

func (*Cluster) InstanceGroups 

func (r *Cluster) InstanceGroups() *pulumi.ArrayOutput

A list of `instance_group` objects for each instance group in the cluster. Exactly one of `master_instance_type` and `instance_group` must be specified. If `instance_group` is set, then it must contain a configuration block for at least the `MASTER` instance group type (as well as any additional instance groups). Defined below

func (*Cluster) KeepJobFlowAliveWhenNoSteps 

func (r *Cluster) KeepJobFlowAliveWhenNoSteps() *pulumi.BoolOutput

Switch on/off run cluster with no steps or when all steps are complete (default is on)

func (*Cluster) KerberosAttributes 

func (r *Cluster) KerberosAttributes() *pulumi.Output

Kerberos configuration for the cluster. Defined below

func (*Cluster) LogUri 

func (r *Cluster) LogUri() *pulumi.StringOutput

S3 bucket to write the log files of the job flow. If a value is not provided, logs are not created

func (*Cluster) MasterInstanceType 

func (r *Cluster) MasterInstanceType() *pulumi.StringOutput

The EC2 instance type of the master node. Exactly one of `master_instance_type` and `instance_group` must be specified.

func (*Cluster) MasterPublicDns 

func (r *Cluster) MasterPublicDns() *pulumi.StringOutput

The public DNS name of the master EC2 instance.

func (*Cluster) Name 

func (r *Cluster) Name() *pulumi.StringOutput

The name of the job flow

func (*Cluster) ReleaseLabel 

func (r *Cluster) ReleaseLabel() *pulumi.StringOutput

The release label for the Amazon EMR release

func (*Cluster) ScaleDownBehavior 

func (r *Cluster) ScaleDownBehavior() *pulumi.StringOutput

The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs or an `instance group` is resized.

func (*Cluster) SecurityConfiguration 

func (r *Cluster) SecurityConfiguration() *pulumi.StringOutput

The security configuration name to attach to the EMR cluster. Only valid for EMR clusters with `release_label` 4.8.0 or greater

func (*Cluster) ServiceRole 

func (r *Cluster) ServiceRole() *pulumi.StringOutput

IAM role that will be assumed by the Amazon EMR service to access AWS resources

func (*Cluster) Steps 

func (r *Cluster) Steps() *pulumi.ArrayOutput

List of steps to run when creating the cluster. Defined below. It is highly recommended to utilize the [lifecycle configuration block](https://www.terraform.io/docs/configuration/resources.html) with `ignore_changes` if other steps are being managed outside of Terraform.

func (*Cluster) Tags 

func (r *Cluster) Tags() *pulumi.MapOutput

list of tags to apply to the EMR Cluster

func (*Cluster) TerminationProtection 

func (r *Cluster) TerminationProtection() *pulumi.BoolOutput

Switch on/off termination protection (default is off)

func (*Cluster) URN 

func (r *Cluster) URN() *pulumi.URNOutput

URN is this resource's unique name assigned by Pulumi.

func (*Cluster) VisibleToAllUsers 

func (r *Cluster) VisibleToAllUsers() *pulumi.BoolOutput

Whether the job flow is visible to all IAM users of the AWS account associated with the job flow. Default `true`

type ClusterArgs 

type ClusterArgs struct {
	// A JSON string for selecting additional features such as adding proxy information. Note: Currently there is no API to retrieve the value of this argument after EMR cluster creation from provider, therefore Terraform cannot detect drift from the actual EMR cluster if its value is changed outside Terraform.
	AdditionalInfo interface{}
	// A list of applications for the cluster. Valid values are: `Flink`, `Hadoop`, `Hive`, `Mahout`, `Pig`, `Spark`, and `JupyterHub` (as of EMR 5.14.0). Case insensitive
	Applications interface{}
	// An IAM role for automatic scaling policies. The IAM role provides permissions that the automatic scaling feature requires to launch and terminate EC2 instances in an instance group.
	AutoscalingRole interface{}
	// List of bootstrap actions that will be run before Hadoop is started on
	// the cluster nodes. Defined below
	BootstrapActions interface{}
	// List of configurations supplied for the EMR cluster you are creating
	Configurations interface{}
	// A JSON string for supplying list of configurations for the EMR cluster.
	ConfigurationsJson interface{}
	// Number of Amazon EC2 instances used to execute the job flow. EMR will use one node as the cluster's master node and use the remainder of the nodes (`core_instance_count`-1) as core nodes. Cannot be specified if `instance_groups` is set. Default `1`
	CoreInstanceCount interface{}
	// The EC2 instance type of the slave nodes. Cannot be specified if `instance_groups` is set
	CoreInstanceType interface{}
	// A custom Amazon Linux AMI for the cluster (instead of an EMR-owned AMI). Available in Amazon EMR version 5.7.0 and later.
	CustomAmiId interface{}
	// Size in GiB of the EBS root device volume of the Linux AMI that is used for each EC2 instance. Available in Amazon EMR version 4.x and later.
	EbsRootVolumeSize interface{}
	// Attributes for the EC2 instances running the job
	// flow. Defined below
	Ec2Attributes interface{}
	// A list of `instance_group` objects for each instance group in the cluster. Exactly one of `master_instance_type` and `instance_group` must be specified. If `instance_group` is set, then it must contain a configuration block for at least the `MASTER` instance group type (as well as any additional instance groups). Defined below
	InstanceGroups interface{}
	// Switch on/off run cluster with no steps or when all steps are complete (default is on)
	KeepJobFlowAliveWhenNoSteps interface{}
	// Kerberos configuration for the cluster. Defined below
	KerberosAttributes interface{}
	// S3 bucket to write the log files of the job flow. If a value
	// is not provided, logs are not created
	LogUri interface{}
	// The EC2 instance type of the master node. Exactly one of `master_instance_type` and `instance_group` must be specified.
	MasterInstanceType interface{}
	// The name of the job flow
	Name interface{}
	// The release label for the Amazon EMR release
	ReleaseLabel interface{}
	// The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs or an `instance group` is resized.
	ScaleDownBehavior interface{}
	// The security configuration name to attach to the EMR cluster. Only valid for EMR clusters with `release_label` 4.8.0 or greater
	SecurityConfiguration interface{}
	// IAM role that will be assumed by the Amazon EMR service to access AWS resources
	ServiceRole interface{}
	// List of steps to run when creating the cluster. Defined below. It is highly recommended to utilize the [lifecycle configuration block](https://www.terraform.io/docs/configuration/resources.html) with `ignore_changes` if other steps are being managed outside of Terraform.
	Steps interface{}
	// list of tags to apply to the EMR Cluster
	Tags interface{}
	// Switch on/off termination protection (default is off)
	TerminationProtection interface{}
	// Whether the job flow is visible to all IAM users of the AWS account associated with the job flow. Default `true`
	VisibleToAllUsers interface{}
}

The set of arguments for constructing a Cluster resource.

type ClusterState 

type ClusterState struct {
	// A JSON string for selecting additional features such as adding proxy information. Note: Currently there is no API to retrieve the value of this argument after EMR cluster creation from provider, therefore Terraform cannot detect drift from the actual EMR cluster if its value is changed outside Terraform.
	AdditionalInfo interface{}
	// A list of applications for the cluster. Valid values are: `Flink`, `Hadoop`, `Hive`, `Mahout`, `Pig`, `Spark`, and `JupyterHub` (as of EMR 5.14.0). Case insensitive
	Applications interface{}
	// An IAM role for automatic scaling policies. The IAM role provides permissions that the automatic scaling feature requires to launch and terminate EC2 instances in an instance group.
	AutoscalingRole interface{}
	// List of bootstrap actions that will be run before Hadoop is started on
	// the cluster nodes. Defined below
	BootstrapActions interface{}
	ClusterState     interface{}
	// List of configurations supplied for the EMR cluster you are creating
	Configurations interface{}
	// A JSON string for supplying list of configurations for the EMR cluster.
	ConfigurationsJson interface{}
	// Number of Amazon EC2 instances used to execute the job flow. EMR will use one node as the cluster's master node and use the remainder of the nodes (`core_instance_count`-1) as core nodes. Cannot be specified if `instance_groups` is set. Default `1`
	CoreInstanceCount interface{}
	// The EC2 instance type of the slave nodes. Cannot be specified if `instance_groups` is set
	CoreInstanceType interface{}
	// A custom Amazon Linux AMI for the cluster (instead of an EMR-owned AMI). Available in Amazon EMR version 5.7.0 and later.
	CustomAmiId interface{}
	// Size in GiB of the EBS root device volume of the Linux AMI that is used for each EC2 instance. Available in Amazon EMR version 4.x and later.
	EbsRootVolumeSize interface{}
	// Attributes for the EC2 instances running the job
	// flow. Defined below
	Ec2Attributes interface{}
	// A list of `instance_group` objects for each instance group in the cluster. Exactly one of `master_instance_type` and `instance_group` must be specified. If `instance_group` is set, then it must contain a configuration block for at least the `MASTER` instance group type (as well as any additional instance groups). Defined below
	InstanceGroups interface{}
	// Switch on/off run cluster with no steps or when all steps are complete (default is on)
	KeepJobFlowAliveWhenNoSteps interface{}
	// Kerberos configuration for the cluster. Defined below
	KerberosAttributes interface{}
	// S3 bucket to write the log files of the job flow. If a value
	// is not provided, logs are not created
	LogUri interface{}
	// The EC2 instance type of the master node. Exactly one of `master_instance_type` and `instance_group` must be specified.
	MasterInstanceType interface{}
	// The public DNS name of the master EC2 instance.
	MasterPublicDns interface{}
	// The name of the job flow
	Name interface{}
	// The release label for the Amazon EMR release
	ReleaseLabel interface{}
	// The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs or an `instance group` is resized.
	ScaleDownBehavior interface{}
	// The security configuration name to attach to the EMR cluster. Only valid for EMR clusters with `release_label` 4.8.0 or greater
	SecurityConfiguration interface{}
	// IAM role that will be assumed by the Amazon EMR service to access AWS resources
	ServiceRole interface{}
	// List of steps to run when creating the cluster. Defined below. It is highly recommended to utilize the [lifecycle configuration block](https://www.terraform.io/docs/configuration/resources.html) with `ignore_changes` if other steps are being managed outside of Terraform.
	Steps interface{}
	// list of tags to apply to the EMR Cluster
	Tags interface{}
	// Switch on/off termination protection (default is off)
	TerminationProtection interface{}
	// Whether the job flow is visible to all IAM users of the AWS account associated with the job flow. Default `true`
	VisibleToAllUsers interface{}
}

Input properties used for looking up and filtering Cluster resources.

type InstanceGroup 

type InstanceGroup struct {
	// contains filtered or unexported fields
}

Provides an Elastic MapReduce Cluster Instance Group configuration. See [Amazon Elastic MapReduce Documentation](https://aws.amazon.com/documentation/emr/) for more information.

~> **NOTE:** At this time, Instance Groups cannot be destroyed through the API nor web interface. Instance Groups are destroyed when the EMR Cluster is destroyed. Terraform will resize any Instance Group to zero when destroying the resource.

func GetInstanceGroup 

func GetInstanceGroup(ctx *pulumi.Context,
	name string, id pulumi.ID, state *InstanceGroupState, opts ...pulumi.ResourceOpt) (*InstanceGroup, error)

GetInstanceGroup gets an existing InstanceGroup resource's state with the given name, ID, and optional state properties that are used to uniquely qualify the lookup (nil if not required).

func NewInstanceGroup 

func NewInstanceGroup(ctx *pulumi.Context,
	name string, args *InstanceGroupArgs, opts ...pulumi.ResourceOpt) (*InstanceGroup, error)

NewInstanceGroup registers a new resource with the given unique name, arguments, and options.

func (*InstanceGroup) ClusterId 

func (r *InstanceGroup) ClusterId() *pulumi.StringOutput

ID of the EMR Cluster to attach to. Changing this forces a new resource to be created.

func (*InstanceGroup) EbsConfigs 

func (r *InstanceGroup) EbsConfigs() *pulumi.ArrayOutput

One or more `ebs_config` blocks as defined below. Changing this forces a new resource to be created.

func (*InstanceGroup) EbsOptimized 

func (r *InstanceGroup) EbsOptimized() *pulumi.BoolOutput

Indicates whether an Amazon EBS volume is EBS-optimized. Changing this forces a new resource to be created.

func (*InstanceGroup) ID 

func (r *InstanceGroup) ID() *pulumi.IDOutput

ID is this resource's unique identifier assigned by its provider.

func (*InstanceGroup) InstanceCount 

func (r *InstanceGroup) InstanceCount() *pulumi.IntOutput

Target number of instances for the instance group. Defaults to 0.

func (*InstanceGroup) InstanceType 

func (r *InstanceGroup) InstanceType() *pulumi.StringOutput

The EC2 instance type for all instances in the instance group. Changing this forces a new resource to be created.

func (*InstanceGroup) Name 

func (r *InstanceGroup) Name() *pulumi.StringOutput

Human friendly name given to the instance group. Changing this forces a new resource to be created.

func (*InstanceGroup) RunningInstanceCount 

func (r *InstanceGroup) RunningInstanceCount() *pulumi.IntOutput

func (*InstanceGroup) Status 

func (r *InstanceGroup) Status() *pulumi.StringOutput

func (*InstanceGroup) URN 

func (r *InstanceGroup) URN() *pulumi.URNOutput

URN is this resource's unique name assigned by Pulumi.

type InstanceGroupArgs 

type InstanceGroupArgs struct {
	// ID of the EMR Cluster to attach to. Changing this forces a new resource to be created.
	ClusterId interface{}
	// One or more `ebs_config` blocks as defined below. Changing this forces a new resource to be created.
	EbsConfigs interface{}
	// Indicates whether an Amazon EBS volume is EBS-optimized. Changing this forces a new resource to be created.
	EbsOptimized interface{}
	// Target number of instances for the instance group. Defaults to 0.
	InstanceCount interface{}
	// The EC2 instance type for all instances in the instance group. Changing this forces a new resource to be created.
	InstanceType interface{}
	// Human friendly name given to the instance group. Changing this forces a new resource to be created.
	Name interface{}
}

The set of arguments for constructing a InstanceGroup resource.

type InstanceGroupState 

type InstanceGroupState struct {
	// ID of the EMR Cluster to attach to. Changing this forces a new resource to be created.
	ClusterId interface{}
	// One or more `ebs_config` blocks as defined below. Changing this forces a new resource to be created.
	EbsConfigs interface{}
	// Indicates whether an Amazon EBS volume is EBS-optimized. Changing this forces a new resource to be created.
	EbsOptimized interface{}
	// Target number of instances for the instance group. Defaults to 0.
	InstanceCount interface{}
	// The EC2 instance type for all instances in the instance group. Changing this forces a new resource to be created.
	InstanceType interface{}
	// Human friendly name given to the instance group. Changing this forces a new resource to be created.
	Name                 interface{}
	RunningInstanceCount interface{}
	Status               interface{}
}

Input properties used for looking up and filtering InstanceGroup resources.

type SecurityConfiguration 

type SecurityConfiguration struct {
	// contains filtered or unexported fields
}

Provides a resource to manage AWS EMR Security Configurations

func GetSecurityConfiguration 

func GetSecurityConfiguration(ctx *pulumi.Context,
	name string, id pulumi.ID, state *SecurityConfigurationState, opts ...pulumi.ResourceOpt) (*SecurityConfiguration, error)

GetSecurityConfiguration gets an existing SecurityConfiguration resource's state with the given name, ID, and optional state properties that are used to uniquely qualify the lookup (nil if not required).

func NewSecurityConfiguration 

func NewSecurityConfiguration(ctx *pulumi.Context,
	name string, args *SecurityConfigurationArgs, opts ...pulumi.ResourceOpt) (*SecurityConfiguration, error)

NewSecurityConfiguration registers a new resource with the given unique name, arguments, and options.

func (*SecurityConfiguration) Configuration 

func (r *SecurityConfiguration) Configuration() *pulumi.StringOutput

A JSON formatted Security Configuration

func (*SecurityConfiguration) CreationDate 

func (r *SecurityConfiguration) CreationDate() *pulumi.StringOutput

Date the Security Configuration was created

func (*SecurityConfiguration) ID 

func (r *SecurityConfiguration) ID() *pulumi.IDOutput

ID is this resource's unique identifier assigned by its provider.

func (*SecurityConfiguration) Name 

func (r *SecurityConfiguration) Name() *pulumi.StringOutput

The name of the EMR Security Configuration. By default generated by Terraform.

func (*SecurityConfiguration) NamePrefix 

func (r *SecurityConfiguration) NamePrefix() *pulumi.StringOutput

Creates a unique name beginning with the specified prefix. Conflicts with `name`.

func (*SecurityConfiguration) URN 

func (r *SecurityConfiguration) URN() *pulumi.URNOutput

URN is this resource's unique name assigned by Pulumi.

type SecurityConfigurationArgs 

type SecurityConfigurationArgs struct {
	// A JSON formatted Security Configuration
	Configuration interface{}
	// The name of the EMR Security Configuration. By default generated by Terraform.
	Name interface{}
	// Creates a unique name beginning with the specified
	// prefix. Conflicts with `name`.
	NamePrefix interface{}
}

The set of arguments for constructing a SecurityConfiguration resource.

type SecurityConfigurationState 

type SecurityConfigurationState struct {
	// A JSON formatted Security Configuration
	Configuration interface{}
	// Date the Security Configuration was created
	CreationDate interface{}
	// The name of the EMR Security Configuration. By default generated by Terraform.
	Name interface{}
	// Creates a unique name beginning with the specified
	// prefix. Conflicts with `name`.
	NamePrefix interface{}
}

Input properties used for looking up and filtering SecurityConfiguration resources.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.