awscdkawsgluealpha

package module
v2.0.0-rc.24 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 13, 2021 License: Apache-2.0 Imports: 14 Imported by: 0

README

AWS Glue Construct Library

All classes with the Cfn prefix in this module (CFN Resources) are always stable and safe to use.

The APIs of higher level constructs in this module are experimental and under active development. They are subject to non-backward compatible changes or removal in any future version. These are not subject to the Semantic Versioning model and breaking changes will be announced in the release notes. This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package.

This module is part of the AWS Cloud Development Kit project.

Job

A Job encapsulates a script that connects to data sources, processes them, and then writes output to a data target.

There are 3 types of jobs supported by AWS Glue: Spark ETL, Spark Streaming, and Python Shell jobs.

The glue.JobExecutable allows you to specify the type of job, the language to use and the code assets required by the job.

glue.Code allows you to refer to the different code assets required by the job, either from an existing S3 location or from a local file path.

Spark Jobs

These jobs run in an Apache Spark environment managed by AWS Glue.

ETL Jobs

An ETL job processes data in batches using Apache Spark.

new glue.Job(stack, 'ScalaSparkEtlJob', {
  executable: glue.JobExecutable.scalaEtl({
    glueVersion: glue.GlueVersion.V2_0,
    script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),
    className: 'com.example.HelloWorld',
    extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],
  }),
  description: 'an example Scala ETL job',
});
Streaming Jobs

A Streaming job is similar to an ETL job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.

new glue.Job(stack, 'PythonSparkStreamingJob', {
  executable: glue.JobExecutable.pythonStreaming({
    glueVersion: glue.GlueVersion.V2_0,
    pythonVersion: glue.PythonVersion.THREE,
    script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),
  }),
  description: 'an example Python Streaming job',
});
Python Shell Jobs

A Python shell job runs Python scripts as a shell and supports a Python version that depends on the AWS Glue version you are using. This can be used to schedule and run tasks that don't require an Apache Spark environment.

new glue.Job(stack, 'PythonShellJob', {
  executable: glue.JobExecutable.pythonShell({
    glueVersion: glue.GlueVersion.V1_0,
    pythonVersion: PythonVersion.THREE,
    script: glue.Code.fromBucket(bucket, 'script.py'),
  }),
  description: 'an example Python Shell job',
});

See documentation for more information on adding jobs in Glue.

Connection

A Connection allows Glue jobs, crawlers and development endpoints to access certain types of data stores. For example, to create a network connection to connect to a data source within a VPC:

new glue.Connection(stack, 'MyConnection', {
  connectionType: glue.ConnectionTypes.NETWORK,
  // The security groups granting AWS Glue inbound access to the data source within the VPC
  securityGroups: [securityGroup],
  // The VPC subnet which contains the data source
  subnet,
});

If you need to use a connection type that doesn't exist as a static member on ConnectionType, you can instantiate a ConnectionType object, e.g: new glue.ConnectionType('NEW_TYPE').

See Adding a Connection to Your Data Store and Connection Structure documentation for more information on the supported data stores and their configurations.

SecurityConfiguration

A SecurityConfiguration is a set of security properties that can be used by AWS Glue to encrypt data at rest.

new glue.SecurityConfiguration(stack, 'MySecurityConfiguration', {
  securityConfigurationName: 'name',
  cloudWatchEncryption: {
    mode: glue.CloudWatchEncryptionMode.KMS,
  },
  jobBookmarksEncryption: {
    mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,
  },
  s3Encryption: {
    mode: glue.S3EncryptionMode.KMS,
  },
});

By default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:

new glue.SecurityConfiguration(stack, 'MySecurityConfiguration', {
  securityConfigurationName: 'name',
  cloudWatchEncryption: {
    mode: glue.CloudWatchEncryptionMode.KMS,
    kmsKey: key,
  },
});

See documentation for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.

Database

A Database is a logical grouping of Tables in the Glue Catalog.

new glue.Database(stack, 'MyDatabase', {
  databaseName: 'my_database'
});

Table

A Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):

new glue.Table(stack, 'MyTable', {
  database: myDatabase,
  tableName: 'my_table',
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING,
  }, {
    name: 'col2',
    type: glue.Schema.array(Schema.STRING),
    comment: 'col2 is an array of strings' // comment is optional
  }],
  dataFormat: glue.DataFormat.JSON
});

By default, a S3 bucket will be created to store the table's data but you can manually pass the bucket and s3Prefix:

new glue.Table(stack, 'MyTable', {
  bucket: myBucket,
  s3Prefix: 'my-table/'
  ...
});

By default, an S3 bucket will be created to store the table's data and stored in the bucket root. You can also manually pass the bucket and s3Prefix:

Partitions

To improve query performance, a table can specify partitionKeys on which data is stored and queried separately. For example, you might partition a table by year and month to optimize queries based on a time window:

new glue.Table(stack, 'MyTable', {
  database: myDatabase,
  tableName: 'my_table',
  columns: [{
    name: 'col1',
    type: glue.Schema.STRING
  }],
  partitionKeys: [{
    name: 'year',
    type: glue.Schema.SMALL_INT
  }, {
    name: 'month',
    type: glue.Schema.SMALL_INT
  }],
  dataFormat: glue.DataFormat.JSON
});

Encryption

You can enable encryption on a Table's data:

  • Unencrypted - files are not encrypted. The default encryption setting.
  • S3Managed - Server side encryption (SSE-S3) with an Amazon S3-managed key.
new glue.Table(stack, 'MyTable', {
  encryption: glue.TableEncryption.S3_MANAGED
  ...
});
  • Kms - Server-side encryption (SSE-KMS) with an AWS KMS Key managed by the account owner.
// KMS key is created automatically
new glue.Table(stack, 'MyTable', {
  encryption: glue.TableEncryption.KMS
  ...
});

// with an explicit KMS key
new glue.Table(stack, 'MyTable', {
  encryption: glue.TableEncryption.KMS,
  encryptionKey: new kms.Key(stack, 'MyKey')
  ...
});
  • KmsManaged - Server-side encryption (SSE-KMS), like Kms, except with an AWS KMS Key managed by the AWS Key Management Service.
new glue.Table(stack, 'MyTable', {
  encryption: glue.TableEncryption.KMS_MANAGED
  ...
});
  • ClientSideKms - Client-side encryption (CSE-KMS) with an AWS KMS Key managed by the account owner.
// KMS key is created automatically
new glue.Table(stack, 'MyTable', {
  encryption: glue.TableEncryption.CLIENT_SIDE_KMS
  ...
});

// with an explicit KMS key
new glue.Table(stack, 'MyTable', {
  encryption: glue.TableEncryption.CLIENT_SIDE_KMS,
  encryptionKey: new kms.Key(stack, 'MyKey')
  ...
});

Note: you cannot provide a Bucket when creating the Table if you wish to use server-side encryption (KMS, KMS_MANAGED or S3_MANAGED).

Types

A table's schema is a collection of columns, each of which have a name and a type. Types are recursive structures, consisting of primitive and complex types:

new glue.Table(stack, 'MyTable', {
  columns: [{
    name: 'primitive_column',
    type: glue.Schema.STRING
  }, {
    name: 'array_column',
    type: glue.Schema.array(glue.Schema.INTEGER),
    comment: 'array<integer>'
  }, {
    name: 'map_column',
    type: glue.Schema.map(
      glue.Schema.STRING,
      glue.Schema.TIMESTAMP),
    comment: 'map<string,string>'
  }, {
    name: 'struct_column',
    type: glue.Schema.struct([{
      name: 'nested_column',
      type: glue.Schema.DATE,
      comment: 'nested comment'
    }]),
    comment: "struct<nested_column:date COMMENT 'nested comment'>"
  }],
  ...
Primitives
Numeric
Name Type Comments
FLOAT Constant A 32-bit single-precision floating point number
INTEGER Constant A 32-bit signed value in two's complement format, with a minimum value of -2^31 and a maximum value of 2^31-1
DOUBLE Constant A 64-bit double-precision floating point number
BIG_INT Constant A 64-bit signed INTEGER in two’s complement format, with a minimum value of -2^63 and a maximum value of 2^63 -1
SMALL_INT Constant A 16-bit signed INTEGER in two’s complement format, with a minimum value of -2^15 and a maximum value of 2^15-1
TINY_INT Constant A 8-bit signed INTEGER in two’s complement format, with a minimum value of -2^7 and a maximum value of 2^7-1
Date and time
Name Type Comments
DATE Constant A date in UNIX format, such as YYYY-MM-DD.
TIMESTAMP Constant Date and time instant in the UNiX format, such as yyyy-mm-dd hh:mm:ss[.f...]. For example, TIMESTAMP '2008-09-15 03:04:05.324'. This format uses the session time zone.
String
Name Type Comments
STRING Constant A string literal enclosed in single or double quotes
decimal(precision: number, scale?: number) Function precision is the total number of digits. scale (optional) is the number of digits in fractional part with a default of 0. For example, use these type definitions: decimal(11,5), decimal(15)
char(length: number) Function Fixed length character data, with a specified length between 1 and 255, such as char(10)
varchar(length: number) Function Variable length character data, with a specified length between 1 and 65535, such as varchar(10)
Miscellaneous
Name Type Comments
BOOLEAN Constant Values are true and false
BINARY Constant Value is in binary
Complex
Name Type Comments
array(itemType: Type) Function An array of some other type
map(keyType: Type, valueType: Type) Function A map of some primitive key type to any value type
struct(collumns: Column[]) Function Nested structure containing individually named and typed collumns

Documentation

Overview

The CDK Construct Library for AWS::Glue

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Connection_IsConstruct 

func Connection_IsConstruct(x interface{}) *bool

Checks if `x` is a construct.

Returns: true if `x` is an object created from a class which extends `Construct`. Deprecated: use `x instanceof Construct` instead

func Connection_IsResource 

func Connection_IsResource(construct constructs.IConstruct) *bool

Check whether the given construct is a Resource. Experimental.

func Database_IsConstruct 

func Database_IsConstruct(x interface{}) *bool

Checks if `x` is a construct.

Returns: true if `x` is an object created from a class which extends `Construct`. Deprecated: use `x instanceof Construct` instead

func Database_IsResource 

func Database_IsResource(construct constructs.IConstruct) *bool

Check whether the given construct is a Resource. Experimental.

func Job_IsConstruct 

func Job_IsConstruct(x interface{}) *bool

Checks if `x` is a construct.

Returns: true if `x` is an object created from a class which extends `Construct`. Deprecated: use `x instanceof Construct` instead

func Job_IsResource 

func Job_IsResource(construct constructs.IConstruct) *bool

Check whether the given construct is a Resource. Experimental.

func NewAssetCode_Override 

func NewAssetCode_Override(a AssetCode, path *string, options *awss3assets.AssetOptions)

Experimental.

func NewClassificationString_Override 

func NewClassificationString_Override(c ClassificationString, value *string)

Experimental.

func NewCode_Override 

func NewCode_Override(c Code)

Experimental.

func NewConnectionType_Override 

func NewConnectionType_Override(c ConnectionType, name *string)

Experimental.

func NewConnection_Override 

func NewConnection_Override(c Connection, scope constructs.Construct, id *string, props *ConnectionProps)

Experimental.

func NewDataFormat_Override 

func NewDataFormat_Override(d DataFormat, props *DataFormatProps)

Experimental.

func NewDatabase_Override 

func NewDatabase_Override(d Database, scope constructs.Construct, id *string, props *DatabaseProps)

Experimental.

func NewInputFormat_Override 

func NewInputFormat_Override(i InputFormat, className *string)

Experimental.

func NewJob_Override 

func NewJob_Override(j Job, scope constructs.Construct, id *string, props *JobProps)

Experimental.

func NewOutputFormat_Override 

func NewOutputFormat_Override(o OutputFormat, className *string)

Experimental.

func NewS3Code_Override 

func NewS3Code_Override(s S3Code, bucket awss3.IBucket, key *string)

Experimental.

func NewSchema_Override 

func NewSchema_Override(s Schema)

Experimental.

func NewSecurityConfiguration_Override 

func NewSecurityConfiguration_Override(s SecurityConfiguration, scope constructs.Construct, id *string, props *SecurityConfigurationProps)

Experimental.

func NewSerializationLibrary_Override 

func NewSerializationLibrary_Override(s SerializationLibrary, className *string)

Experimental.

func NewTable_Override 

func NewTable_Override(t Table, scope constructs.Construct, id *string, props *TableProps)

Experimental.

func SecurityConfiguration_IsConstruct 

func SecurityConfiguration_IsConstruct(x interface{}) *bool

Checks if `x` is a construct.

Returns: true if `x` is an object created from a class which extends `Construct`. Deprecated: use `x instanceof Construct` instead

func SecurityConfiguration_IsResource 

func SecurityConfiguration_IsResource(construct constructs.IConstruct) *bool

Check whether the given construct is a Resource. Experimental.

func Table_IsConstruct 

func Table_IsConstruct(x interface{}) *bool

Checks if `x` is a construct.

Returns: true if `x` is an object created from a class which extends `Construct`. Deprecated: use `x instanceof Construct` instead

func Table_IsResource 

func Table_IsResource(construct constructs.IConstruct) *bool

Check whether the given construct is a Resource. Experimental.

Types

type AssetCode 

type AssetCode interface {
	Code
	Bind(scope constructs.Construct, grantable awsiam.IGrantable) *CodeConfig
}

Job Code from a local file. Experimental.

func AssetCode_FromAsset 

func AssetCode_FromAsset(path *string, options *awss3assets.AssetOptions) AssetCode

Job code from a local disk path. Experimental.

func Code_FromAsset 

func Code_FromAsset(path *string, options *awss3assets.AssetOptions) AssetCode

Job code from a local disk path. Experimental.

func NewAssetCode 

func NewAssetCode(path *string, options *awss3assets.AssetOptions) AssetCode

Experimental.

func S3Code_FromAsset 

func S3Code_FromAsset(path *string, options *awss3assets.AssetOptions) AssetCode

Job code from a local disk path. Experimental.

type ClassificationString 

type ClassificationString interface {
	Value() *string
}

Classification string given to tables with this data format. See: https://docs.aws.amazon.com/glue/latest/dg/add-classifier.html#classifier-built-in

Experimental.

func ClassificationString_AVRO 

func ClassificationString_AVRO() ClassificationString

func ClassificationString_CSV 

func ClassificationString_CSV() ClassificationString

func ClassificationString_JSON 

func ClassificationString_JSON() ClassificationString

func ClassificationString_ORC 

func ClassificationString_ORC() ClassificationString

func ClassificationString_PARQUET 

func ClassificationString_PARQUET() ClassificationString

func ClassificationString_XML 

func ClassificationString_XML() ClassificationString

func NewClassificationString 

func NewClassificationString(value *string) ClassificationString

Experimental.

type CloudWatchEncryption 

type CloudWatchEncryption struct {
	// Encryption mode.
	// Experimental.
	Mode CloudWatchEncryptionMode `json:"mode"`
	// The KMS key to be used to encrypt the data.
	// Experimental.
	KmsKey awskms.IKey `json:"kmsKey"`
}

CloudWatch Logs encryption configuration. Experimental.

type CloudWatchEncryptionMode 

type CloudWatchEncryptionMode string

Encryption mode for CloudWatch Logs. See: https://docs.aws.amazon.com/glue/latest/webapi/API_CloudWatchEncryption.html#Glue-Type-CloudWatchEncryption-CloudWatchEncryptionMode

Experimental. 

const (
	CloudWatchEncryptionMode_KMS CloudWatchEncryptionMode = "KMS"
)

type Code 

type Code interface {
	Bind(scope constructs.Construct, grantable awsiam.IGrantable) *CodeConfig
}

Represents a Glue Job's Code assets (an asset can be a scripts, a jar, a python file or any other file). Experimental.

type CodeConfig 

type CodeConfig struct {
	// The location of the code in S3.
	// Experimental.
	S3Location *awss3.Location `json:"s3Location"`
}

Result of binding `Code` into a `Job`. Experimental.

type Column 

type Column struct {
	// Name of the column.
	// Experimental.
	Name *string `json:"name"`
	// Type of the column.
	// Experimental.
	Type *Type `json:"type"`
	// Coment describing the column.
	// Experimental.
	Comment *string `json:"comment"`
}

A column of a table. Experimental.

type Connection 

type Connection interface {
	awscdk.Resource
	IConnection
	ConnectionArn() *string
	ConnectionName() *string
	Env() *awscdk.ResourceEnvironment
	Node() constructs.Node
	PhysicalName() *string
	Stack() awscdk.Stack
	AddProperty(key *string, value *string)
	ApplyRemovalPolicy(policy awscdk.RemovalPolicy)
	GeneratePhysicalName() *string
	GetResourceArnAttribute(arnAttr *string, arnComponents *awscdk.ArnComponents) *string
	GetResourceNameAttribute(nameAttr *string) *string
	ToString() *string
}

An AWS Glue connection to a data source. Experimental.

func NewConnection 

func NewConnection(scope constructs.Construct, id *string, props *ConnectionProps) Connection

Experimental.

type ConnectionOptions 

type ConnectionOptions struct {
	// The name of the connection.
	// Experimental.
	ConnectionName *string `json:"connectionName"`
	// The description of the connection.
	// Experimental.
	Description *string `json:"description"`
	// A list of criteria that can be used in selecting this connection.
	//
	// This is useful for filtering the results of https://awscli.amazonaws.com/v2/documentation/api/latest/reference/glue/get-connections.html
	// Experimental.
	MatchCriteria *[]*string `json:"matchCriteria"`
	// Key-Value pairs that define parameters for the connection.
	// See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect.html
	//
	// Experimental.
	Properties *map[string]*string `json:"properties"`
	// The list of security groups needed to successfully make this connection e.g. to successfully connect to VPC.
	// Experimental.
	SecurityGroups *[]awsec2.ISecurityGroup `json:"securityGroups"`
	// The VPC subnet to connect to resources within a VPC.
	//
	// See more at https://docs.aws.amazon.com/glue/latest/dg/start-connecting.html.
	// Experimental.
	Subnet awsec2.ISubnet `json:"subnet"`
}

Base Connection Options. Experimental.

type ConnectionProps 

type ConnectionProps struct {
	// The name of the connection.
	// Experimental.
	ConnectionName *string `json:"connectionName"`
	// The description of the connection.
	// Experimental.
	Description *string `json:"description"`
	// A list of criteria that can be used in selecting this connection.
	//
	// This is useful for filtering the results of https://awscli.amazonaws.com/v2/documentation/api/latest/reference/glue/get-connections.html
	// Experimental.
	MatchCriteria *[]*string `json:"matchCriteria"`
	// Key-Value pairs that define parameters for the connection.
	// See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect.html
	//
	// Experimental.
	Properties *map[string]*string `json:"properties"`
	// The list of security groups needed to successfully make this connection e.g. to successfully connect to VPC.
	// Experimental.
	SecurityGroups *[]awsec2.ISecurityGroup `json:"securityGroups"`
	// The VPC subnet to connect to resources within a VPC.
	//
	// See more at https://docs.aws.amazon.com/glue/latest/dg/start-connecting.html.
	// Experimental.
	Subnet awsec2.ISubnet `json:"subnet"`
	// The type of the connection.
	// Experimental.
	Type ConnectionType `json:"type"`
}

Construction properties for {@link Connection}. Experimental.

type ConnectionType 

type ConnectionType interface {
	Name() *string
	ToString() *string
}

The type of the glue connection.

If you need to use a connection type that doesn't exist as a static member, you can instantiate a `ConnectionType` object, e.g: `new ConnectionType('NEW_TYPE')`. Experimental.

func ConnectionType_JDBC 

func ConnectionType_JDBC() ConnectionType

func ConnectionType_KAFKA 

func ConnectionType_KAFKA() ConnectionType

func ConnectionType_MONGODB 

func ConnectionType_MONGODB() ConnectionType

func ConnectionType_NETWORK 

func ConnectionType_NETWORK() ConnectionType

func NewConnectionType 

func NewConnectionType(name *string) ConnectionType

Experimental.

type ContinuousLoggingProps 

type ContinuousLoggingProps struct {
	// Enable continouous logging.
	// Experimental.
	Enabled *bool `json:"enabled"`
	// Apply the provided conversion pattern.
	//
	// This is a Log4j Conversion Pattern to customize driver and executor logs.
	// Experimental.
	ConversionPattern *string `json:"conversionPattern"`
	// Specify a custom CloudWatch log group name.
	// Experimental.
	LogGroup awslogs.ILogGroup `json:"logGroup"`
	// Specify a custom CloudWatch log stream prefix.
	// Experimental.
	LogStreamPrefix *string `json:"logStreamPrefix"`
	// Filter out non-useful Apache Spark driver/executor and Apache Hadoop YARN heartbeat log messages.
	// Experimental.
	Quiet *bool `json:"quiet"`
}

Properties for enabling Continuous Logging for Glue Jobs. See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Experimental.

type DataFormat 

type DataFormat interface {
	ClassificationString() ClassificationString
	InputFormat() InputFormat
	OutputFormat() OutputFormat
	SerializationLibrary() SerializationLibrary
}

Defines the input/output formats and ser/de for a single DataFormat. Experimental.

func DataFormat_APACHE_LOGS 

func DataFormat_APACHE_LOGS() DataFormat

func DataFormat_AVRO 

func DataFormat_AVRO() DataFormat

func DataFormat_CLOUDTRAIL_LOGS 

func DataFormat_CLOUDTRAIL_LOGS() DataFormat

func DataFormat_CSV 

func DataFormat_CSV() DataFormat

func DataFormat_JSON 

func DataFormat_JSON() DataFormat

func DataFormat_LOGSTASH 

func DataFormat_LOGSTASH() DataFormat

func DataFormat_ORC 

func DataFormat_ORC() DataFormat

func DataFormat_PARQUET 

func DataFormat_PARQUET() DataFormat

func DataFormat_TSV 

func DataFormat_TSV() DataFormat

func NewDataFormat 

func NewDataFormat(props *DataFormatProps) DataFormat

Experimental.

type DataFormatProps 

type DataFormatProps struct {
	// `InputFormat` for this data format.
	// Experimental.
	InputFormat InputFormat `json:"inputFormat"`
	// `OutputFormat` for this data format.
	// Experimental.
	OutputFormat OutputFormat `json:"outputFormat"`
	// Serialization library for this data format.
	// Experimental.
	SerializationLibrary SerializationLibrary `json:"serializationLibrary"`
	// Classification string given to tables with this data format.
	// Experimental.
	ClassificationString ClassificationString `json:"classificationString"`
}

Properties of a DataFormat instance. Experimental.

type Database 

type Database interface {
	awscdk.Resource
	IDatabase
	CatalogArn() *string
	CatalogId() *string
	DatabaseArn() *string
	DatabaseName() *string
	Env() *awscdk.ResourceEnvironment
	LocationUri() *string
	Node() constructs.Node
	PhysicalName() *string
	Stack() awscdk.Stack
	ApplyRemovalPolicy(policy awscdk.RemovalPolicy)
	GeneratePhysicalName() *string
	GetResourceArnAttribute(arnAttr *string, arnComponents *awscdk.ArnComponents) *string
	GetResourceNameAttribute(nameAttr *string) *string
	ToString() *string
}

A Glue database. Experimental.

func NewDatabase 

func NewDatabase(scope constructs.Construct, id *string, props *DatabaseProps) Database

Experimental.

type DatabaseProps 

type DatabaseProps struct {
	// The name of the database.
	// Experimental.
	DatabaseName *string `json:"databaseName"`
	// The location of the database (for example, an HDFS path).
	// See: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-database-databaseinput.html
	//
	// Experimental.
	LocationUri *string `json:"locationUri"`
}

Experimental.

type GlueVersion 

type GlueVersion interface {
	Name() *string
}

AWS Glue version determines the versions of Apache Spark and Python that are available to the job. See: https://docs.aws.amazon.com/glue/latest/dg/add-job.html.

If you need to use a GlueVersion that doesn't exist as a static member, you can instantiate a `GlueVersion` object, e.g: `GlueVersion.of('1.5')`.

Experimental.

func GlueVersion_Of 

func GlueVersion_Of(version *string) GlueVersion

Custom Glue version. Experimental.

func GlueVersion_V0_9 

func GlueVersion_V0_9() GlueVersion

func GlueVersion_V1_0 

func GlueVersion_V1_0() GlueVersion

func GlueVersion_V2_0 

func GlueVersion_V2_0() GlueVersion

func GlueVersion_V3_0 

func GlueVersion_V3_0() GlueVersion

type IConnection 

type IConnection interface {
	awscdk.IResource
	// The ARN of the connection.
	// Experimental.
	ConnectionArn() *string
	// The name of the connection.
	// Experimental.
	ConnectionName() *string
}

Interface representing a created or an imported {@link Connection}. Experimental.

func Connection_FromConnectionArn 

func Connection_FromConnectionArn(scope constructs.Construct, id *string, connectionArn *string) IConnection

Creates a Connection construct that represents an external connection. Experimental.

func Connection_FromConnectionName 

func Connection_FromConnectionName(scope constructs.Construct, id *string, connectionName *string) IConnection

Creates a Connection construct that represents an external connection. Experimental.

type IDatabase 

type IDatabase interface {
	awscdk.IResource
	// The ARN of the catalog.
	// Experimental.
	CatalogArn() *string
	// The catalog id of the database (usually, the AWS account id).
	// Experimental.
	CatalogId() *string
	// The ARN of the database.
	// Experimental.
	DatabaseArn() *string
	// The name of the database.
	// Experimental.
	DatabaseName() *string
}

Experimental.

func Database_FromDatabaseArn 

func Database_FromDatabaseArn(scope constructs.Construct, id *string, databaseArn *string) IDatabase

Experimental.

type IJob 

type IJob interface {
	awsiam.IGrantable
	awscdk.IResource
	// Create a CloudWatch metric.
	// See: https://docs.aws.amazon.com/glue/latest/dg/monitoring-awsglue-with-cloudwatch-metrics.html
	//
	// Experimental.
	Metric(metricName *string, type_ MetricType, props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Create a CloudWatch Metric indicating job failure.
	// Experimental.
	MetricFailure(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Create a CloudWatch Metric indicating job success.
	// Experimental.
	MetricSuccess(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Create a CloudWatch Metric indicating job timeout.
	// Experimental.
	MetricTimeout(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Defines a CloudWatch event rule triggered when something happens with this job.
	// See: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types
	//
	// Experimental.
	OnEvent(id *string, options *awsevents.OnEventOptions) awsevents.Rule
	// Defines a CloudWatch event rule triggered when this job moves to the FAILED state.
	// See: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types
	//
	// Experimental.
	OnFailure(id *string, options *awsevents.OnEventOptions) awsevents.Rule
	// Defines a CloudWatch event rule triggered when this job moves to the input jobState.
	// See: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types
	//
	// Experimental.
	OnStateChange(id *string, jobState JobState, options *awsevents.OnEventOptions) awsevents.Rule
	// Defines a CloudWatch event rule triggered when this job moves to the SUCCEEDED state.
	// See: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types
	//
	// Experimental.
	OnSuccess(id *string, options *awsevents.OnEventOptions) awsevents.Rule
	// Defines a CloudWatch event rule triggered when this job moves to the TIMEOUT state.
	// See: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types
	//
	// Experimental.
	OnTimeout(id *string, options *awsevents.OnEventOptions) awsevents.Rule
	// The ARN of the job.
	// Experimental.
	JobArn() *string
	// The name of the job.
	// Experimental.
	JobName() *string
}

Interface representing a created or an imported {@link Job}. Experimental.

func Job_FromJobAttributes 

func Job_FromJobAttributes(scope constructs.Construct, id *string, attrs *JobAttributes) IJob

Creates a Glue Job. Experimental.

type ISecurityConfiguration 

type ISecurityConfiguration interface {
	awscdk.IResource
	// The name of the security configuration.
	// Experimental.
	SecurityConfigurationName() *string
}

Interface representing a created or an imported {@link SecurityConfiguration}. Experimental.

func SecurityConfiguration_FromSecurityConfigurationName 

func SecurityConfiguration_FromSecurityConfigurationName(scope constructs.Construct, id *string, securityConfigurationName *string) ISecurityConfiguration

Creates a Connection construct that represents an external security configuration. Experimental.

type ITable 

type ITable interface {
	awscdk.IResource
	// Experimental.
	TableArn() *string
	// Experimental.
	TableName() *string
}

Experimental.

func Table_FromTableArn 

func Table_FromTableArn(scope constructs.Construct, id *string, tableArn *string) ITable

Experimental.

func Table_FromTableAttributes 

func Table_FromTableAttributes(scope constructs.Construct, id *string, attrs *TableAttributes) ITable

Creates a Table construct that represents an external table. Experimental.

type InputFormat 

type InputFormat interface {
	ClassName() *string
}

Absolute class name of the Hadoop `InputFormat` to use when reading table files. Experimental.

func InputFormat_AVRO 

func InputFormat_AVRO() InputFormat

func InputFormat_CLOUDTRAIL 

func InputFormat_CLOUDTRAIL() InputFormat

func InputFormat_ORC 

func InputFormat_ORC() InputFormat

func InputFormat_PARQUET 

func InputFormat_PARQUET() InputFormat

func InputFormat_TEXT 

func InputFormat_TEXT() InputFormat

func NewInputFormat 

func NewInputFormat(className *string) InputFormat

Experimental.

func OutputFormat_AVRO 

func OutputFormat_AVRO() InputFormat

func OutputFormat_ORC 

func OutputFormat_ORC() InputFormat

type Job 

type Job interface {
	awscdk.Resource
	IJob
	Env() *awscdk.ResourceEnvironment
	GrantPrincipal() awsiam.IPrincipal
	JobArn() *string
	JobName() *string
	Node() constructs.Node
	PhysicalName() *string
	Role() awsiam.IRole
	SparkUILoggingLocation() *SparkUILoggingLocation
	Stack() awscdk.Stack
	ApplyRemovalPolicy(policy awscdk.RemovalPolicy)
	GeneratePhysicalName() *string
	GetResourceArnAttribute(arnAttr *string, arnComponents *awscdk.ArnComponents) *string
	GetResourceNameAttribute(nameAttr *string) *string
	Metric(metricName *string, type_ MetricType, props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	MetricFailure(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	MetricSuccess(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	MetricTimeout(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	OnEvent(id *string, options *awsevents.OnEventOptions) awsevents.Rule
	OnFailure(id *string, options *awsevents.OnEventOptions) awsevents.Rule
	OnStateChange(id *string, jobState JobState, options *awsevents.OnEventOptions) awsevents.Rule
	OnSuccess(id *string, options *awsevents.OnEventOptions) awsevents.Rule
	OnTimeout(id *string, options *awsevents.OnEventOptions) awsevents.Rule
	ToString() *string
}

A Glue Job. Experimental.

func NewJob 

func NewJob(scope constructs.Construct, id *string, props *JobProps) Job

Experimental.

type JobAttributes 

type JobAttributes struct {
	// The name of the job.
	// Experimental.
	JobName *string `json:"jobName"`
	// The IAM role assumed by Glue to run this job.
	// Experimental.
	Role awsiam.IRole `json:"role"`
}

Attributes for importing {@link Job}. Experimental.

type JobBookmarksEncryption 

type JobBookmarksEncryption struct {
	// Encryption mode.
	// Experimental.
	Mode JobBookmarksEncryptionMode `json:"mode"`
	// The KMS key to be used to encrypt the data.
	// Experimental.
	KmsKey awskms.IKey `json:"kmsKey"`
}

Job bookmarks encryption configuration. Experimental.

type JobBookmarksEncryptionMode 

type JobBookmarksEncryptionMode string

Encryption mode for Job Bookmarks. See: https://docs.aws.amazon.com/glue/latest/webapi/API_JobBookmarksEncryption.html#Glue-Type-JobBookmarksEncryption-JobBookmarksEncryptionMode

Experimental. 

const (
	JobBookmarksEncryptionMode_CLIENT_SIDE_KMS JobBookmarksEncryptionMode = "CLIENT_SIDE_KMS"
)

type JobExecutable 

type JobExecutable interface {
	Bind() *JobExecutableConfig
}

The executable properties related to the Glue job's GlueVersion, JobType and code. Experimental.

func JobExecutable_Of 

func JobExecutable_Of(config *JobExecutableConfig) JobExecutable

Create a custom JobExecutable. Experimental.

func JobExecutable_PythonEtl 

func JobExecutable_PythonEtl(props *PythonSparkJobExecutableProps) JobExecutable

Create Python executable props for Apache Spark ETL job. Experimental.

func JobExecutable_PythonShell 

func JobExecutable_PythonShell(props *PythonShellExecutableProps) JobExecutable

Create Python executable props for python shell jobs. Experimental.

func JobExecutable_PythonStreaming 

func JobExecutable_PythonStreaming(props *PythonSparkJobExecutableProps) JobExecutable

Create Python executable props for Apache Spark Streaming job. Experimental.

func JobExecutable_ScalaEtl 

func JobExecutable_ScalaEtl(props *ScalaJobExecutableProps) JobExecutable

Create Scala executable props for Apache Spark ETL job. Experimental.

func JobExecutable_ScalaStreaming 

func JobExecutable_ScalaStreaming(props *ScalaJobExecutableProps) JobExecutable

Create Scala executable props for Apache Spark Streaming job. Experimental.

type JobExecutableConfig 

type JobExecutableConfig struct {
	// Glue version.
	// See: https://docs.aws.amazon.com/glue/latest/dg/release-notes.html
	//
	// Experimental.
	GlueVersion GlueVersion `json:"glueVersion"`
	// The language of the job (Scala or Python).
	// See: `--job-language` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	Language JobLanguage `json:"language"`
	// The script that is executed by a job.
	// Experimental.
	Script Code `json:"script"`
	// Specify the type of the job whether it's an Apache Spark ETL or streaming one or if it's a Python shell job.
	// Experimental.
	Type JobType `json:"type"`
	// The Scala class that serves as the entry point for the job.
	//
	// This applies only if your the job langauage is Scala.
	// See: `--class` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ClassName *string `json:"className"`
	// Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.
	// See: `--extra-files` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraFiles *[]Code `json:"extraFiles"`
	// Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script.
	// See: `--extra-jars` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraJars *[]Code `json:"extraJars"`
	// Setting this value to true prioritizes the customer's extra JAR files in the classpath.
	// See: `--user-jars-first` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraJarsFirst *bool `json:"extraJarsFirst"`
	// Additional Python files that AWS Glue adds to the Python path before executing your script.
	// See: `--extra-py-files` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraPythonFiles *[]Code `json:"extraPythonFiles"`
	// The Python version to use.
	// Experimental.
	PythonVersion PythonVersion `json:"pythonVersion"`
}

Result of binding a `JobExecutable` into a `Job`. Experimental.

type JobLanguage 

type JobLanguage string

Runtime language of the Glue job. Experimental. 

const (
	JobLanguage_SCALA  JobLanguage = "SCALA"
	JobLanguage_PYTHON JobLanguage = "PYTHON"
)

type JobProps 

type JobProps struct {
	// The job's executable properties.
	// Experimental.
	Executable JobExecutable `json:"executable"`
	// The {@link Connection}s used for this job.
	//
	// Connections are used to connect to other AWS Service or resources within a VPC.
	// Experimental.
	Connections *[]IConnection `json:"connections"`
	// Enables continuous logging with the specified props.
	// See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ContinuousLogging *ContinuousLoggingProps `json:"continuousLogging"`
	// The default arguments for this job, specified as name-value pairs.
	// See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters
	//
	// Experimental.
	DefaultArguments *map[string]*string `json:"defaultArguments"`
	// The description of the job.
	// Experimental.
	Description *string `json:"description"`
	// Enables the collection of metrics for job profiling.
	// See: `--enable-metrics` at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	EnableProfilingMetrics *bool `json:"enableProfilingMetrics"`
	// The name of the job.
	// Experimental.
	JobName *string `json:"jobName"`
	// The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
	//
	// Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead.
	// Experimental.
	MaxCapacity *float64 `json:"maxCapacity"`
	// The maximum number of concurrent runs allowed for the job.
	//
	// An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.
	// Experimental.
	MaxConcurrentRuns *float64 `json:"maxConcurrentRuns"`
	// The maximum number of times to retry this job after a job run fails.
	// Experimental.
	MaxRetries *float64 `json:"maxRetries"`
	// The number of minutes to wait after a job run starts, before sending a job run delay notification.
	// Experimental.
	NotifyDelayAfter awscdk.Duration `json:"notifyDelayAfter"`
	// The IAM role assumed by Glue to run this job.
	//
	// If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.
	// See: https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html
	//
	// Experimental.
	Role awsiam.IRole `json:"role"`
	// The {@link SecurityConfiguration} to use for this job.
	// Experimental.
	SecurityConfiguration ISecurityConfiguration `json:"securityConfiguration"`
	// Enables the Spark UI debugging and monitoring with the specified props.
	// See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	SparkUI *SparkUIProps `json:"sparkUI"`
	// The tags to add to the resources on which the job runs.
	// Experimental.
	Tags *map[string]*string `json:"tags"`
	// The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
	// Experimental.
	Timeout awscdk.Duration `json:"timeout"`
	// The number of workers of a defined {@link WorkerType} that are allocated when a job runs.
	// Experimental.
	WorkerCount *float64 `json:"workerCount"`
	// The type of predefined worker that is allocated when a job runs.
	// Experimental.
	WorkerType WorkerType `json:"workerType"`
}

Construction properties for {@link Job}. Experimental.

type JobState 

type JobState string

Job states emitted by Glue to CloudWatch Events. See: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types for more information.

Experimental. 

const (
	JobState_SUCCEEDED JobState = "SUCCEEDED"
	JobState_FAILED    JobState = "FAILED"
	JobState_TIMEOUT   JobState = "TIMEOUT"
	JobState_STARTING  JobState = "STARTING"
	JobState_RUNNING   JobState = "RUNNING"
	JobState_STOPPING  JobState = "STOPPING"
	JobState_STOPPED   JobState = "STOPPED"
)

type JobType 

type JobType interface {
	Name() *string
}

The job type.

If you need to use a JobType that doesn't exist as a static member, you can instantiate a `JobType` object, e.g: `JobType.of('other name')`. Experimental.

func JobType_ETL 

func JobType_ETL() JobType

func JobType_Of 

func JobType_Of(name *string) JobType

Custom type name. Experimental.

func JobType_PYTHON_SHELL 

func JobType_PYTHON_SHELL() JobType

func JobType_STREAMING 

func JobType_STREAMING() JobType

type MetricType 

type MetricType string

The Glue CloudWatch metric type. See: https://docs.aws.amazon.com/glue/latest/dg/monitoring-awsglue-with-cloudwatch-metrics.html

Experimental. 

const (
	MetricType_GAUGE MetricType = "GAUGE"
	MetricType_COUNT MetricType = "COUNT"
)

type OutputFormat 

type OutputFormat interface {
	ClassName() *string
}

Absolute class name of the Hadoop `OutputFormat` to use when writing table files. Experimental.

func NewOutputFormat 

func NewOutputFormat(className *string) OutputFormat

Experimental.

func OutputFormat_HIVE_IGNORE_KEY_TEXT 

func OutputFormat_HIVE_IGNORE_KEY_TEXT() OutputFormat

func OutputFormat_PARQUET 

func OutputFormat_PARQUET() OutputFormat

type PythonShellExecutableProps 

type PythonShellExecutableProps struct {
	// Glue version.
	// See: https://docs.aws.amazon.com/glue/latest/dg/release-notes.html
	//
	// Experimental.
	GlueVersion GlueVersion `json:"glueVersion"`
	// The Python version to use.
	// Experimental.
	PythonVersion PythonVersion `json:"pythonVersion"`
	// The script that executes a job.
	// Experimental.
	Script Code `json:"script"`
	// Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.
	//
	// Only individual files are supported, directories are not supported.
	// See: `--extra-files` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraFiles *[]Code `json:"extraFiles"`
	// Additional Python files that AWS Glue adds to the Python path before executing your script.
	//
	// Only individual files are supported, directories are not supported.
	// See: `--extra-py-files` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraPythonFiles *[]Code `json:"extraPythonFiles"`
}

Props for creating a Python shell job executable. Experimental.

type PythonSparkJobExecutableProps 

type PythonSparkJobExecutableProps struct {
	// Glue version.
	// See: https://docs.aws.amazon.com/glue/latest/dg/release-notes.html
	//
	// Experimental.
	GlueVersion GlueVersion `json:"glueVersion"`
	// The Python version to use.
	// Experimental.
	PythonVersion PythonVersion `json:"pythonVersion"`
	// The script that executes a job.
	// Experimental.
	Script Code `json:"script"`
	// Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.
	//
	// Only individual files are supported, directories are not supported.
	// See: `--extra-files` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraFiles *[]Code `json:"extraFiles"`
	// Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported.
	// See: `--extra-jars` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraJars *[]Code `json:"extraJars"`
	// Setting this value to true prioritizes the customer's extra JAR files in the classpath.
	// See: `--user-jars-first` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraJarsFirst *bool `json:"extraJarsFirst"`
	// Additional Python files that AWS Glue adds to the Python path before executing your script.
	//
	// Only individual files are supported, directories are not supported.
	// See: `--extra-py-files` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraPythonFiles *[]Code `json:"extraPythonFiles"`
}

Props for creating a Python Spark (ETL or Streaming) job executable. Experimental.

type PythonVersion 

type PythonVersion string

Python version. Experimental. 

const (
	PythonVersion_TWO   PythonVersion = "TWO"
	PythonVersion_THREE PythonVersion = "THREE"
)

type S3Code 

type S3Code interface {
	Code
	Bind(_scope constructs.Construct, grantable awsiam.IGrantable) *CodeConfig
}

Glue job Code from an S3 bucket. Experimental.

func AssetCode_FromBucket 

func AssetCode_FromBucket(bucket awss3.IBucket, key *string) S3Code

Job code as an S3 object. Experimental.

func Code_FromBucket 

func Code_FromBucket(bucket awss3.IBucket, key *string) S3Code

Job code as an S3 object. Experimental.

func NewS3Code 

func NewS3Code(bucket awss3.IBucket, key *string) S3Code

Experimental.

func S3Code_FromBucket 

func S3Code_FromBucket(bucket awss3.IBucket, key *string) S3Code

Job code as an S3 object. Experimental.

type S3Encryption 

type S3Encryption struct {
	// Encryption mode.
	// Experimental.
	Mode S3EncryptionMode `json:"mode"`
	// The KMS key to be used to encrypt the data.
	// Experimental.
	KmsKey awskms.IKey `json:"kmsKey"`
}

S3 encryption configuration. Experimental.

type S3EncryptionMode 

type S3EncryptionMode string

Encryption mode for S3. See: https://docs.aws.amazon.com/glue/latest/webapi/API_S3Encryption.html#Glue-Type-S3Encryption-S3EncryptionMode

Experimental. 

const (
	S3EncryptionMode_S3_MANAGED S3EncryptionMode = "S3_MANAGED"
	S3EncryptionMode_KMS        S3EncryptionMode = "KMS"
)

type ScalaJobExecutableProps 

type ScalaJobExecutableProps struct {
	// The fully qualified Scala class name that serves as the entry point for the job.
	// See: `--class` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ClassName *string `json:"className"`
	// Glue version.
	// See: https://docs.aws.amazon.com/glue/latest/dg/release-notes.html
	//
	// Experimental.
	GlueVersion GlueVersion `json:"glueVersion"`
	// The script that executes a job.
	// Experimental.
	Script Code `json:"script"`
	// Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.
	//
	// Only individual files are supported, directories are not supported.
	// See: `--extra-files` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraFiles *[]Code `json:"extraFiles"`
	// Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported.
	// See: `--extra-jars` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraJars *[]Code `json:"extraJars"`
	// Setting this value to true prioritizes the customer's extra JAR files in the classpath.
	// See: `--user-jars-first` in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
	//
	// Experimental.
	ExtraJarsFirst *bool `json:"extraJarsFirst"`
}

Props for creating a Scala Spark (ETL or Streaming) job executable. Experimental.

type Schema 

type Schema interface {
}

See: https://docs.aws.amazon.com/athena/latest/ug/data-types.html

Experimental.

func NewSchema 

func NewSchema() Schema

Experimental.

type SecurityConfiguration 

type SecurityConfiguration interface {
	awscdk.Resource
	ISecurityConfiguration
	CloudWatchEncryptionKey() awskms.IKey
	Env() *awscdk.ResourceEnvironment
	JobBookmarksEncryptionKey() awskms.IKey
	Node() constructs.Node
	PhysicalName() *string
	S3EncryptionKey() awskms.IKey
	SecurityConfigurationName() *string
	Stack() awscdk.Stack
	ApplyRemovalPolicy(policy awscdk.RemovalPolicy)
	GeneratePhysicalName() *string
	GetResourceArnAttribute(arnAttr *string, arnComponents *awscdk.ArnComponents) *string
	GetResourceNameAttribute(nameAttr *string) *string
	ToString() *string
}

A security configuration is a set of security properties that can be used by AWS Glue to encrypt data at rest.

The following scenarios show some of the ways that you can use a security configuration. - Attach a security configuration to an AWS Glue crawler to write encrypted Amazon CloudWatch Logs. - Attach a security configuration to an extract, transform, and load (ETL) job to write encrypted Amazon Simple Storage Service (Amazon S3) targets and encrypted CloudWatch Logs. - Attach a security configuration to an ETL job to write its jobs bookmarks as encrypted Amazon S3 data. - Attach a security configuration to a development endpoint to write encrypted Amazon S3 targets. Experimental.

func NewSecurityConfiguration 

func NewSecurityConfiguration(scope constructs.Construct, id *string, props *SecurityConfigurationProps) SecurityConfiguration

Experimental.

type SecurityConfigurationProps 

type SecurityConfigurationProps struct {
	// The name of the security configuration.
	// Experimental.
	SecurityConfigurationName *string `json:"securityConfigurationName"`
	// The encryption configuration for Amazon CloudWatch Logs.
	// Experimental.
	CloudWatchEncryption *CloudWatchEncryption `json:"cloudWatchEncryption"`
	// The encryption configuration for Glue Job Bookmarks.
	// Experimental.
	JobBookmarksEncryption *JobBookmarksEncryption `json:"jobBookmarksEncryption"`
	// The encryption configuration for Amazon Simple Storage Service (Amazon S3) data.
	// Experimental.
	S3Encryption *S3Encryption `json:"s3Encryption"`
}

Constructions properties of {@link SecurityConfiguration}. Experimental.

type SerializationLibrary 

type SerializationLibrary interface {
	ClassName() *string
}

Serialization library to use when serializing/deserializing (SerDe) table records. See: https://cwiki.apache.org/confluence/display/Hive/SerDe

Experimental.

func NewSerializationLibrary 

func NewSerializationLibrary(className *string) SerializationLibrary

Experimental.

func SerializationLibrary_AVRO 

func SerializationLibrary_AVRO() SerializationLibrary

func SerializationLibrary_CLOUDTRAIL 

func SerializationLibrary_CLOUDTRAIL() SerializationLibrary

func SerializationLibrary_GROK 

func SerializationLibrary_GROK() SerializationLibrary

func SerializationLibrary_HIVE_JSON 

func SerializationLibrary_HIVE_JSON() SerializationLibrary

func SerializationLibrary_LAZY_SIMPLE 

func SerializationLibrary_LAZY_SIMPLE() SerializationLibrary

func SerializationLibrary_OPENX_JSON 

func SerializationLibrary_OPENX_JSON() SerializationLibrary

func SerializationLibrary_OPEN_CSV 

func SerializationLibrary_OPEN_CSV() SerializationLibrary

func SerializationLibrary_ORC 

func SerializationLibrary_ORC() SerializationLibrary

func SerializationLibrary_PARQUET 

func SerializationLibrary_PARQUET() SerializationLibrary

func SerializationLibrary_REGEXP 

func SerializationLibrary_REGEXP() SerializationLibrary

type SparkUILoggingLocation 

type SparkUILoggingLocation struct {
	// The bucket where the Glue job stores the logs.
	// Experimental.
	Bucket awss3.IBucket `json:"bucket"`
	// The path inside the bucket (objects prefix) where the Glue job stores the logs.
	// Experimental.
	Prefix *string `json:"prefix"`
}

The Spark UI logging location. See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Experimental.

type SparkUIProps 

type SparkUIProps struct {
	// Enable Spark UI.
	// Experimental.
	Enabled *bool `json:"enabled"`
	// The bucket where the Glue job stores the logs.
	// Experimental.
	Bucket awss3.IBucket `json:"bucket"`
	// The path inside the bucket (objects prefix) where the Glue job stores the logs.
	// Experimental.
	Prefix *string `json:"prefix"`
}

Properties for enabling Spark UI monitoring feature for Spark-based Glue jobs. See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Experimental.

type Table 

type Table interface {
	awscdk.Resource
	ITable
	Bucket() awss3.IBucket
	Columns() *[]*Column
	Compressed() *bool
	Database() IDatabase
	DataFormat() DataFormat
	Encryption() TableEncryption
	EncryptionKey() awskms.IKey
	Env() *awscdk.ResourceEnvironment
	Node() constructs.Node
	PartitionKeys() *[]*Column
	PhysicalName() *string
	S3Prefix() *string
	Stack() awscdk.Stack
	TableArn() *string
	TableName() *string
	ApplyRemovalPolicy(policy awscdk.RemovalPolicy)
	GeneratePhysicalName() *string
	GetResourceArnAttribute(arnAttr *string, arnComponents *awscdk.ArnComponents) *string
	GetResourceNameAttribute(nameAttr *string) *string
	GrantRead(grantee awsiam.IGrantable) awsiam.Grant
	GrantReadWrite(grantee awsiam.IGrantable) awsiam.Grant
	GrantWrite(grantee awsiam.IGrantable) awsiam.Grant
	ToString() *string
}

A Glue table. Experimental.

func NewTable 

func NewTable(scope constructs.Construct, id *string, props *TableProps) Table

Experimental.

type TableAttributes 

type TableAttributes struct {
	// Experimental.
	TableArn *string `json:"tableArn"`
	// Experimental.
	TableName *string `json:"tableName"`
}

Experimental.

type TableEncryption 

type TableEncryption string

Encryption options for a Table. See: https://docs.aws.amazon.com/athena/latest/ug/encryption.html

Experimental. 

const (
	TableEncryption_UNENCRYPTED     TableEncryption = "UNENCRYPTED"
	TableEncryption_S3_MANAGED      TableEncryption = "S3_MANAGED"
	TableEncryption_KMS             TableEncryption = "KMS"
	TableEncryption_KMS_MANAGED     TableEncryption = "KMS_MANAGED"
	TableEncryption_CLIENT_SIDE_KMS TableEncryption = "CLIENT_SIDE_KMS"
)

type TableProps 

type TableProps struct {
	// Columns of the table.
	// Experimental.
	Columns *[]*Column `json:"columns"`
	// Database in which to store the table.
	// Experimental.
	Database IDatabase `json:"database"`
	// Storage type of the table's data.
	// Experimental.
	DataFormat DataFormat `json:"dataFormat"`
	// Name of the table.
	// Experimental.
	TableName *string `json:"tableName"`
	// S3 bucket in which to store data.
	// Experimental.
	Bucket awss3.IBucket `json:"bucket"`
	// Indicates whether the table's data is compressed or not.
	// Experimental.
	Compressed *bool `json:"compressed"`
	// Description of the table.
	// Experimental.
	Description *string `json:"description"`
	// The kind of encryption to secure the data with.
	//
	// You can only provide this option if you are not explicitly passing in a bucket.
	//
	// If you choose `SSE-KMS`, you *can* provide an un-managed KMS key with `encryptionKey`.
	// If you choose `CSE-KMS`, you *must* provide an un-managed KMS key with `encryptionKey`.
	// Experimental.
	Encryption TableEncryption `json:"encryption"`
	// External KMS key to use for bucket encryption.
	//
	// The `encryption` property must be `SSE-KMS` or `CSE-KMS`.
	// Experimental.
	EncryptionKey awskms.IKey `json:"encryptionKey"`
	// Partition columns of the table.
	// Experimental.
	PartitionKeys *[]*Column `json:"partitionKeys"`
	// S3 prefix under which table objects are stored.
	// Experimental.
	S3Prefix *string `json:"s3Prefix"`
	// Indicates whether the table data is stored in subdirectories.
	// Experimental.
	StoredAsSubDirectories *bool `json:"storedAsSubDirectories"`
}

Experimental.

type Type 

type Type struct {
	// Glue InputString for this type.
	// Experimental.
	InputString *string `json:"inputString"`
	// Indicates whether this type is a primitive data type.
	// Experimental.
	IsPrimitive *bool `json:"isPrimitive"`
}

Represents a type of a column in a table schema. Experimental.

func Schema_Array 

func Schema_Array(itemType *Type) *Type

Creates an array of some other type. Experimental.

func Schema_BIG_INT 

func Schema_BIG_INT() *Type

func Schema_BINARY 

func Schema_BINARY() *Type

func Schema_BOOLEAN 

func Schema_BOOLEAN() *Type

func Schema_Char 

func Schema_Char(length *float64) *Type

Fixed length character data, with a specified length between 1 and 255. Experimental.

func Schema_DATE 

func Schema_DATE() *Type

func Schema_DOUBLE 

func Schema_DOUBLE() *Type

func Schema_Decimal 

func Schema_Decimal(precision *float64, scale *float64) *Type

Creates a decimal type.

TODO: Bounds Experimental.

func Schema_FLOAT 

func Schema_FLOAT() *Type

func Schema_INTEGER 

func Schema_INTEGER() *Type

func Schema_Map 

func Schema_Map(keyType *Type, valueType *Type) *Type

Creates a map of some primitive key type to some value type. Experimental.

func Schema_SMALL_INT 

func Schema_SMALL_INT() *Type

func Schema_STRING 

func Schema_STRING() *Type

func Schema_Struct 

func Schema_Struct(columns *[]*Column) *Type

Creates a nested structure containing individually named and typed columns. Experimental.

func Schema_TIMESTAMP 

func Schema_TIMESTAMP() *Type

func Schema_TINY_INT 

func Schema_TINY_INT() *Type

func Schema_Varchar 

func Schema_Varchar(length *float64) *Type

Variable length character data, with a specified length between 1 and 65535. Experimental.

type WorkerType 

type WorkerType interface {
	Name() *string
}

The type of predefined worker that is allocated when a job runs.

If you need to use a WorkerType that doesn't exist as a static member, you can instantiate a `WorkerType` object, e.g: `WorkerType.of('other type')`. Experimental.

func WorkerType_G_1X 

func WorkerType_G_1X() WorkerType

func WorkerType_G_2X 

func WorkerType_G_2X() WorkerType

func WorkerType_Of 

func WorkerType_Of(workerType *string) WorkerType

Custom worker type. Experimental.

func WorkerType_STANDARD 

func WorkerType_STANDARD() WorkerType

Source Files

View all Source files

Directories

Path Synopsis
Package jsii contains the functionaility needed for jsii packages to initialize their dependencies and themselves.
 Package jsii contains the functionaility needed for jsii packages to initialize their dependencies and themselves.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.