athena

package
v0.0.35 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 9, 2023 License: Apache-2.0 Imports: 9 Imported by: 0

README

Vibe-io CDK-Extensions Athena Construct Library

The @cdk-extensions/athena package contains advanced constructs and patterns for setting up named queries. The constructs presented here are intended to be replacements for equivalent AWS constructs in the CDK Athena module, but with additional features included.

AWS CDK Athena API Reference

To import and use this module within your CDK project:

import * as athena from 'cdk-extensions/athena';

Objective

The Athena module is a component of the logging strategy provided by this project defined by stacks/AwsLoggingStack. Athena uses the AWS Glue Data Catalog to store and retrieve table metadata for the Amazon S3 data in your Amazon Web Services account. The table metadata lets the Athena query engine know how to find, read, and process the data that you want to query.

The logging strategy defined in this project accounts for all AWS services that log to S3 including ALB, CloudFront, CloudTrail, Flow Logs, S3 access logs, SES, and WAF. For each service a Glue crawler preforms an ETL process to analyze and categorize data in Amazon S3 and store the associated metadata in AWS Glue Data Catalog.

Usage

The Athena module creates CfnNamedQuery resources when the createQueries property is set to true in the glue-tables module. Several default named queires are defined that aid in improving the security posture of your AWS Account. This package introduces several named queries for the following AWS services:

Examples for each of the services below can be found in src/glue-tables

Example of an Athena query to retrive the 100 most active IP addresses by request count:

if (this.createQueries) {
    this.topIpsNamedQuery = new NamedQuery(this, 'top-ips-named-query', {
    database: this.database,
    description: 'Gets the 100 most actvie IP addresses by request count.',
    name: this.friendlyQueryNames ? 'alb-top-ips' : undefined,
    queryString: [
        'SELECT client_ip,',
        '    COUNT(*) AS requests,',
        '    COUNT_IF(elb_status_code BETWEEN 400 AND 499) AS errors_4xx,',
        '    COUNT_IF(elb_status_code BETWEEN 500 AND 599) AS errors_5xx,',
        '    SUM(sent_bytes) AS sent,',
        '    SUM(received_bytes) AS received,',
        '    SUM(sent_bytes + received_bytes) AS total,',
        '    ARBITRARY(user_agent) as user_agent',
        `FROM ${this.tableName}`,
        "WHERE day >= DATE_FORMAT(NOW() - PARSE_DURATION('1d'), '%Y/%m/%d')",
        "    AND FROM_ISO8601_TIMESTAMP(time) >= NOW() - PARSE_DURATION('1d')",
        'GROUP BY client_ip',
        'ORDER by total DESC LIMIT 100;',
    ].join('\n'),
    });
ALB

See src/glue-tables/alb-logs-table.ts

Gets the 100 most actvie IP addresses by request count.

Gets the 100 most recent ELB 5XX responses.

CloudFront

See src/glue-tables/cloudfront-logs-table.ts

Gets statistics for CloudFront distributions for the last day.

Gets the 100 most recent requests that resulted in an error from CloudFront.

Gets the 100 most active IP addresses by request count.

Gets the 100 most requested CloudFront objects.

CloudTrail

See src/glue-tables/cloudtrail-logs-table.ts

Gets the 100 most recent unauthorized AWS API calls.

Gets the 100 most recent AWS user logins.

Flow Logs

See src/glue-tables/flow-logs-table.ts

Gets the 100 most recent rejected packets that stayed within the private network ranges.

S3 Access Logs

See src/glue-tables/s3-access-logs-table.ts

Gets the 100 most recent failed S3 access requests.

SES Logs

See src/glue-tables/ses-logs-table.ts

Gets the 100 most recent bounces from the last day.

Gets the 100 most recent complaints from the last day.

WAF Logs

See src/glue-tables/waf-logs-table.ts

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NamedQuery_IsConstruct 

func NamedQuery_IsConstruct(x interface{}) *bool

Checks if `x` is a construct.

Returns: true if `x` is an object created from a class which extends `Construct`. Deprecated: use `x instanceof Construct` instead.

func NamedQuery_IsOwnedResource 

func NamedQuery_IsOwnedResource(construct constructs.IConstruct) *bool

Returns true if the construct was created by CDK, and false otherwise.

func NamedQuery_IsResource 

func NamedQuery_IsResource(construct constructs.IConstruct) *bool

Check whether the given construct is a Resource.

func NewNamedQuery_Override 

func NewNamedQuery_Override(n NamedQuery, scope constructs.Construct, id *string, props *NamedQueryProps)

Creates a new instance of the NamedQuery class.

Types

type NamedQuery 

type NamedQuery interface {
	awscdk.Resource
	// The Glue database to which the query belongs.
	// See: [NamedQuery Database](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-athena-namedquery.html#cfn-athena-namedquery-database)
	//
	Database() glue.Database
	// A human friendly description explaining the functionality of the query.
	// See: [NamedQuery Description](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-athena-namedquery.html#cfn-athena-namedquery-description)
	//
	Description() *string
	// The environment this resource belongs to.
	//
	// For resources that are created and managed by the CDK
	// (generally, those created by creating new class instances like Role, Bucket, etc.),
	// this is always the same as the environment of the stack they belong to;
	// however, for imported resources
	// (those obtained from static methods like fromRoleArn, fromBucketName, etc.),
	// that might be different than the stack they were imported into.
	Env() *awscdk.ResourceEnvironment
	// The name of the query.
	// See: [NamedQuery Name](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-athena-namedquery.html#cfn-athena-namedquery-name)
	//
	Name() *string
	// The unique ID of the query.
	NamedQueryId() *string
	// The name of the query.
	NamedQueryName() *string
	// The tree node.
	Node() constructs.Node
	// Returns a string-encoded token that resolves to the physical name that should be passed to the CloudFormation resource.
	//
	// This value will resolve to one of the following:
	// - a concrete value (e.g. `"my-awesome-bucket"`)
	// - `undefined`, when a name should be generated by CloudFormation
	// - a concrete name generated automatically during synthesis, in
	//    cross-environment scenarios.
	PhysicalName() *string
	// The SQL statements that make up the query.
	// See: [Athena SQL reference](https://docs.aws.amazon.com/athena/latest/ug/ddl-sql-reference.html)
	//
	QueryString() *string
	// The underlying NamedQuery CloudFormation resource.
	// See: [AWS::Athena::NamedQuery](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-athena-namedquery.html)
	//
	Resource() awsathena.CfnNamedQuery
	// The stack in which this resource is defined.
	Stack() awscdk.Stack
	// Apply the given removal policy to this resource.
	//
	// The Removal Policy controls what happens to this resource when it stops
	// being managed by CloudFormation, either because you've removed it from the
	// CDK application or because you've made a change that requires the resource
	// to be replaced.
	//
	// The resource can be deleted (`RemovalPolicy.DESTROY`), or left in your AWS
	// account for data recovery and cleanup later (`RemovalPolicy.RETAIN`).
	ApplyRemovalPolicy(policy awscdk.RemovalPolicy)
	GeneratePhysicalName() *string
	// Returns an environment-sensitive token that should be used for the resource's "ARN" attribute (e.g. `bucket.bucketArn`).
	//
	// Normally, this token will resolve to `arnAttr`, but if the resource is
	// referenced across environments, `arnComponents` will be used to synthesize
	// a concrete ARN with the resource's physical name. Make sure to reference
	// `this.physicalName` in `arnComponents`.
	GetResourceArnAttribute(arnAttr *string, arnComponents *awscdk.ArnComponents) *string
	// Returns an environment-sensitive token that should be used for the resource's "name" attribute (e.g. `bucket.bucketName`).
	//
	// Normally, this token will resolve to `nameAttr`, but if the resource is
	// referenced across environments, it will be resolved to `this.physicalName`,
	// which will be a concrete name.
	GetResourceNameAttribute(nameAttr *string) *string
	// Returns a string representation of this construct.
	ToString() *string
}

func NewNamedQuery 

func NewNamedQuery(scope constructs.Construct, id *string, props *NamedQueryProps) NamedQuery

Creates a new instance of the NamedQuery class.

type NamedQueryProps 

type NamedQueryProps struct {
	// The AWS account ID this resource belongs to.
	Account *string `field:"optional" json:"account" yaml:"account"`
	// ARN to deduce region and account from.
	//
	// The ARN is parsed and the account and region are taken from the ARN.
	// This should be used for imported resources.
	//
	// Cannot be supplied together with either `account` or `region`.
	EnvironmentFromArn *string `field:"optional" json:"environmentFromArn" yaml:"environmentFromArn"`
	// The value passed in by users to the physical name prop of the resource.
	//
	// - `undefined` implies that a physical name will be allocated by
	//    CloudFormation during deployment.
	// - a concrete value implies a specific physical name
	// - `PhysicalName.GENERATE_IF_NEEDED` is a marker that indicates that a physical will only be generated
	//    by the CDK if it is needed for cross-environment references. Otherwise, it will be allocated by CloudFormation.
	PhysicalName *string `field:"optional" json:"physicalName" yaml:"physicalName"`
	// The AWS region this resource belongs to.
	Region *string `field:"optional" json:"region" yaml:"region"`
	// The Glue database to which the query belongs.
	// See: [NamedQuery Database](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-athena-namedquery.html#cfn-athena-namedquery-database)
	//
	Database glue.Database `field:"required" json:"database" yaml:"database"`
	// The SQL statements that make up the query.
	// See: [Athena SQL reference](https://docs.aws.amazon.com/athena/latest/ug/ddl-sql-reference.html)
	//
	QueryString *string `field:"required" json:"queryString" yaml:"queryString"`
	// A human friendly description explaining the functionality of the query.
	// See: [NamedQuery Description](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-athena-namedquery.html#cfn-athena-namedquery-description)
	//
	Description *string `field:"optional" json:"description" yaml:"description"`
	// The name of the query.
	// See: [NamedQuery Name](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-athena-namedquery.html#cfn-athena-namedquery-name)
	//
	Name *string `field:"optional" json:"name" yaml:"name"`
}

Configuration for Database.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.