https://github.com/asecurityteam/awsconfig-transformer
Overview
AWS Config provides a detailed view of the configuration of AWS resources, potentially across
multiple AWS accounts, and can provide a stream of configuration change events via an SNS topic
which publishes to SQS. The awsconfig-transformerd service provides a lambda handler which accepts
the configuration item change notification payload,
extracts the changed network information, and returns a transformed version of the configuration change.
The goal of the transformation is to highlight changes to the network interfaces associated with AWS resources.
Example topic payloads can be seen at AWS's Developer Guide page, but beware, the data is old. To
gain a complete understanding of the variances in notification payloads, it is recommended to
gather real notifications of actual change events.
For the current list of supported resource types, see Supported Resources.
All other config change types are ignored. The payload emitted from this transformer adheres to the
following JSON specification:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"title": "Schema for cloud asset change events",
"properties": {
"changeTime": {
"type": "string",
"title": "time at which the asset change occurred",
"format": "date-time"
},
"resourceType": {
"type": "string",
"title": "the AWS resource type"
},
"accountId": {
"type": "string",
"title": "the ID of the AWS account"
},
"region": {
"type": "string",
"title": "the AWS region"
},
"resourceId": {
"type": "string",
"title": "the ID of the AWS resource"
},
"tags": {
"type": "object",
"title": "AWS tags",
"additionalProperties": {
"type": "string"
}
},
"changes": {
"type": "array",
"title": "list of changes which occurred",
"items": {
"type": "object",
"properties": {
"publicIpAddresses": {
"type": "array",
"title": "public IP addresses for the asset",
"items": {
"type": "string"
}
},
"privateIpAddresses": {
"type": "array",
"title": "private IP addresses for the asset",
"items": {
"type": "string"
}
},
"hostnames": {
"type": "array",
"title": "hostnames of the asset",
"items": {
"type": "string"
}
},
"relatedResources": {
"type": "array",
"title": "resources that have a relation to this asset",
"items": {
"type": "string"
}
},
"changeType": {
"type": "string",
"title": "the type of change which occurred",
"enum": ["ADDED", "DELETED"]
}
}
}
}
},
"required": [
"changeTime",
"resourceType",
"accountId",
"region",
"resourceId",
"tags",
"changes"
]
}
Quick Start
Install docker and docker-compose.
The app can be run locally by running make run
.
This will run docker-compose
for the serverfull project
as well as the supplied serverfull-gateway configuration.
The sample configration provided assumes there will be a stats
collector running. To disable this, remove the stats configuration
lines from the server configuration and the serverfull-gateway
configuration.
The app should now be running on port 8080.
curl -vX POST "http://localhost:8080" -H "Content-Type:application/json" -d @pkg/handlers/v1/testdata/ec2.0.json
Configuration
Images of this project are built, and hosted on DockerHub.
This code functions as a stand-alone Lambda function, and can be deployed to AWS Lambda directly. To run in the AWS Lambda environment,
create a new Go project, import this project as a dependency, and run the lambda using the aws-lambda-sdk:
func main() {
transformer := &v1.Transformer{
LogFn: <LOGGER_PROVIDER>,
StatFn: <STATS_PROVIDER>,
}
lambda.Start(transformer.Handle)
}
For those who do not have access to AWS Lambda, you can run your own configuration by composing this
image with your own custom configuration of serverfull-gateway.
Logging
This project makes use of logevent which provides structured logging
using Go structs and tags. By default the project will set a logger value in the context for each request. The handler
uses the LogFn
function defined in pkg/domain/alias.go
to extract the logger instance from the context.
The built in logger can be configured through the serverfull runtime configuration.
Stats
This project uses xstats as its underlying stats library. By default the project will
set a stat client value in the context for each request. The handler uses the StatFn
function defined in
pkg/domain/alias.go
to extract the logger instance from the context.
The built in stats client can be configured through the serverfull runtime configuration.
Additional resources:
Supported Resources
The current version only supports extracting network changes from:
- EC2 instances
- Elastic Load Balancers
- Application Load Balancers
- Elastic Network Interfaces
- Subnets
Caveat to this list: Not all ENIs are supported. Currently this targets a subset of them,
those that are requester managed.
Status
This project is in incubation which means we are not yet operating this tool in production
and the interfaces are subject to change.
Contributing
If you are interested in contributing to the project, feel free to open an issue or PR.
Building And Testing
We publish a docker image called SDCLI that
bundles all of our build dependencies. It is used by the included Makefile to help make
building and testing a bit easier. The following actions are available through the Makefile:
-
make dep
Install the project dependencies into a vendor directory
-
make lint
Run our static analysis suite
-
make test
Run unit tests and generate a coverage artifact
-
make integration
Run integration tests and generate a coverage artifact
-
make coverage
Report the combined coverage for unit and integration tests
-
make build
Generate a local build of the project (if applicable)
-
make run
Run a local instance of the project (if applicable)
-
make doc
Generate the project code documentation and make it viewable
locally.
Quality Gates
Our build process will run the following checks before going green:
- make lint
- make test
- make integration
- make coverage (combined result must be 85% or above for the project)
Running these locally, will give early indicators of pass/fail.
License
This project is licensed under Apache 2.0. See LICENSE.txt for details.
Contributing Agreement
Atlassian requires signing a contributor's agreement before we can accept a
patch. If you are an individual you can fill out the
individual CLA.
If you are contributing on behalf of your company then please fill out the
corporate CLA.