cloud-inventory

command module
v0.0.0-...-62343dc Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 8, 2022 License: MIT Imports: 0 Imported by: 0

README

cloud-inventory

A data warehousing solution for storing periodic cloud infrastructure configuration snapshots in S3, enabling queries through AWS Athena, and custom API, and a web interface.

Differentials of given resources can be calculated between two points in time, allowing you to track changes in your environment.

For a complete list of resources ingested by cloud-inventory, see IMPLEMENTED_RESOURCES.md

Testing it out

Read the quickstart guide to set up a development environment and start ingesting data

Once data is ingested, you can either query the data using Athena, the API, or by viewing the UI

What it does

cloud-inventory scrapes cloud APIs (just AWS currently, adding Azure support in the future) and stores the response structures in various different backend database options (S3, DynamoDB, MongoDB). These scrapes can occur periodically to allow for historical querying of your cloud resources.

The databases are indexed to allow efficient querying of data for a given date, allowing efficient historical analysis.

cloud-inventory exposes an API for querying data from your desired database, enabling quick and easy access to historic information on your cloud resources across all accounts.

How it works

cloud-inventory works on various defined data sources such as AWS EC2 instances. Each of these datasources is explicitly defined and has a mapping defined of it's API response to an internal database schema. These database schemas are auto-generated based on the cloud providers own API spec, meaning cloud-inventory ingests all possible information. In some scenarios, cloud-inventory combines multiple cloud provider APIs/resources into a single object. For example, the AWS S3 API exposes buckets and each piece of their configuration as separate API calls. cloud-inventory combines all the API responses for a given bucket into a single object, making it easy to view the full configuration of the resource in a single place.

Each time cloud-inventory runs, it queries each resource against each unique location (i.e. a given region in a given AWS account) and stores the API responses in whichever database(s) you have enabled, indexed by the date that the scrape occured.

You can have multiple databases enabled for writing data. This allows you to write data to a high-performance database like DynamoDB or MongoDB for use with the API, and also to S3 for use with Athena or other big-data tools.

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
cmd
Package docs GENERATED BY THE COMMAND ABOVE; DO NOT EDIT This file was generated by swaggo/swag
Package docs GENERATED BY THE COMMAND ABOVE; DO NOT EDIT This file was generated by swaggo/swag
internal
api
api/routes/aws
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_router.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_router.tmpl
api/routes/aws/acm
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/apigateway
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/apigatewayv2
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/applicationautoscaling
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/athena
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/autoscaling
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/backup
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/cloudformation
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/cloudfront
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/cloudtrail
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/cloudwatch
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/cloudwatchlogs
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/dynamodb
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/ec2
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/ecr
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/ecs
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/efs
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/elasticache
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/elasticloadbalancing
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/elasticloadbalancingv2
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/iam
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/kms
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/lambda
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/rds
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/redshift
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/route53
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/s3
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
api/routes/aws/secretsmanager
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/sns
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/sqs
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/ssm
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_metadata_route.tmpl
api/routes/aws/storagegateway
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_api_routes.tmpl
db
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/dao.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/dao.tmpl
db/dynamodb
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/dynamodb_dao.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/dynamodb_dao.tmpl
db/mongo
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/mongo_dao.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/mongo_dao.tmpl
db/multi
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/multi_dao.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/multi_dao.tmpl
db/s3ion
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/s3_ion_dao.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/s3_ion_dao.tmpl
db/s3parquet
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/s3_parquet_dao.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/s3_parquet_dao.tmpl
inventory
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_inventory_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_inventory_file.tmpl
lambda
pkg
aws
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_client_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_client_file.tmpl
aws/acm
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
aws/apigateway
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_referenced_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_referenced_resource_file.tmpl
aws/apigatewayv2
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/applicationautoscaling
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_referenced_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_referenced_resource_file.tmpl
aws/athena
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
aws/autoscaling
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/backup
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/cloudformation
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/cloudfront
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
aws/cloudtrail
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_referenced_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_referenced_resource_file.tmpl
aws/cloudwatch
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/cloudwatchlogs
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/dynamodb
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/ec2
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/ecr
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/ecs
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
aws/efs
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/elasticache
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/elasticloadbalancing
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/elasticloadbalancingv2
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/iam
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/interfaces
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_client_interface_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_service_client_interface_file.tmpl
aws/kms
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
aws/lambda
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/rds
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/redshift
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/route53
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/s3
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_fetching_file.tmpl
aws/secretsmanager
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/sns
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/sqs
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
aws/ssm
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_helpers_file.tmpl
aws/storagegateway
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl
AUTOGENERATED CODE DO NOT EDIT This file is automatically generated from /internal/codegen/templates/aws_resource_file.tmpl

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL