cost-api

command module
v0.15.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 7, 2023 License: AGPL-3.0 Imports: 11 Imported by: 0

README

cost-api

This API provides simple restful API access to Amazon's Cost explorer and cloudwatch metrics service.

Endpoints

GET /v1/cost/ping
GET /v1/cost/version
GET /v1/cost/metrics

GET /v1/cost/{account}/spaces/{spaceid}[?start=2019-10-01&end=2019-10-30][&groupBy=SERVICE]

POST /v1/cost/{account}/spaces/{spaceid}/budgets
GET /v1/cost/{account}/spaces/{spaceid}/budgets
DELETE /v1/cost/{account}/spaces/{spaceid}/budgets/{budget}

GET /v1/cost/{account}/spaces/{space}/instances/{id}/optimizer

GET /v1/inventory/{account}/spaces/{spaceid}

GET /v1/metrics/{account}/instances/{id}/graph?metric={metric1}[&metric={metric2}&start=-P1D&end=PT0H&period=300]
GET /v1/metrics/{account}/clusters/{cluster}/services/{service}/graph?metric={metric1}[&metric={metric2}&start=-P1D&end=PT0H&period=300]
GET /v1/metrics/{account}/buckets/{bucket}/graph?metric={BucketSizeBytes|NumberOfObjects}
GET /v1/metrics/{account}/rds/{type}/{id}/graph?metric={metric1}[&metric={metric2}&start=-P1D&end=PT0H&period=300]

Cost Usage

Get the cost and usage for a space ID

By default, this will get the month to date costs for a space id (based on the spinup:spaceid tag). Date ranges and grouping by different dimensions is supported by passing query parameters.

Request month to date costs for a space

GET /v1/cost/{account}/spaces/{spaceid}

Response
[
    {
        "Estimated": true,
        "Groups": [],
        "TimePeriod": {
            "End": "2020-01-15",
            "Start": "2020-01-01"
        },
        "Total": {
            "BlendedCost": {
                "Amount": "0",
                "Unit": "USD"
            },
            "UnblendedCost": {
                "Amount": "0",
                "Unit": "USD"
            },
            "UsageQuantity": {
                "Amount": "0",
                "Unit": "N/A"
            }
        }
    }
]
Request costs for a space for a date range

GET /v1/cost/{account}/spaces/{spaceid}?start=2021-04-01&end=2021-05-31

Response
[
    {
        "Estimated": false,
        "Groups": [],
        "TimePeriod": {
            "End": "2021-05-01",
            "Start": "2021-04-01"
        },
        "Total": {
            "BlendedCost": {
                "Amount": "8.1395432009",
                "Unit": "USD"
            },
            "UnblendedCost": {
                "Amount": "8.1395437889",
                "Unit": "USD"
            },
            "UsageQuantity": {
                "Amount": "37095.8855728516",
                "Unit": "N/A"
            }
        }
    },
    {
        "Estimated": true,
        "Groups": [],
        "TimePeriod": {
            "End": "2021-05-31",
            "Start": "2021-05-01"
        },
        "Total": {
            "BlendedCost": {
                "Amount": "3.2187441928",
                "Unit": "USD"
            },
            "UnblendedCost": {
                "Amount": "3.2187483458",
                "Unit": "USD"
            },
            "UsageQuantity": {
                "Amount": "8466.9944576915",
                "Unit": "N/A"
            }
        }
    }
]
Request costs for a space by date range and grouped by a dimension

GET /v1/cost/{account}/spaces/{spaceid}?start=2021-04-01&end=2021-05-31&groupby=INSTANCE_TYPE_FAMILY

Supported default 'groupby' values are AZ, INSTANCE_TYPE, LINKED_ACCOUNT, OPERATION, PURCHASE_TYPE, SERVICE, USAGE_TYPE, PLATFORM, TENANCY, RECORD_TYPE, LEGAL_ENTITY_NAME, DEPLOYMENT_OPTION, DATABASE_ENGINE, CACHE_ENGINE, INSTANCE_TYPE_FAMILY, REGION, BILLING_ENTITY, RESERVATION_ID, SAVINGS_PLANS_TYPE, SAVINGS_PLAN_ARN, OPERATING_SYSTEM. In addition, the custom RESOURCE_NAME 'groupby' is supported using the Name tag.

Response
[
    {
        "Estimated": false,
        "Groups": [
            {
                "Keys": [
                    "NoInstanceTypeFamily"
                ],
                "Metrics": {
                    "BlendedCost": {
                        "Amount": "2.3325985365",
                        "Unit": "USD"
                    },
                    "UnblendedCost": {
                        "Amount": "2.3325991245",
                        "Unit": "USD"
                    },
                    "UsageQuantity": {
                        "Amount": "36869.1647378516",
                        "Unit": "N/A"
                    }
                }
            },
            {
                "Keys": [
                    "m5a"
                ],
                "Metrics": {
                    "BlendedCost": {
                        "Amount": "4.008678362",
                        "Unit": "USD"
                    },
                    "UnblendedCost": {
                        "Amount": "4.008678362",
                        "Unit": "USD"
                    },
                    "UsageQuantity": {
                        "Amount": "39.124167",
                        "Unit": "N/A"
                    }
                }
            },
            {
                "Keys": [
                    "t3"
                ],
                "Metrics": {
                    "BlendedCost": {
                        "Amount": "0.0238622176",
                        "Unit": "USD"
                    },
                    "UnblendedCost": {
                        "Amount": "0.0238622176",
                        "Unit": "USD"
                    },
                    "UsageQuantity": {
                        "Amount": "0.573611",
                        "Unit": "N/A"
                    }
                }
            },
            {
                "Keys": [
                    "t3a"
                ],
                "Metrics": {
                    "BlendedCost": {
                        "Amount": "1.7744040848",
                        "Unit": "USD"
                    },
                    "UnblendedCost": {
                        "Amount": "1.7744040848",
                        "Unit": "USD"
                    },
                    "UsageQuantity": {
                        "Amount": "187.023057",
                        "Unit": "N/A"
                    }
                }
            }
        ],
        "TimePeriod": {
            "End": "2021-05-01",
            "Start": "2021-04-01"
        },
        "Total": {}
    },
    {
        "Estimated": true,
        "Groups": [
            {
                "Keys": [
                    "NoInstanceTypeFamily"
                ],
                "Metrics": {
                    "BlendedCost": {
                        "Amount": "3.1292908776",
                        "Unit": "USD"
                    },
                    "UnblendedCost": {
                        "Amount": "3.1292950306",
                        "Unit": "USD"
                    },
                    "UsageQuantity": {
                        "Amount": "8466.2786246915",
                        "Unit": "N/A"
                    }
                }
            },
            {
                "Keys": [
                    "c5"
                ],
                "Metrics": {
                    "BlendedCost": {
                        "Amount": "0.06346661",
                        "Unit": "USD"
                    },
                    "UnblendedCost": {
                        "Amount": "0.06346661",
                        "Unit": "USD"
                    },
                    "UsageQuantity": {
                        "Amount": "0.373333",
                        "Unit": "N/A"
                    }
                }
            },
            {
                "Keys": [
                    "m5a"
                ],
                "Metrics": {
                    "BlendedCost": {
                        "Amount": "0.025561092",
                        "Unit": "USD"
                    },
                    "UnblendedCost": {
                        "Amount": "0.025561092",
                        "Unit": "USD"
                    },
                    "UsageQuantity": {
                        "Amount": "0.297222",
                        "Unit": "N/A"
                    }
                }
            },
            {
                "Keys": [
                    "t3a"
                ],
                "Metrics": {
                    "BlendedCost": {
                        "Amount": "0.0004256132",
                        "Unit": "USD"
                    },
                    "UnblendedCost": {
                        "Amount": "0.0004256132",
                        "Unit": "USD"
                    },
                    "UsageQuantity": {
                        "Amount": "0.045278",
                        "Unit": "N/A"
                    }
                }
            }
        ],
        "TimePeriod": {
            "End": "2021-05-31",
            "Start": "2021-05-01"
        },
        "Total": {}
    }
]

Budget Usage

Create Budgets Alerts
Request

POST /v1/cost/{account}/spaces/{spaceid}/budgets

{
    "Amount": "10",
    "TimeUnit": "MONTHLY",
    "Alerts": [
        {
            "ComparisonOperator": "GREATER_THAN",
            "NotificationType": "FORECASTED",
            "Threshold": 100,
            "ThresholdType": "PERCENTAGE",
            "Addresses": ["some.user@yale.edu", "some.other@yale.edu"]
        }
    ]
}
Response
{
    "Amount": "10",
    "Name": "spintst-000028-MONTHLY-01",
    "TimeUnit": "MONTHLY",
    "Alerts": [
        {
            "ComparisonOperator": "GREATER_THAN",
            "NotificationState": "",
            "NotificationType": "FORECASTED",
            "Threshold": 100,
            "ThresholdType": "PERCENTAGE",
            "Addresses": ["some.user@yale.edu", "some.other@yale.edu"]
        }
    ]
}
List Budgets Alerts

GET /v1/cost/{account}/spaces/{spaceid}/budgets

Response
[
    "spintst-000028-MONTHLY-01"
]
GET details about a Budgets Alert

GET /v1/cost/{account}/spaces/{spaceid}/budgets/{budget}

Response
{
    "Amount": "10",
    "Name": "spintst-000028-MONTHLY-01",
    "TimeUnit": "MONTHLY",
    "Alerts": [
        {
            "ComparisonOperator": "GREATER_THAN",
            "NotificationState": "",
            "NotificationType": "FORECASTED",
            "Threshold": 100,
            "ThresholdType": "PERCENTAGE",
            "Addresses": ["some.user@yale.edu", "some.other@yale.edu"]
        }
    ]
}
Delete Budgets Alert

DELETE /v1/cost/{account}/spaces/{spaceid}/budgets/{budget}

Response
"OK"

Compute Optimizer recommendations

Get recommendations for an instance id

GET /v1/cost/{account}/spaces/{space}/instances/{id}/optimizer

Empty response

Empty response is usually a result of not enough data for a recommendation.

[]
Response for optimized instance
[
    {
        "AccountId": "1234567890",
        "CurrentInstanceType": "t3a.medium",
        "Finding": "OPTIMIZED",
        "FindingReasonCodes": [],
        "InstanceArn": "arn:aws:ec2:us-east-1:1234567890:instance/i-1122334455",
        "InstanceName": "best.bobs.edu",
        "LastRefreshTimestamp": "2021-06-16T18:21:42.595Z",
        "LookBackPeriodInDays": 14,
        "RecommendationOptions": [
            {
                "InstanceType": "t3a.medium",
                "PerformanceRisk": 1,
                "PlatformDifferences": [],
                "ProjectedUtilizationMetrics": [
                    {
                        "Name": "CPU",
                        "Statistic": "MAXIMUM",
                        "Value": 82.72781814545695
                    }
                ],
                "Rank": 1
            }
        ],
        "RecommendationSources": [
            {
                "RecommendationSourceArn": "arn:aws:ec2:us-east-1:1234567890:instance/i-1122334455",
                "RecommendationSourceType": "Ec2Instance"
            }
        ],
        "UtilizationMetrics": [
            {
                "Name": "CPU",
                "Statistic": "MAXIMUM",
                "Value": 82.72781814545695
            },
            {
                "Name": "EBS_READ_OPS_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 1127.2433333333333
            },
            {
                "Name": "EBS_WRITE_OPS_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 74.5
            },
            {
                "Name": "EBS_READ_BYTES_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 29981835.9375
            },
            {
                "Name": "EBS_WRITE_BYTES_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 1845286.4583333333
            },
            {
                "Name": "NETWORK_IN_BYTES_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 10330.019722222223
            },
            {
                "Name": "NETWORK_OUT_BYTES_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 507890.63972222223
            },
            {
                "Name": "NETWORK_PACKETS_IN_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 39.420111111111105
            },
            {
                "Name": "NETWORK_PACKETS_OUT_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 328.95061111111113
            }
        ]
    }
]
Response with recommendations
[
    {
        "AccountId": "1234567890",
        "CurrentInstanceType": "m4.xlarge",
        "Finding": "UNDER_PROVISIONED",
        "FindingReasonCodes": [
            "CPUOverprovisioned",
            "EBSIOPSOverprovisioned",
            "EBSThroughputUnderprovisioned"
        ],
        "InstanceArn": "arn:aws:ec2:us-east-1:1234567890:instance/i-0987654321",
        "InstanceName": "knowledge.bobs.edu",
        "LastRefreshTimestamp": "2021-06-16T18:53:25.669Z",
        "LookBackPeriodInDays": 14,
        "RecommendationOptions": [
            {
                "InstanceType": "t3.xlarge",
                "PerformanceRisk": 3,
                "PlatformDifferences": [
                    "NetworkInterface",
                    "Hypervisor",
                    "StorageInterface"
                ],
                "ProjectedUtilizationMetrics": [
                    {
                        "Name": "CPU",
                        "Statistic": "MAXIMUM",
                        "Value": 44.64812085482684
                    }
                ],
                "Rank": 1
            },
            {
                "InstanceType": "m5.xlarge",
                "PerformanceRisk": 1,
                "PlatformDifferences": [
                    "NetworkInterface",
                    "Hypervisor",
                    "StorageInterface"
                ],
                "ProjectedUtilizationMetrics": [
                    {
                        "Name": "CPU",
                        "Statistic": "MAXIMUM",
                        "Value": 44.64812085482684
                    }
                ],
                "Rank": 2
            },
            {
                "InstanceType": "m4.xlarge",
                "PerformanceRisk": 1,
                "PlatformDifferences": [],
                "ProjectedUtilizationMetrics": [
                    {
                        "Name": "CPU",
                        "Statistic": "MAXIMUM",
                        "Value": 55.5084745762712
                    }
                ],
                "Rank": 3
            }
        ],
        "RecommendationSources": [
            {
                "RecommendationSourceArn": "arn:aws:ec2:us-east-1:1234567890:instance/i-0987654321",
                "RecommendationSourceType": "Ec2Instance"
            }
        ],
        "UtilizationMetrics": [
            {
                "Name": "CPU",
                "Statistic": "MAXIMUM",
                "Value": 55.5084745762712
            },
            {
                "Name": "EBS_READ_OPS_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 753.56
            },
            {
                "Name": "EBS_WRITE_OPS_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 917.98
            },
            {
                "Name": "EBS_READ_BYTES_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 89428489.58333333
            },
            {
                "Name": "EBS_WRITE_BYTES_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 83148230.79666667
            },
            {
                "Name": "NETWORK_IN_BYTES_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 414224.09777777776
            },
            {
                "Name": "NETWORK_OUT_BYTES_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 29260.118055555555
            },
            {
                "Name": "NETWORK_PACKETS_IN_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 177.9728888888889
            },
            {
                "Name": "NETWORK_PACKETS_OUT_PER_SECOND",
                "Statistic": "MAXIMUM",
                "Value": 35.34688888888889
            }
        ]
    }
]

Inventory Usage

The inventory endpoint returns resources belonging to a space by tag. It uses the resourcegroupstaggingapi and also parses the ARN to determine the resource type information. Since the resource prefix is inconsistent, it shouldn't be relied upon for categorization, but the service should be accurate.

Request

GET /v1/inventory/{account}/spaces/{spaceid}

Response
[
    {
        "name": "spintst-000a16-TestFS",
        "arn": "arn:aws:elasticfilesystem:us-east-1:1234567890:file-system/fs-aaaabbbb11",
        "partition": "aws",
        "service": "elasticfilesystem",
        "region": "us-east-1",
        "account_id": "1234567890",
        "resource": "file-system/fs-aaaabbbb11"
    },
    {
        "name": "",
        "arn": "arn:aws:elasticloadbalancing:us-east-1:1234567890:targetgroup/testTargetGroup-HTTP80/abcdefg12",
        "partition": "aws",
        "service": "elasticloadbalancing",
        "region": "us-east-1",
        "account_id": "1234567890",
        "resource": "targetgroup/testTargetGroup-HTTP80/abcdefg12"
    },
    {
        "name": "spintst-000028",
        "arn": "arn:aws:logs:us-east-1:1234567890:log-group:spintst-000028",
        "partition": "aws",
        "service": "logs",
        "region": "us-east-1",
        "account_id": "1234567890",
        "resource": "log-group:spintst-000028"
    },
    {
        "name": "spintst-000b67-webServiceTest",
        "arn": "arn:aws:secretsmanager:us-east-1:1234567890:secret:spinup/sstst/spintst-000028/spintst-000b67-webServiceTest-api-cred-ibdIk7",
        "partition": "aws",
        "service": "secretsmanager",
        "region": "us-east-1",
        "account_id": "1234567890",
        "resource": "secret:spinup/sstst/spintst-000028/spintst-000b67-webServiceTest-api-cred-ibdIk7"
    }
]

Metrics Usage

Get Cloudwatch metrics widgets URL from S3 for an instance ID

This will get the passed metric(s) for the passed instance ID or container cluster/service in a image/png graph for the past 1 day by default, cache it in S3 and return the URL. URLs are cached in the API for 5 minutes, the images should be purged from the S3 cache on a schedule. It's also possible to pass the height, width, start time, end time and period (e. 300s for 300 seconds, 5m for 5 minutes). Query parameters must follow the CloudWatch Metric Widget Structure.

Documentation on Cloudwatch metrics
Get a list of metrics per AWS service
aws --region us-east-1 cloudwatch list-metrics --namespace AWS/RDS |grep MetricName |sort| uniq
Helpful references
Example metrics request
{
  "metrics": [
    [ "AWS/ECS", "CPUUtilization", "ClusterName", "spinup-000393", "ServiceName", "spinup-0010a3-testsvc" ]
  ],
  "stat": "Average"
  "period": 300,
  "start": "-P1D",
  "end": "PT0H"
}
Request
GET /v1/metrics/{account}/instances/{id}/graph?metric={metric1}[&metric={metric2}&....]
GET /v1/metrics/{account}/instances/{id}/graph?metric={metric1}[&metric={metric2}&start={start}&end={end}&period={period}]

GET /v1/metrics/{account}/clusters/{cluster}/services/{service}/graph?metric={metric1}[&metric={metric2}&....]
GET /v1/metrics/{account}/clusters/{cluster}/services/{service}/graph?metric={metric1}[&metric={metric2}&start={start}&end={end}&period={period}]
Response
{
    "ImageURL": "https://s3.amazonaws.com/sometestbucket/aabbccddeeff-Y3_yCKckBrkUNt3Lh4LzXBFeLXBY5IP1oUED4hyY0cdKneYelKv-xlV7K2F_d0ccwp677A=="
}

with an image like this

WidgetExample

Image Caching

When image urls are returned for metrics graph data, they are cached in the image cache. The default implementation of this cache is an S3 bucket where the URLs are returned in the response (and cached in the data cache).

Data Caching

AWS Cost Explorer data and metrics graph image url is cached (using go-cache). The cache TTLs are configurable via config.json: CacheExpireTime and CachePurgeTime. The cache can also be purged via daemon restart.

Authentication

Authentication is accomplished via a pre-shared key (hashed string). This is done via the X-Auth-Token header.

Author

E Camden Fisher camden.fisher@yale.edu

License

GNU Affero General Public License v3.0 (GNU AGPLv3)
Copyright (c) 2019 Yale University

Documentation

The Go Gopher

There is no documentation for this package.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.