README
¶
AWS S3 Receiver
Status | |
---|---|
Stability | development: traces, metrics, logs |
Distributions | [] |
Issues | |
Code Owners | @atoulme, @adcharre |
Overview
Receiver for retrieving trace previously stored in S3 by the AWS S3 Exporter.
Configuration
The following exporter configuration parameters are supported.
Name | Description | Default | Required |
---|---|---|---|
starttime |
The time at which to start retrieving data. | Required | |
endtime |
The time at which to stop retrieving data. | Required | |
s3downloader: |
|||
region |
AWS region. | "us-east-1" | Optional |
s3_bucket |
S3 bucket | Required | |
s3_prefix |
prefix for the S3 key (root directory inside bucket). | Required | |
s3_partition |
time granularity of S3 key: hour or minute | "minute" | Optional |
file_prefix |
file prefix defined by user | Optional | |
endpoint |
overrides the endpoint used by the exporter instead of constructing it from region and s3_bucket |
Optional | |
endpoint_partition_id |
partition id to use if endpoint is specified. |
"aws" | Optional |
s3_force_path_style |
set this to true to force the request to use path-style addressing |
false | Optional |
encodings: |
An array of entries with the following properties: | Optional | |
extension |
Extension to use for decoding a key with a matching suffix. | Required | |
suffix |
Key suffix to match against. | Required |
Time format for starttime
and endtime
The starttime
and endtime
fields are used to specify the time range for which to retrieve data.
The time format is either YYYY-MM-DD HH:MM
or simply YYYY-MM-DD
, in which case the time is assumed to be 00:00
.
Encodings
By default, the receiver understands the following encodings:
- otlp_json (OpenTelemetry Protocol format represented as json) with a suffix of
.json
- otlp_proto (OpenTelemetry Protocol format represented as Protocol Buffers) with a suffix of
.binpb
The encodings
options allows you to specify Encoding Extensions to use to decode keys with matching suffixes.
Example Configuration
extension:
# example of text encoding extension
text_encoding:
encoding: utf8
marshaling_separator: "\n"
unmarshaling_separator: "\r?\n"
receivers:
awss3:
starttime: "2024-01-01 01:00"
endtime: "2024-01-02"
s3downloader:
region: "us-west-1"
s3_bucket: "mybucket"
s3_prefix: "trace"
s3_partition: "minute"
encodings:
- extension: text_encoding
suffix: ".txt"
Documentation
¶
Overview ¶
Package awss3receiver implements a receiver that can be used by the Opentelemetry collector to retrieve traces previously stored in S3 by the AWS S3 Exporter.
Index ¶
Constants ¶
const ( S3PartitionMinute = "minute" S3PartitionHour = "hour" )
Variables ¶
This section is empty.
Functions ¶
func NewFactory ¶
Types ¶
type Config ¶
type Config struct { S3Downloader S3DownloaderConfig `mapstructure:"s3downloader"` StartTime string `mapstructure:"starttime"` EndTime string `mapstructure:"endtime"` Encodings []Encoding `mapstructure:"encodings"` }
Config defines the configuration for the file receiver.
type GetObjectAPI ¶ added in v0.101.0
type GetObjectAPI interface {
GetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error)
}
type ListObjectsAPI ¶ added in v0.101.0
type ListObjectsAPI interface {
NewListObjectsV2Paginator(params *s3.ListObjectsV2Input) ListObjectsV2Pager
}
type ListObjectsV2Pager ¶ added in v0.101.0
type S3DownloaderConfig ¶
type S3DownloaderConfig struct { Region string `mapstructure:"region"` S3Bucket string `mapstructure:"s3_bucket"` S3Prefix string `mapstructure:"s3_prefix"` S3Partition string `mapstructure:"s3_partition"` FilePrefix string `mapstructure:"file_prefix"` Endpoint string `mapstructure:"endpoint"` EndpointPartitionID string `mapstructure:"endpoint_partition_id"` S3ForcePathStyle bool `mapstructure:"s3_force_path_style"` }
S3DownloaderConfig contains aws s3 downloader related config to controls things like bucket, prefix, batching, connections, retries, etc.