README ¶
AWS S3 Exporter for OpenTelemetry Collector
Status | |
---|---|
Stability | alpha: traces, metrics, logs |
Distributions | contrib, observiq, sumo |
Issues | |
Code Owners | @atoulme, @pdelewski |
Schema supported
This exporter targets to support proto/json format.
Exporter Configuration
The following exporter configuration parameters are supported.
Name | Description | Default |
---|---|---|
region |
AWS region. | "us-east-1" |
s3_bucket |
S3 bucket | |
s3_prefix |
prefix for the S3 key (root directory inside bucket). | |
s3_partition |
time granularity of S3 key: hour or minute | "minute" |
file_prefix |
file prefix defined by user | |
marshaler |
marshaler used to produce output data | otlp_json |
endpoint |
overrides the endpoint used by the exporter instead of constructing it from region and s3_bucket |
Marshaler
Marshaler determines the format of data sent to AWS S3. Currently, the following marshalers are implemented:
otlp_json
(default): the OpenTelemetry Protocol format, represented as json.sumo_ic
: the Sumo Logic Installed Collector Archive format. This format is supported only for logs.
Example Configuration
Following example configuration defines to store output in 'eu-central' region and bucket named 'databucket'.
exporters:
awss3:
s3uploader:
region: 'eu-central-1'
s3_bucket: 'databucket'
s3_prefix: 'metric'
s3_partition: 'minute'
Logs and traces will be stored inside 'databucket' in the following path format.
metric/year=XXXX/month=XX/day=XX/hour=XX/minute=XX
AWS Credential Configuration
This exporter follows default credential resolution for the aws-sdk-go.
Follow the guidelines for the credential configuration.
Documentation ¶
Overview ¶
Package awss3exporter stores OpenTelemetry data as an AWS S3 exporter.
Index ¶
Constants ¶
const ( SourceCategoryKey = "_sourceCategory" SourceHostKey = "_sourceHost" SourceNameKey = "_sourceName" )
Variables ¶
var (
ErrUnknownMarshaler = errors.New("unknown marshaler")
)
Functions ¶
func NewMarshaler ¶
func NewMarshaler(mType MarshalerType, logger *zap.Logger) (marshaler, error)
Types ¶
type Config ¶
type Config struct { S3Uploader S3UploaderConfig `mapstructure:"s3uploader"` MarshalerName MarshalerType `mapstructure:"marshaler"` FileFormat string `mapstructure:"file_format"` }
Config contains the main configuration options for the s3 exporter
type MarshalerType ¶ added in v0.76.1
type MarshalerType string
const ( OtlpJSON MarshalerType = "otlp_json" SumoIC MarshalerType = "sumo_ic" )
type S3UploaderConfig ¶
type S3UploaderConfig struct { Region string `mapstructure:"region"` S3Bucket string `mapstructure:"s3_bucket"` S3Prefix string `mapstructure:"s3_prefix"` S3Partition string `mapstructure:"s3_partition"` FilePrefix string `mapstructure:"file_prefix"` Endpoint string `mapstructure:"endpoint"` }
S3UploaderConfig contains aws s3 uploader related config to controls things like bucket, prefix, batching, connections, retries, etc.