Datadog Exporter
This exporter sends metric and trace data to Datadog. For environment specific setup instructions visit the Datadog Documentation.
Please review the Collector's security
documentation,
which contains recommendations on securing sensitive information such as the
API key required by this exporter.
Note: The metrics exporter is not considered stable yet and may suffer breaking changes.
Configuration
The only required setting is a Datadog API key.
datadog:
api:
key: "<API key>"
To send data to the Datadog EU site, set the api.site
parameter to datadoghq.eu
:
datadog:
api:
key: "<API key>"
site: datadoghq.eu
If you want to use the OpenTelemetry Span Name as the Datadog Resource Name you can set the span_name_as_resource_name
configuration option to true
(default is false
). For more info on the downsides of this option check this issue.
datadog:
api:
key: "<API key>"
traces:
span_name_as_resource_name: true
The hostname, environment, service and version can be set in the configuration for unified service tagging.
The exporter will try to retrieve a hostname following the OpenTelemetry semantic conventions if there is one available.
See the sample configuration files under the example
folder for other available options, as well as an example K8s Manifest.
This exporter also supports the exporterhelper
queuing, retry and timeout settings documented here.
Retry settings will only affect metrics.
Trace exporter
Important Pipeline Setup Details
This exporter assumes a pipeline using the datadog exporter also includes a batch processor configured with the following:
- a
timeout
setting of 10s
(10 seconds).
Please make sure to include this processor in your pipeline. An example pipeline can be found below.
A batch representing 10 seconds of traces is a constraint of Datadog's API Intake for Trace Related Statistics. Without this setting, trace related metrics including .hits
.errors
and .duration
for different services and service resources may be inaccurate over periods of time.
Example:
receivers:
examplereceiver:
processors:
batch:
timeout: 10s
exporters:
datadog/api:
hostname: customhostname
host_metadata:
tags: [example:tag]
api:
key: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
site: datadoghq.eu
service:
pipelines:
traces:
receivers: [examplereceiver]
processors: [batch]
exporters: [datadog/api]
Span Events
Please Note: Currently Span Events are extracted and added to Spans as Json on the Datadog Span Tag events
.
Recommended Samplers
While the OpenTelemetry Specification for Sampling remains undecided and in active development, the Datadog Exporter currently supports two sampling approaches to ensure accuracy in generated Trace Stats payloads for details such as hits, errors, and latency within Datadog.
-
ParentBased(root=AlwaysOn)
: This is the default SDK sampler, and emits 100% of Spans. Depending on your use case, in high volume environments this approach may result in higher than expected resource consumption.
-
DatadogTraceIdRatioBased
, a customized TraceIdRatioBased
sampler used as the root of a ParentBased
Sampler, ex: ParentBased(root=DatadogTraceIdRatioBased)
: In certain high volume environments, an AlwaysOn
Sampler may not be feasible. In these cases, Datadog supports a global, head based, probability sampling approach for OpenTelemetry by including a custom Datadog TraceIdRatioBased
Sampler used as the root in a ParentBased
composite Sampler. This sampler ensures that the global probability
sampling rate information is appended to Sampled Spans as an attribute _sample_rate
. This attribute has semantic significance within Datadog and ensures that the Datadog Exporter can correctly "upscale" generated Trace Stats payloads to ensure accurate hits, errors, and latency statistics that include dropped spans. For an example of this custom sampler, see the example application in example/example_custom_sampler.
Metric exporter
The metrics exporter does not assume any specific pipeline setup.
There are a number of optional settings for configuring how to send your metrics:
Option name |
Description |
Default |
send_monotonic_counter |
Cumulative monotonic metrics are sent as deltas between successive measurements. Disable this flag to send get the raw, monotonically increasing value. |
true |
delta_ttl |
Maximum number of seconds values from cumulative monotonic metrics are kept in memory. |
3600 |
summaries::mode |
Mode for summaries. Valid values are noquantiles (no quantiles metrics) and gauges (one metric per quantile). |
gauges |
histograms::mode |
Mode for histograms. Valid values are nobuckets (no bucket metrics), counters (one metric per bucket) and distributions (send as Datadog distributions, recommended). |
distributions |
histograms::send_count_sum_metrics |
Whether to report sum and count for histograms as separate metrics. |
false |