Derivative Aggregator Plugin
The Derivative Aggregator Plugin estimates the derivative for all fields of the
aggregated metrics.
Time Derivatives
In its default configuration it determines the first and last measurement of
the period. From these measurements the time difference in seconds is
calculated. This time difference is than used to divide the difference of each
field using the following formula:
field_last - field_first
derivative = --------------------------
time_difference
For each field the derivative is emitted with a naming pattern
<fieldname>_rate
.
Custom Derivation Variable
The plugin supports to use a field of the aggregated measurements as derivation
variable in the denominator. This variable is assumed to be a monotonically
increasing value. In this feature the following formula is used:
field_last - field_first
derivative = --------------------------------
variable_last - variable_first
Make sure the specified variable is not filtered and exists in the metrics passed to this aggregator!
When using a custom derivation variable, you should change the suffix
of the derivative name.
See the next section on customizing the derivative name for details.
Customize the Derivative Name
The derivatives generated by the aggregator are named <fieldname>_rate
, i.e. they are composed of the field name and a suffix _rate
.
You can configure the suffix to be used by changing the suffix
parameter.
Roll-Over to next Period
Calculating the derivative for a period requires at least two distinct measurements during that period.
Whether those are available depends on the configuration of the aggregator period
and the agent interval
.
By default the last measurement is used as first measurement in the next
aggregation period. This enables a continuous calculation of the derivative. If
within the next period an earlier timestamp is encountered this measurement will
replace the roll-over metric. A main benefit of this roll-over is the ability to
cope with multiple "quiet" periods, where no new measurement is pushed to the
aggregator. The roll-over will take place at most max_roll_over
times.
Example of Roll-Over
Let us assume we have an input plugin, that generates a measurement with a single metric "test" every 2 seconds.
Let this metric increase the first 10 seconds from 0.0 to 10.0 and then decrease the next 10 seconds form 10.0 to 0.0:
timestamp |
value |
0 |
0.0 |
2 |
2.0 |
4 |
4.0 |
6 |
6.0 |
8 |
8.0 |
10 |
10.0 |
12 |
8.0 |
14 |
6.0 |
16 |
4.0 |
18 |
2.0 |
20 |
0.0 |
To avoid thinking about border values, we consider periods to be inclusive at the start but exclusive in the end.
Using period = "10s"
and max_roll_over = 0
we would get the following aggregates:
timestamp |
value |
aggregate |
explanantion |
0 |
0.0 |
|
|
2 |
2.0 |
|
|
4 |
4.0 |
|
|
6 |
6.0 |
|
|
8 |
8.0 |
|
|
|
|
1.0 |
(8.0 - 0.0) / (8 - 0) |
10 |
10.0 |
|
|
12 |
8.0 |
|
|
14 |
6.0 |
|
|
16 |
4.0 |
|
|
18 |
2.0 |
|
|
|
|
-1.0 |
(2.0 - 10.0) / (18 - 10) |
20 |
0.0 |
|
|
If we now decrease the period with period = 2s
, no derivative could be calculated since there would only one measurement for each period.
The aggregator will emit the log messages Same first and last event for "test", skipping.
.
This changes, if we use max_roll_over = 1
, since now end measurements of a period are taking as start for the next period.
timestamp |
value |
aggregate |
explanantion |
0 |
0.0 |
|
|
2 |
2.0 |
1.0 |
(2.0 - 0.0) / (2 - 0) |
4 |
4.0 |
1.0 |
(4.0 - 2.0) / (4 - 2) |
6 |
6.0 |
1.0 |
(6.0 - 4.0) / (6 - 4) |
8 |
8.0 |
1.0 |
(8.0 - 6.0) / (8 - 6) |
10 |
10.0 |
1.0 |
(10.0 - 8.0) / (10 - 8) |
12 |
8.0 |
-1.0 |
(8.0 - 10.0) / (12 - 10) |
14 |
6.0 |
-1.0 |
(6.0 - 8.0) / (14 - 12) |
16 |
4.0 |
-1.0 |
(4.0 - 6.0) / (16 - 14) |
18 |
2.0 |
-1.0 |
(2.0 - 4.0) / (18 - 16) |
20 |
0.0 |
-1.0 |
(0.0 - 2.0) / (20 - 18) |
The default max_roll_over = 10
allows for multiple periods without measurements either due to configuration or missing input.
There may be a slight difference in the calculation when using max_roll_over
compared to running without.
To illustrate this, let us compare the derivatives for period = "7s"
.
timestamp |
value |
max_roll_over = 0 |
max_roll_over = 1 |
0 |
0.0 |
|
|
2 |
2.0 |
|
|
4 |
4.0 |
|
|
6 |
6.0 |
|
|
|
|
1.0 |
1.0 |
8 |
8.0 |
|
|
10 |
10.0 |
|
|
12 |
8.0 |
|
|
|
|
0.0 |
0.33... |
14 |
6.0 |
|
|
16 |
4.0 |
|
|
18 |
2.0 |
|
|
20 |
0.0 |
|
|
|
|
-1.0 |
-1.0 |
The difference stems from the change of the value between periods, e.g. from 6.0 to 8.0 between first and second period.
Thoses changes are omitted with max_roll_over = 0
but are respected with max_roll_over = 1
.
That there are no more differences in the calculated derivatives is due to the example data, which has constant derivatives in during the first and last period, even when including the gap between the periods.
Using max_roll_over
with a value greater 0 may be important, if you need to detect changes between periods, e.g. when you have very few measurements in a period or quasi-constant metrics with only occasional changes.
Configuration
[[aggregators.derivative]]
## Specific Derivative Aggregator Arguments:
## Configure a custom derivation variable. Timestamp is used if none is given.
# variable = ""
## Suffix to add to the field name for the derivative name.
# suffix = "_rate"
## Roll-Over last measurement to first measurement of next period
# max_roll_over = 10
## General Aggregator Arguments:
## calculate derivative every 30 seconds
period = "30s"
No tags are applied by this aggregator.
Existing tags are passed throug the aggregator untouched.
Example Output
net bytes_recv=15409i,packets_recv=164i,bytes_sent=16649i,packets_sent=120i 1508843640000000000
net bytes_recv=73987i,packets_recv=364i,bytes_sent=87328i,packets_sent=452i 1508843660000000000
net bytes_recv_by_packets_recv=292.89 1508843660000000000
net packets_sent_rate=16.6,bytes_sent_rate=3533.95 1508843660000000000
net bytes_sent_by_packet=292.89 1508843660000000000