README
¶
ElasticSearch Support
This provides a storage backend for Jaeger using Elasticsearch. More information is available on the Jaeger documentation website.
Indices
Indices will be created depending on the spans timestamp. i.e., a span with
a timestamp on 2017/04/21 will be stored in an index named jaeger-2017-04-21.
ElasticSearch also has no support for TTL, so there exists a script ./esCleaner.py
that deletes older indices automatically. The Elastic Curator
can also be used instead to do a similar job.
Using ./esCleaner.py
The script is using python3. All dependencies can be installed with: python3 -m pip install elasticsearch elasticsearch-curator.
Parameters:
- Environment variable TIMEOUT that sets the timeout in seconds for indices deletion (default: 120)
- Optional environment variable ES_USERNAME and ES_PASSWORD
- a number that will delete any indices older than that number in days
- ElasticSearch hostnames
- Example usage:
TIMEOUT=120 ./esCleaner.py 4 localhost:9200
Timestamps
Because ElasticSearch's Date datatype has only millisecond granularity and Jaeger
requires microsecond granularity, Jaeger spans' StartTime is saved as a long type.
The conversion is done automatically.
Nested fields (tags)
Tags are nested fields in the
ElasticSearch schema used for Jaeger. This allows for better search capabilities and data retention. However, because
ElasticSearch creates a new document for every nested field, there is currently a limit of 50 nested fields per document.
Shards and Replicas
Number of shards and replicas per index can be specified as parameters to the writer and/or through configs under
./pkg/es/config/config.go. If not specified, it defaults to ElasticSearch defaults: 5 shards and 1 replica.
This article goes into more information
about choosing how many shards should be chosen for optimization.
Limitations
Tag query over multiple spans
This plugin queries against spans. This means that all tags in a query must lie under the same span for a query to successfully return a trace.
Case-sensitivity
Queries are case-sensitive. For example, if a document with service name ABC is searched using a query abc,
the document will not be retrieved.
Testing
To locally test the ElasticSearch storage plugin,
- have ElasticSearch running on port 9200
- run
STORAGE=es make storage-integration-testin the top folder.
All integration tests also run on pull request via Travis. This integration test is against ElasticSearch v5.4.0.
- The script used in Travis can be found under
./travis/es-integration-test.sh, and that script be run from the top folder to integration test ElasticSearch as well. This script requires Docker to be running.
Adding tests
Integration test framework for storage lie under ../integration.
Add to ../integration/fixtures/traces/*.json and ../integration/fixtures/queries.json to add more
trace cases.
Documentation
¶
Index ¶
- func GetDependenciesMappings(shards, replicas int64, esVersion uint) string
- func GetSpanServiceMappings(shards, replicas int64, esVersion uint) (string, string)
- type Factory
- func (f *Factory) AddFlags(flagSet *flag.FlagSet)
- func (f *Factory) Close() error
- func (f *Factory) CreateArchiveSpanReader() (spanstore.Reader, error)
- func (f *Factory) CreateArchiveSpanWriter() (spanstore.Writer, error)
- func (f *Factory) CreateDependencyReader() (dependencystore.Reader, error)
- func (f *Factory) CreateSpanReader() (spanstore.Reader, error)
- func (f *Factory) CreateSpanWriter() (spanstore.Writer, error)
- func (f *Factory) InitFromOptions(o Options)
- func (f *Factory) InitFromViper(v *viper.Viper)
- func (f *Factory) Initialize(metricsFactory metrics.Factory, logger *zap.Logger) error
- type Options
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetDependenciesMappings ¶
GetDependenciesMappings returns dependencies mappings
Types ¶
type Factory ¶
type Factory struct {
Options *Options
// contains filtered or unexported fields
}
Factory implements storage.Factory for Elasticsearch backend.
func (*Factory) CreateArchiveSpanReader ¶
CreateArchiveSpanReader implements storage.ArchiveFactory
func (*Factory) CreateArchiveSpanWriter ¶
CreateArchiveSpanWriter implements storage.ArchiveFactory
func (*Factory) CreateDependencyReader ¶
func (f *Factory) CreateDependencyReader() (dependencystore.Reader, error)
CreateDependencyReader implements storage.Factory
func (*Factory) CreateSpanReader ¶
CreateSpanReader implements storage.Factory
func (*Factory) CreateSpanWriter ¶
CreateSpanWriter implements storage.Factory
func (*Factory) InitFromOptions ¶
InitFromOptions configures factory from Options struct.
func (*Factory) InitFromViper ¶
InitFromViper implements plugin.Configurable
type Options ¶
type Options struct {
Primary namespaceConfig `mapstructure:",squash"`
// contains filtered or unexported fields
}
Options contains various type of Elasticsearch configs and provides the ability to bind them to command line flag and apply overlays, so that some configurations (e.g. archive) may be underspecified and infer the rest of its parameters from primary.
func NewOptions ¶
NewOptions creates a new Options struct.
func NewOptionsFromConfig ¶
func NewOptionsFromConfig(primary config.Configuration, archive config.Configuration) *Options
NewOptionsFromConfig creates Options from primary and archive config
func (*Options) Get ¶
func (opt *Options) Get(namespace string) *config.Configuration
Get returns auxiliary named configuration.
func (*Options) GetPrimary ¶
func (opt *Options) GetPrimary() *config.Configuration
GetPrimary returns primary configuration.
func (*Options) InitFromViper ¶
InitFromViper initializes Options with properties from viper
Source Files
Directories