index-pattern-synchro

README

Index Pattern Synchro - シンクロ

Introduction

Index Pattern synchro is tool for synchronize elasticsearch indices with Kibana index pattern automatically.

This tool will be the best for EFK implementation that you have many index and need you are too lazy to separate index pattern in Kibana. For example:

• log-nginx-2020.05.01
• log-nginx-2020.05.02
• log-nginx-2020.05.03
• log-tomcat-2020.05.01
• log-tomcat-2020.05.02
• log-app-2020.05.01

it will be generated an index pattern each of them so it will be:

• log-nginx*
• log-tomcat*
• log-app*

Of course this tool support for multiple Kibana Spaces or Opendistro Tenants!

Configuration

This app need config.yaml to run, you can generate the config using the command line tool.

Initialize or Generate the config file

To create config.yaml in current directory.

index-pattern-synchro init

To create config.yaml in specific directory, use the flag -o or --output.

index-pattern-synchro init -o /opt/index-pattern-synchro/config.yaml

Configuration example is also in the example/config.yaml.

How config file works

This tool will discover the config file in these directories, the priority is from top to bottom.

• /etc/index-pattern-synchro/config.yaml
• /opt/index-pattern-synchro/config.yaml
• $HOME/index-pattern-synchro/config.yaml (user home directory)
• ./config.yaml (current directory)

An explanation of each field of the config can be read in config markdown.

Build

Build Binary

go build

Docker Build image

docker built -f build/Dockerfile -t index-pattern-synchro:latest .

Docker Run Container

docker run --rm --name index-pattern-synchro --mount type=bind,source="$(pwd)"/config.yaml,target=/app/config.yaml index-pattern-synchro

Prebuild docker images exist in my docker hub here.

Run

For running this app, you just need to execute the binary or run the container.

index-pattern-synchro run

You can also view the command line help, by adding the flag on it.

index-pattern-synchro -h
index-pattern-synchro run -h

Tips:

• Run the app with auto restart, so if the app exited it will be auto recovered.

Documentation

Overview

Copyright © 2021 Misbahul Ardani <misbahulard@gmail.com>

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.