carbonzipper

README

carbonzipper: carbonserver proxy for graphite-web

DEPRECATED carbonzipper was fully merged with carbonapi and all new fixes, etc are now buildable from carbonapi/cmd/carbonzipper. This repo is available for historical purposes, no future changes will be made to master branch. Minor bugfixes might still be commited to 0.x branch which represents carbonzipper state at version 0.74.

We are using to host our packages!

Please note that since carbonapi 0.8 it's no longer needed to run separate carbonzipper. This repo still contains a buildable daemon, but it's mostly for compatibility reasons and for those who don't need carbonapi functionality.

CarbonZipper is the central part of a replacement graphite storage stack. It proxies requests from graphite-web to a cluster of carbon storage backends. Previous versions (available in the git history) were able to talk to python carbon stores, but the current version requires the use of go-carbon or graphite-clickhouse.

Installation

Stable versions: Stable repo

Autobuilds (master, might be unstable): Autobuild repo

General information

Configuration is done via a YAML file loaded at startup. The only required field is the list of carbonserver backends to connect to.

Other pieces of the stack are:

• carbonapi
• carbonmem
• carbonsearch

For an overview of the stack and how the pieces fit together, watch Graphite@Scale or How to store millions metrics per second from FOSDEM '17 or Graphite@Scale or How to Store Millions of metrics per Second from SRECon17 Asia.

Requirements

carbonzipper requires Go 1.8+ to build. It's recommended to always use latest stable.

CarbonZipper uses protobuf-based protocol to talk with underlying storages. For current version the compatibility list is:

1. carbonzipper >= 0.50
2. go-carbon >= 0.9.0 (Note: you need to enable carbonserver in go-carbon).
3. carbonserver@master (Note: you should probably switch to go-carbon in that case).
4. graphite-clickhouse any. That's alternative storage that doesn't use Whisper. Limitations: /info handler won't work properly.
5. carbonapi >= 0.5. Note: we are not sure if there is any point in running carbonzipper over carbonapi at this moment.

Changes and versioning

Version policy - all the versions we run in production is taged.

In case change will require simultanious upgrade of different components, it will be stated in Upgrading notes below.

Also we will try to maintain backward compatibility from down to up.

For example with protobuf2 -> protobuf3 migration - carbonzipper can still send results to older versions of carbonapi, but can't get results from older versions of carbonserver/go-carbon.

See CHANGES.md

Upgrading to 0.60 from 0.50 or earlier

Starting from 0.60, carbonzipper will be able to talk only with storages compatible with protobuf3.

At this moment (0.60) it's only go-carbon, starting from commit ee2bc24 (post 0.9.1)

Carbonzipper can still return results in protobuf and compatibility won't be removed at least until Summer 2017.

If you want to upgrade, the best option is to do follwing steps:

1. Migrate to go-carbon post 0.9.1 release. (note: carbonserver isn't compatible with this version of zipper)
2. Migrate to carbonsearch 0.16.0 (if you are using any)
3. Upgrade carbonzipper to 0.60 or newer.
4. Upgrade carbonapi to 0.6.0 (commit 119e346 or newer) (optional, but advised)

Acknowledgement

This program was originally developed for Booking.com. With approval from Booking.com, the code was generalised and published as Open Source on github, for which the author would like to express his gratitude.

License

This code is licensed under the MIT license.

Contact

If you have questions or problems there are two ways to contact us:

1. Open issue on a github page
2. #zipperstack on gophers slack