sybil

README

Sybil is an append only analytics datastore with no up front table schema requirements; just log JSON records to a table and run queries. Written in Go, sybil is designed for fast full table scans of multi-dimensional data on a single machine.

more documentation is available on the wiki and in the repo.

if sybil by itself is uninteresting (who wants to run command line queries, anyways?), sybil is a supported backend for snorkel

advantages

• Easy to setup and get data inside sybil - just pipe JSON on stdin to sybil
• Supports histograms (and percentiles), standard deviations and time series roll ups
• Runs fast full table queries (performance notes)
• Lower disk usage through per column compression schemes
• Serverless design with controlled memory usage
• Per table retention policies (specify max age and/or size of tables)
• Per block query cache (optional) that avoids recomputation

disadvantages

• JOINS not supported
• No UPDATE operation on data - only writes
• No sharding

installation

go get github.com/logv/sybil

additional information

• command line tour
• want to contribute?
• notes on performance
• abadi survey of column stores