skywalking-banyandb

module
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 17, 2023 License: Apache-2.0

README

BanyanDB

Continuous Integration Go Report Card GitHub release GitHub release date GoDoc

BanyanDB, as an observability database, aims to ingest, analyze and store Metrics, Tracing and Logging data. It's designed to handle observability data generated by observability platform and APM system, like Apache SkyWalking etc.

Introduction

BanyanDB, as an observability database, aims to ingest, analyze and store Metrics, Tracing, and Logging data. It's designed to handle observability data generated by Apache SkyWalking. Before BanyanDB emerges, the Databases that SkyWalking adopted are not ideal for the APM data model, especially for saving tracing and logging data. Consequently, There’s room to improve the performance and resource usage based on the nature of SkyWalking data patterns.

The database research community usually uses RUM conjecture to describe how a database access data. BanyanDB combines several access methods to build a comprehensive APM database to balance read cost, update cost, and memory overhead.

Contact us

  • Submit an issue by selecting the BanyanDB component.
  • Mail list: dev@skywalking.apache.org. Mail to dev-subscribe@skywalking.apache.org, follow the reply to subscribe the mail list.
  • Send Request to join SkyWalking slack mail to the mail list(dev@skywalking.apache.org), we will invite you in.
  • For Chinese speaker, send [CN] Request to join SkyWalking slack mail to the mail list(dev@skywalking.apache.org), we will invite you in.
  • Twitter, ASFSkyWalking

Documents

Documents

RoadMap

Client manager
  • gRPC server
  • HTTP server
Distributed manager
  • Sharding
  • Load balance
  • Distributed query optimizer
  • Data queue
Data processor
  • Schema management
  • Time-series abstract layer
  • Stream data processor
  • Measure data processor
  • Property data processor
  • TopNAggregation processor
  • Index processor
  • TTL
  • Cold data processor
  • WAL
Query processor
  • Stream query processor
  • Measure query processor
  • Index reader
  • Streaming pipeline processor(OR and nested querying)
  • Parallel executor
  • Cost-based optimizer
Storage
  • Compaction
  • Merge data files
  • Sparse index
Tools
  • Command-line
  • Webapp

Contributing

For developers who want to contribute to this project, see the Contribution Guide.

License

Apache 2.0 License.

Directories

Path Synopsis
api
common
Package common defines reusable objects for APIs.
 Package common defines reusable objects for APIs.
data
Package data contains data transmission topics.
 Package data contains data transmission topics.
Package banyand implements a executable database server.
 Package banyand implements a executable database server.
cmd/server
Package main implements the executable banyandb server named banyand.
 Package main implements the executable banyandb server named banyand.
dquery
Package dquery implement the distributed query.
 Package dquery implement the distributed query.
kv
Package kv implements a key-value engine.
 Package kv implements a key-value engine.
liaison/grpc
Package grpc implements the gRPC services defined by APIs.
 Package grpc implements the gRPC services defined by APIs.
liaison/http
Package http implements the gRPC gateway.
 Package http implements the gRPC gateway.
measure
Package measure implements a time-series-based storage which is consists of a sequence of data points.
 Package measure implements a time-series-based storage which is consists of a sequence of data points.
metadata
Package metadata implements a Raft-based distributed metadata storage system.
 Package metadata implements a Raft-based distributed metadata storage system.
metadata/embeddedetcd
Package embeddedetcd implements an embedded etcd server.
 Package embeddedetcd implements an embedded etcd server.
metadata/schema
Package schema implements CRUD schema.
 Package schema implements CRUD schema.
observability
Package observability provides metrics, profiling, and etc.
 Package observability provides metrics, profiling, and etc.
query
Package query implement the query module for liaison and other modules to retrieve data.
 Package query implement the query module for liaison and other modules to retrieve data.
queue
Package queue implements the data transmission queue.
 Package queue implements the data transmission queue.
queue/pub
Package pub implements the queue client.
 Package pub implements the queue client.
queue/sub
Package sub implements the queue server.
 Package sub implements the queue server.
stream
Package stream implements a time-series-based storage which is consists of a sequence of element.
 Package stream implements a time-series-based storage which is consists of a sequence of element.
tsdb
Package tsdb implements a time-series-based storage engine.
 Package tsdb implements a time-series-based storage engine.
tsdb/bucket
Package bucket implements a rolling bucket system.
 Package bucket implements a rolling bucket system.
tsdb/index
Package index implements transferring data to indices.
 Package index implements transferring data to indices.
bydbctl
cmd/bydbctl
Package main provides main entry for the command-line toolkit, i.e.
 Package main provides main entry for the command-line toolkit, i.e.
internal/cmd
Package cmd is an internal package defining cli commands for bydbctl.
 Package cmd is an internal package defining cli commands for bydbctl.
pkg/file
Package file provides utils to handle files.
 Package file provides utils to handle files.
pkg
accesslog
Package accesslog provides access log for banyandb.
 Package accesslog provides access log for banyandb.
bus
Package bus implements a message bus which is a common data model and a messaging infrastructure to allow different modules to communicate locally or remotely.
 Package bus implements a message bus which is a common data model and a messaging infrastructure to allow different modules to communicate locally or remotely.
cmdsetup
Package cmdsetup implements a real env in which to run tests.
 Package cmdsetup implements a real env in which to run tests.
config
Package config implements a configuration system which could load configuration from flags and env vars.
 Package config implements a configuration system which could load configuration from flags and env vars.
convert
Package convert implements conversions to and from data.
 Package convert implements conversions to and from data.
encoding
Package encoding implements encoding/decoding data points.
 Package encoding implements encoding/decoding data points.
flow
Package flow implements a streaming calculation framework.
 Package flow implements a streaming calculation framework.
flow/streaming
Package streaming implement the flow framework to provide the sliding window, top-n aggregation, and etc.
 Package streaming implement the flow framework to provide the sliding window, top-n aggregation, and etc.
flow/streaming/sources
Package sources implements data sources to sink data into the flow framework.
 Package sources implements data sources to sink data into the flow framework.
fs
Package fs (file system) is an independent component to operate file and directory.
 Package fs (file system) is an independent component to operate file and directory.
grpchelper
Package grpchelper implements helpers to access gRPC services.
 Package grpchelper implements helpers to access gRPC services.
host
Package host provides information about the host.
 Package host provides information about the host.
index
Package index implements the index system for searching data.
 Package index implements the index system for searching data.
index/inverted
Package inverted implements a inverted index repository.
 Package inverted implements a inverted index repository.
index/lsm
Package lsm implements a tree-based index repository.
 Package lsm implements a tree-based index repository.
index/posting
Package posting implements a posting list contains a list of document ids.
 Package posting implements a posting list contains a list of document ids.
index/posting/roaring
Package roaring implements the posting list by a roaring bitmap.
 Package roaring implements the posting list by a roaring bitmap.
index/testcases
Package testcases implements common helpers for testing inverted and lsm indices.
 Package testcases implements common helpers for testing inverted and lsm indices.
iter
Package iter implement a generic Iterator.
 Package iter implement a generic Iterator.
iter/sort
Package sort provides a generic iterator that merges multiple sorted iterators.
 Package sort provides a generic iterator that merges multiple sorted iterators.
logger
Package logger implements a logging system with a module tag.
 Package logger implements a logging system with a module tag.
meter
Package meter provides a simple meter system for metrics.
 Package meter provides a simple meter system for metrics.
meter/prom
Package prom provides a prometheus implementation for the meter system.
 Package prom provides a prometheus implementation for the meter system.
node
Package node provides node selector for liaison.
 Package node provides node selector for liaison.
partition
Package partition implements a location system to find a shard or index rule.
 Package partition implements a location system to find a shard or index rule.
pb/v1
Package v1 implements helpers to access data defined by API v1.
 Package v1 implements helpers to access data defined by API v1.
pb/v1/tsdb
Package tsdb implements helpers around tsdb.IntervalRule.
 Package tsdb implements helpers around tsdb.IntervalRule.
query/aggregation
Package aggregation implements aggregation functions to statistic a range of values.
 Package aggregation implements aggregation functions to statistic a range of values.
query/executor
Package executor defines the specifications accessing underlying data repositories.
 Package executor defines the specifications accessing underlying data repositories.
query/logical
Package logical implements the executable operations.
 Package logical implements the executable operations.
query/logical/measure
Package measure implements execution operations for querying measure data.
 Package measure implements execution operations for querying measure data.
query/logical/stream
Package stream implements execution operations for querying stream data.
 Package stream implements execution operations for querying stream data.
run
Package run implements a lifecycle framework to control modules.
 Package run implements a lifecycle framework to control modules.
schema
Package schema implements a framework to sync schema info from the metadata repository.
 Package schema implements a framework to sync schema info from the metadata repository.
signal
Package signal implements a handler to listen to system signals.
 Package signal implements a handler to listen to system signals.
test
Package test implements helpers for testing.
 Package test implements helpers for testing.
test/flags
Package flags implements injected flags to run tests.
 Package flags implements injected flags to run tests.
test/flow
Package flow implements mockers for testing flow.
 Package flow implements mockers for testing flow.
test/helpers
Package helpers implements several helpers for testing.
 Package helpers implements several helpers for testing.
test/measure
Package measure implements helpers to load schemas for testing.
 Package measure implements helpers to load schemas for testing.
test/setup
Package setup implements a real env in which to run tests.
 Package setup implements a real env in which to run tests.
test/stream
Package stream implements helpers to load schemas for testing.
 Package stream implements helpers to load schemas for testing.
timestamp
Package timestamp implements several component related to time.
 Package timestamp implements several component related to time.
version
Package version can be used to implement embedding versioning details from git branches and tags into the binary importing this package.
 Package version can be used to implement embedding versioning details from git branches and tags into the binary importing this package.
wal
Package wal (Write-ahead logging) is an independent component to ensure data reliability.
 Package wal (Write-ahead logging) is an independent component to ensure data reliability.
test
cases
Package cases provides some tools to access test data.
 Package cases provides some tools to access test data.
cases/measure
Package measure_test contains integration test cases of the measure.
 Package measure_test contains integration test cases of the measure.
cases/measure/data
Package data contains integration test cases of the measure.
 Package data contains integration test cases of the measure.
cases/stream
Package stream_test contains integration test cases of the stream.
 Package stream_test contains integration test cases of the stream.
cases/stream/data
Package data contains integration test cases of the stream.
 Package data contains integration test cases of the stream.
cases/topn
Package topn_test contains integration test cases of the TopN.
 Package topn_test contains integration test cases of the TopN.
cases/topn/data
Package data contains integration test cases of the topN.
 Package data contains integration test cases of the topN.
e2e-v2/cases/profiling/ebpf/oncpu
Package main is the entry point of the process for EBPF profiling.
 Package main is the entry point of the process for EBPF profiling.
integration/standalone
Package standalone is a standalone integration test suite.
 Package standalone is a standalone integration test suite.
stress/cases/istio

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.