ChronoBase

README

ChronoBase

ChronoBase is a high-performance, embedded time-series database written in Go, designed for efficient storage and retrieval of log and metric data. It features a modular architecture with configurable storage policies, compression, and automatic data management.

Features

• Time-Series Optimized: Purpose-built for time-series data with efficient timestamp-based indexing
• Configurable Storage: Flexible tablet/shard-based storage with automatic data management
• Data Compression: Built-in GZIP compression support for optimal storage efficiency
• Automatic Data Management:
  • Configurable TTL (Time-To-Live) for data retention
  • Background merging of tablets/shard for storage optimization
  • Periodic cleanup of expired data
• Concurrent Operations: Thread-safe design supporting parallel reads and writes
• Block-Based Storage: Efficient block-based storage format for better I/O performance
• JSON Configuration: Easy configuration via JSON files

Getting Started

Prerequisites

• Go 1.21 or higher
• Git

Installation

git clone https://github.com/naimulh247/ChronoBase.git
cd ChronoBase
go build

Basic Usage

// Create and configure database
cfg := config.DefaultConfig()
database, err := db.OpenDB(cfg)
if err != nil {
    log.Fatal(err)
}
defer database.Close()

// Create a table
err = database.CreateTable("metrics", db.TableOptions{
    TTL: 24 * time.Hour,
    PrimaryKeys: []string{"id"},
})

// Insert data
row := storage.Row{
    Key:       []byte("metric1"),
    Timestamp: time.Now(),
    Data:      []byte("example data"),
}
err = database.Insert("metrics", row)

// Query data
startTime := time.Now().Add(-time.Hour)
endTime := time.Now()
results, err := database.Query("metrics", startTime, endTime)

Configuration

ChronoBase can be configured via a JSON configuration file. Example configuration:

{
    "data_dir": "./data",
    "max_tablet_size": 268435456,
    "block_size": 65536,
    "flush_interval": "10m",
    "merge_interval": "1m",
    "compression_enabled": true,
    "compression_type": "gzip",
    "compression_level": 6,
    "max_memory_usage": 1073741824,
    "max_open_files": 1000,
    "buffer_pool_size": 134217728,
    "concurrent_merges": 2,
    "query_concurrency": 4,
    "maintenance_interval": "1h",
    "max_file_age": "720h",
    "backup_interval": "24h",
    "backup_retention": 7,
    "log_level": "info",
    "log_file": "db.log",
    "enable_metrics": true
}

Architecture

ChronoBase uses a tablet/shard-based storage architecture:

• Data is organized into tables
• Each table contains multiple tablets
• Tablets/shard store data in compressed blocks
• In-memory buffer for recent writes
• Background processes for merging and cleanup

Key Components:

• Table Manager: Handles table creation and management
• Tablet Storage: Manages data storage in tablets/shard
• Block Engine: Efficient block-based storage format
• Index Manager: Maintains time-based indices
• Compression Engine: Handles data compression/decompression

Work in Progress

This project is under active development. Upcoming features include:

• Advanced querying capabilities
• Additional compression algorithms
• Replication support
• Metrics and monitoring
• Enhanced performance optimizations
• HTTP API interface

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.