badgerutils

README

Badger Utils

Go package with utilities for interacting with Badger.

Getting Started

Stream Stdin to Badger

To create a CLI to stream stdin into Badger, use badgerutils.WriteStdin. It takes a function lineToKeyed as a parameter, which converts a string into a struct that implements the Keyed interface.

Example

// examples/writer.go
package main

import (
	"fmt"
	"github.com/Surfline/badgerutils"
	"log"
	"strings"
)

type sampleRecord struct {
	Field1 string
	Field2 string
	Field3 string
}

func (r sampleRecord) Key() string {
	return fmt.Sprintf("%v,%v,%v", r.Field1, r.Field2, r.Field3)
}

func lineToKeyed(line string) (badgerutils.Keyed, error) {
	values := strings.Split(line, ",")
	return sampleRecord{values[0], values[1], values[2]}, nil
}

func main() {
	if err := badgerutils.WriteStdin(lineToKeyed); err != nil {
		log.Fatal(err)
	}
}

The code above can be called with the following flags:

• -dir - (required) The path to the directory to persist Badger files.
• -batch-size - (default: 1000) The size of each transaction (or batch of writes). This can be tuned for optimal performance depending on the machine.

For example:

$ for i in {1..10}; do echo "field${i}1,field${i}2,field${i}3"; done | go run main.go -dir=temp -batch-size=1
Directory: temp
Batch Size: 3
...
Records: 3
Records: 6
Records: 9
Records: 10
Inserted 10 records in 474.69µs

Development

Dependency Management

dep is required for dependency management.

$ make install

Format Code

Run this before opening pull requests to ensure code is properly formatted.

$ make fmt

Unit Tests

$ make test

Documentation

Overview

Package badgerutils provides functions for interacting with the underlying database.

Index

• func WriteStdin(lineToKeyed func(string) (Keyed, error)) error
• type Keyed

Functions

func WriteStdin

func WriteStdin(lineToKeyed func(string) (Keyed, error)) error

WriteStdin translates stdin into key/value pairs that are written into the Badger. lineToKeyed function parameter defines how stdin is translated to a value and how to define a key from that value.

Types

type Keyed

type Keyed interface {
	Key() string
}