ltsv

README

ltsv

High performance LTSV (Labeled Tab Separeted Value) parser for Go.

About LTSV: http://ltsv.org/

Labeled Tab-separated Values (LTSV) format is a variant of 
Tab-separated Values (TSV). Each record in a LTSV file is represented 
as a single line. Each field is separated by TAB and has a label and
 a value. The label and the value have been separated by ':'. With 
the LTSV format, you can parse each line by spliting with TAB (like 
original TSV format) easily, and extend any fields with unique labels 
in no particular order.

Installation

go get github.com/Wing924/ltsv

Examples

package main

import (
	"fmt"
	"github.com/Wing924/ltsv"
)

func main() {
	line := []byte("foo:123\tbar:456")
    record, err := ltsv.ParseLineAsMap(line, nil)
    if err != nil {
        panic(err)
    }
    fmt.Printf("%#v", record) // map[string]string{"foo":"123", "bar":"456"}
}

Benchmarks

Benchmark against

• Songmu/go-ltsv: 635% faster
• ymotongpoo/goltsv: 365% faster
• najeira/ltsv: 782% faster

Source code: bench/line_test.go.

Result

$ go test -bench . -benchmem
goos: darwin
goarch: amd64
pkg: github.com/Wing924/ltsv/bench
Benchmark_line_Wing924_ltsv-4          	 2000000	       626 ns/op	     224 B/op	      17 allocs/op
Benchmark_line_Wing924_ltsv_strict-4   	 2000000	       788 ns/op	     224 B/op	      17 allocs/op
Benchmark_line_Songmu_goltsv-4         	  300000	      3975 ns/op	    1841 B/op	      32 allocs/op
Benchmark_line_ymotongpoo_goltsv-4     	  500000	      2286 ns/op	    5793 B/op	      17 allocs/op
Benchmark_line_najeira_ltsv-4          	  300000	      4896 ns/op	    5529 B/op	      26 allocs/op
PASS
ok  	github.com/Wing924/ltsv/bench	8.245s

Documentation

Index

• Variables
• func ParseField(field []byte) (label []byte, value []byte, err error)
• func ParseLine(line []byte, callback func(label []byte, value []byte) error) error
• func ParseLineAsMap(line []byte, record map[string]string) (map[string]string, error)
• type Field
  • func ParseLineAsSlice(line []byte, record []Field) ([]Field, error)
• type Parser
  • func (p Parser) ParseField(field []byte) (label []byte, value []byte, err error)
  • func (p Parser) ParseLine(line []byte, callback func(label []byte, value []byte) error) error
  • func (p Parser) ParseLineAsMap(line []byte, record map[string]string) (map[string]string, error)
  • func (p Parser) ParseLineAsSlice(line []byte, record []Field) ([]Field, error)

Examples

• ParseLineAsMap
• ParseLineAsSlice

Variables

var (
	// ErrMissingLabel is an error to describe label is missing (ex. 'my_value')
	ErrMissingLabel = errors.New("missing label")
	// ErrEmptyLabel is an error to describe label is empty (ex. ':my_value')
	ErrEmptyLabel = errors.New("empty label")
	// ErrInvalidLabel is an error to describe label contains invalid char (ex. 'my\tlabel:my_value')
	ErrInvalidLabel = errors.New("invalid label")
	// ErrInvalidValue is an error to describe value contains invalid char (ex. 'my_label:my_value\n')
	ErrInvalidValue = errors.New("invalid value")
	// Break is an error for break loop
	Break = errors.New("break")
)

var DefaultParser = Parser{
	FieldDelimiter: '\t',
	ValueDelimiter: ':',
	StrictMode:     true,
}

DefaultParser is the default parser

Functions

func ParseField

func ParseField(field []byte) (label []byte, value []byte, err error)

ParseField parse LTSV-encoded field and return the label and value. The result share same memory with inputted field

func ParseLine

func ParseLine(line []byte, callback func(label []byte, value []byte) error) error

ParseLine parse one line of LTSV-encoded data and call callback. The callback function will be called for each field.

func ParseLineAsMap

func ParseLineAsMap(line []byte, record map[string]string) (map[string]string, error)

ParseLineAsMap parse one line of LTSV-encoded data and return the map[string]string. For reducing memory allocation, you can pass a map to record to reuse the given map.

Example

line := []byte("foo:123\tbar:456")
record, err := ParseLineAsMap(line, nil)
if err != nil {
	panic(err)
}
fmt.Printf("%#v", record) // map[string]string{"foo":"123", "bar":"456"}

Types

type Field

type Field struct {
	Label string
	Value string
}

Field is a struct to hold label-value pair.

func ParseLineAsSlice

func ParseLineAsSlice(line []byte, record []Field) ([]Field, error)

ParseLineAsSlice parse one line of LTSV-encoded data and return the []Field. For reducing memory allocation, you can pass a slice to record to reuse the given slice.

Example

line := []byte("foo:123\tbar:456")
record, err := ParseLineAsSlice(line, nil)
if err != nil {
	panic(err)
}
fmt.Printf("%+v", record) // [{Label:foo Value:123} {Label:bar Value:456}]

type Parser

type Parser struct {
	// FieldDelimiter is the delimiter of fields. It defaults to '\t'.
	FieldDelimiter byte
	// ValueDelimiter is the delimiter of label-value pairs. It defaults to ':'.
	ValueDelimiter byte
	// StrictMode is a flag to check if labels and values are valid.
	// If strictMode is false,
	// the parser just split fields with `FieldDelimiter`
	// and split label and value with `ValueDelimiter` without checking if they are valid.
	// The valid label is `/[0-9A-Za-z_.-]+/`.
	// The valid value is `/[^\b\t\r\n]*/`.
	StrictMode bool
}

Parser is for parsing LTSV-encoded format.

func (Parser) ParseField

func (p Parser) ParseField(field []byte) (label []byte, value []byte, err error)

ParseField parse LTSV-encoded field and return the label and value. The result share same memory with inputted field.

func (Parser) ParseLine

func (p Parser) ParseLine(line []byte, callback func(label []byte, value []byte) error) error

ParseLine parse one line of LTSV-encoded data and call callback. The callback function will be called for each field.

func (Parser) ParseLineAsMap

func (p Parser) ParseLineAsMap(line []byte, record map[string]string) (map[string]string, error)

ParseLineAsMap parse one line of LTSV-encoded data and return the map[string]string. For reducing memory allocation, you can pass a map to record to reuse the given map.

func (Parser) ParseLineAsSlice

func (p Parser) ParseLineAsSlice(line []byte, record []Field) ([]Field, error)

ParseLineAsSlice parse one line of LTSV-encoded data and return the []Field. For reducing memory allocation, you can pass a slice to record to reuse the given slice.