dasel

README

dasel

Dasel (short for data-selector) allows you to query and modify data structures using selector strings.

Comparable to jq / yq, but supports JSON, YAML, TOML, XML and CSV with zero runtime dependencies.

One tool to rule them all

Say good bye to learning new tools just to work with a different data format.

Dasel uses a standard selector syntax no matter the data format. This means that once you learn how to use dasel you immediately have the ability to query/modify any of the supported data types without any additional tools or effort.

Table of contents

• Dasel
• One tool to rule them all
• Quickstart
• Completion
• Issue vs discussion
• Features
• Table of contents
• Documentation
• Playground
• Benchmarks
• Pre-Commit

Quickstart

Dasel is available on homebrew, ASDF, scoop, docker, Nix or as compiled binaries from the latest release.

brew install dasel

You can also install a development version with:

go install github.com/tomwright/dasel/cmd/dasel@master

For more information see the installation documentation.

Select

echo '{"name": "Tom"}' | dasel -r json '.name'
"Tom"

See select documentation.

Convert json to yaml

echo '{"name": "Tom"}' | dasel -r json -w yaml
name: Tom

See select documentation.

Put

echo '{"name": "Tom"}' | dasel put string -r json '.email' 'contact@tomwright.me'
{
  "email": "contact@tomwright.me",
  "name": "Tom"
}

See put documentation.

Delete

echo '{
  "email": "contact@tomwright.me",
  "name": "Tom"
}' | dasel delete -p json '.email'
{
  "name": "Tom"
}

See delete documentation.

Completion

If you want to use completion from the terminal you can do the following (using zsh in this example):

Add the following to ~/.zshrc and reload your terminal.

export fpath=(~/zsh/site-functions $fpath)
mkdir -p ~/zsh/site-functions
dasel completion zsh > ~/zsh/site-functions/_dasel
compinit

Pre-Commit

Add dasel hooks to .pre-commit-config.yaml file

- repo: https://github.com/TomWright/dasel
  rev: v1.25.1
  hooks:
    - id: dasel-validate

for a native execution of dasel, or use:

• dasel-validate-docker pre-commit hook for executing dasel using the official Docker images
• dasel-validate-bin pre-commit hook for executing dasel using the official binary

Issue vs Discussion

I have enabled discussions on this repository.

I am aware there may be some confusion when deciding where you should communicate when reporting issues, asking questions or raising feature requests so this section aims to help us align on that.

Please raise an issue if:

• You find a bug.
• You have a feature request and can clearly describe your request.

Please open a discussion if:

• You have a question.
• You're not sure how to achieve something with dasel.
• You have an idea but don't quite know how you would like it to work.
• You have achieved something cool with dasel and want to show it off.
• Anything else!

Features

• Query/select data from structured data files.
• Update data in structured data files.
• Create data files.
• Supports multiple data formats/types.
• Convert between data formats/types.
• Uses a standard query/selector syntax across all data formats.
• Zero runtime dependencies.
• Available on Linux, Mac and Windows.
• Available to import and use in your own projects.
• Run via Docker.
• Faster than jq/yq.
• Self update.
• Pre-commit hooks.

Documentation

The official dasel docs can be found at daseldocs.tomwright.me.

Playground

You can test out dasel commands using the playground.

Source code for the playground can be found at github.com/TomWright/daselplayground.

Benchmarks

In my tests dasel has been up to 3x faster than jq and 15x faster than yq.

See the benchmark directory.

Stargazers over time

Documentation

Index

• Variables
• func DynamicSelectorToGroups(selector string) ([]string, error)
• func ExtractNextSelector(input string) (string, int)
• func FormatNode(node *Node, format string) (*bytes.Buffer, error)
• func FormatNodes(nodes []*Node, format string) (*bytes.Buffer, error)
• type Condition
• type DynamicSelectorParts
  • func FindDynamicSelectorParts(selector string) DynamicSelectorParts
• type EqualCondition
  • func (c EqualCondition) Check(other reflect.Value) (bool, error)
• type InvalidIndexErr
  • func (e InvalidIndexErr) Error() string
  • func (e InvalidIndexErr) Is(err error) bool
• type KeyEqualCondition
  • func (c KeyEqualCondition) Check(other reflect.Value) (bool, error)
• type Node
  • func New(value interface{}) *Node
  • func NewFromFile(filename, parser string) (*Node, error)
  • func NewFromReader(reader io.Reader, parser string) (*Node, error)
  • func (n *Node) Delete(selector string) error
  • func (n *Node) DeleteMultiple(selector string) error
  • func (n *Node) InterfaceValue() interface{}
  • func (n *Node) Put(selector string, newValue interface{}) error
  • func (n *Node) PutMultiple(selector string, newValue interface{}) error
  • func (n *Node) Query(selector string) (*Node, error)
  • func (n *Node) QueryMultiple(selector string) ([]*Node, error)
  • func (n *Node) String() string
  • func (n *Node) Write(writer io.Writer, parser string, writeOptions []storage.ReadWriteOption) error
  • func (n *Node) WriteToFile(filename, parser string, writeOptions []storage.ReadWriteOption) error
• type Selector
  • func ParseSelector(selector string) (Selector, error)
  • func (s Selector) Copy() Selector
• type SortedComparisonCondition
  • func (c SortedComparisonCondition) Check(other reflect.Value) (bool, error)
• type UnexpectedPreviousNilValue
  • func (e UnexpectedPreviousNilValue) Error() string
  • func (e UnexpectedPreviousNilValue) Is(err error) bool
• type UnhandledCheckType
  • func (e UnhandledCheckType) Error() string
  • func (e UnhandledCheckType) Is(err error) bool
• type UnknownComparisonOperatorErr
  • func (e UnknownComparisonOperatorErr) Error() string
  • func (e UnknownComparisonOperatorErr) Is(err error) bool
• type UnsupportedSelector
  • func (e UnsupportedSelector) Error() string
  • func (e UnsupportedSelector) Is(err error) bool
• type UnsupportedTypeForSelector
  • func (e UnsupportedTypeForSelector) Error() string
  • func (e UnsupportedTypeForSelector) Is(err error) bool
• type ValueNotFound
  • func (e ValueNotFound) Error() string
  • func (e ValueNotFound) Is(err error) bool

Examples

• Node.Put
• Node.Query

Variables

var ErrDynamicSelectorBracketMismatch = errors.New("dynamic selector bracket mismatch")

ErrDynamicSelectorBracketMismatch is returned when the number of opening brackets doesn't equal that of the closing brackets.

var ErrMissingPreviousNode = errors.New("missing previous node")

ErrMissingPreviousNode is returned when findValue doesn't have access to the previous node.

Functions

func DynamicSelectorToGroups

func DynamicSelectorToGroups(selector string) ([]string, error)

DynamicSelectorToGroups takes a dynamic selector and splits it into groups.

func ExtractNextSelector

func ExtractNextSelector(input string) (string, int)

ExtractNextSelector returns the next selector from the given input.

func FormatNode

func FormatNode(node *Node, format string) (*bytes.Buffer, error)

FormatNode formats a node with the format template and returns the result.

func FormatNodes

func FormatNodes(nodes []*Node, format string) (*bytes.Buffer, error)

FormatNodes formats a slice of nodes with the format template and returns the result.

Types

type Condition

type Condition interface {
	Check(other reflect.Value) (bool, error)
}

Condition defines a Check we can use within dynamic selectors.

type DynamicSelectorParts

type DynamicSelectorParts struct {
	Key        string
	Comparison string
	Value      string
}

DynamicSelectorParts contains the parts for a dynamic selector.

func FindDynamicSelectorParts

func FindDynamicSelectorParts(selector string) DynamicSelectorParts

FindDynamicSelectorParts extracts the parts from the dynamic selector given.

type EqualCondition

type EqualCondition struct {
	// Key is the key of the value to check against.
	Key string
	// Value is the value we are looking for.
	Value string
	// Not is true if this is a not equal check.
	Not bool
}

EqualCondition lets you check for an exact match.

func (EqualCondition) Check

func (c EqualCondition) Check(other reflect.Value) (bool, error)

Check checks to see if other contains the required key value pair.

type InvalidIndexErr

type InvalidIndexErr struct {
	Index string
}

InvalidIndexErr is returned when a selector targets an index that does not exist.

func (InvalidIndexErr) Error

func (e InvalidIndexErr) Error() string

Error returns the error message.

func (InvalidIndexErr) Is

func (e InvalidIndexErr) Is(err error) bool

Is implements the errors interface, so the errors.Is() function can be used.

type KeyEqualCondition

type KeyEqualCondition struct {
	// Value is the value we are looking for.
	Value string
	// Not is true if this is a not equal check.
	Not bool
}

KeyEqualCondition lets you check for an exact match.

func (KeyEqualCondition) Check

func (c KeyEqualCondition) Check(other reflect.Value) (bool, error)

Check checks to see if other contains the required key value pair.

type Node

type Node struct {
	// Previous is the previous node in the chain.
	Previous *Node `json:"-"`
	// Next contains the next node in the chain.
	// This is used with Query and Put requests.
	Next *Node `json:"next,omitempty"`
	// NextMultiple contains the next nodes in the chain.
	// This is used with QueryMultiple and PutMultiple requests.
	// When a major version change occurs this will completely replace Next.
	NextMultiple []*Node `json:"nextMultiple,omitempty"`
	// OriginalValue is the value returned from the parser.
	// In most cases this is the same as Value, but is different for thr YAML parser
	// as it contains information on the original document.
	OriginalValue interface{} `json:"-"`
	// Value is the value of the current node.
	Value reflect.Value `json:"value"`
	// Selector is the selector for the current node.
	Selector Selector `json:"selector"`
	// contains filtered or unexported fields
}

Node represents a single node in the chain of nodes for a selector.

func New

func New(value interface{}) *Node

New returns a new root node with the given value.

func NewFromFile

func NewFromFile(filename, parser string) (*Node, error)

NewFromFile returns a new root node by parsing file using specified read parser.

func NewFromReader

func NewFromReader(reader io.Reader, parser string) (*Node, error)

NewFromReader returns a new root node by parsing from Reader using specified read parser.

func (*Node) Delete

func (n *Node) Delete(selector string) error

Delete uses the given selector to find and delete the final node from the current node.

func (*Node) DeleteMultiple

func (n *Node) DeleteMultiple(selector string) error

DeleteMultiple uses the given selector to query the current node for every match possible and deletes them from the current node.

func (*Node) InterfaceValue

func (n *Node) InterfaceValue() interface{}

InterfaceValue returns the value stored within the node as an interface{}.

func (*Node) Put

func (n *Node) Put(selector string, newValue interface{}) error

Put finds the node using the given selector and updates it's value. It then attempts to propagate the value back up the chain to the root element.

Example

ExampleNode_Put shows how to update data from go code.

package main

import (
	"encoding/json"
	"fmt"

	"github.com/tomwright/dasel"
)

func main() {
	myData := []byte(`{"name": "Tom"}`)
	var data interface{}
	if err := json.Unmarshal(myData, &data); err != nil {
		panic(err)
	}
	rootNode := dasel.New(data)
	if err := rootNode.Put(".name", "Jim"); err != nil {
		panic(err)
	}
	fmt.Println(rootNode.InterfaceValue())

}

Output:

map[name:Jim]

func (*Node) PutMultiple

func (n *Node) PutMultiple(selector string, newValue interface{}) error

PutMultiple all applicable nodes for the given selector and updates all of their values to the given value. It then attempts to propagate the value back up the chain to the root element.

func (*Node) Query

func (n *Node) Query(selector string) (*Node, error)

Query uses the given selector to query the current node and return the result.

Example

ExampleNode_Query shows how to query data from go code.

package main

import (
	"encoding/json"
	"fmt"

	"github.com/tomwright/dasel"
)

func main() {
	myData := []byte(`{"name": "Tom"}`)
	var data interface{}
	if err := json.Unmarshal(myData, &data); err != nil {
		panic(err)
	}
	rootNode := dasel.New(data)
	result, err := rootNode.Query(".name")
	if err != nil {
		panic(err)
	}
	fmt.Println(result.InterfaceValue())

}

Output:

Tom

func (*Node) QueryMultiple

func (n *Node) QueryMultiple(selector string) ([]*Node, error)

QueryMultiple uses the given selector to query the current node for every match possible and returns all of the end nodes.

func (*Node) String

func (n *Node) String() string

String returns the value of the node as a string. No formatting is done here, you get the raw value.

func (*Node) Write

func (n *Node) Write(writer io.Writer, parser string, writeOptions []storage.ReadWriteOption) error

Write writes data to Writer using specified write parser and options.

func (*Node) WriteToFile

func (n *Node) WriteToFile(filename, parser string, writeOptions []storage.ReadWriteOption) error

WriteToFile writes data to the given file with the specified options.

type Selector

type Selector struct {
	// Raw is the full selector.
	Raw string `json:"raw"`
	// Current is the selector to be used with the current node.
	Current string `json:"current"`
	// Remaining is the remaining parts of the Raw selector.
	Remaining string `json:"remaining"`
	// Type is the type of the selector.
	Type string `json:"type"`
	// Property is the name of the property this selector targets, if applicable.
	Property string `json:"property,omitempty"`
	// Index is the index to use if applicable.
	Index int `json:"index,omitempty"`
	// Conditions contains a set of conditions to optionally match a target.
	Conditions []Condition `json:"conditions,omitempty"`
}

Selector represents the selector for a node.

func ParseSelector

func ParseSelector(selector string) (Selector, error)

ParseSelector parses the given selector string and returns a Selector.

func (Selector) Copy

func (s Selector) Copy() Selector

Copy returns a copy of the selector.

type SortedComparisonCondition

type SortedComparisonCondition struct {
	// Key is the key of the value to check against.
	Key string
	// Value is the value we are looking for.
	Value string
	// Equal is true if the values can match.
	Equal bool
	// After is true if the input value should be sorted after the Value.
	After bool
}

SortedComparisonCondition lets you check for an exact match.

func (SortedComparisonCondition) Check

func (c SortedComparisonCondition) Check(other reflect.Value) (bool, error)

Check checks to see if other contains the required key value pair.

type UnexpectedPreviousNilValue

type UnexpectedPreviousNilValue struct {
	Selector string
}

UnexpectedPreviousNilValue is returned when the previous node contains a nil value.

func (UnexpectedPreviousNilValue) Error

func (e UnexpectedPreviousNilValue) Error() string

Error returns the error message.

func (UnexpectedPreviousNilValue) Is

func (e UnexpectedPreviousNilValue) Is(err error) bool

Is implements the errors interface, so the errors.Is() function can be used.

type UnhandledCheckType

type UnhandledCheckType struct {
	Value interface{}
}

UnhandledCheckType is returned when the a check doesn't know how to deal with the given type

func (UnhandledCheckType) Error

func (e UnhandledCheckType) Error() string

Error returns the error message.

func (UnhandledCheckType) Is

func (e UnhandledCheckType) Is(err error) bool

Is implements the errors interface, so the errors.Is() function can be used.

type UnknownComparisonOperatorErr

type UnknownComparisonOperatorErr struct {
	Operator string
}

UnknownComparisonOperatorErr is returned when

func (UnknownComparisonOperatorErr) Error

func (e UnknownComparisonOperatorErr) Error() string

Error returns the error message.

func (UnknownComparisonOperatorErr) Is

func (e UnknownComparisonOperatorErr) Is(err error) bool

Is implements the errors interface, so the errors.Is() function can be used.

type UnsupportedSelector

type UnsupportedSelector struct {
	Selector string
}

UnsupportedSelector is returned when a specific selector type is used in the wrong context.

func (UnsupportedSelector) Error

func (e UnsupportedSelector) Error() string

Error returns the error message.

func (UnsupportedSelector) Is

func (e UnsupportedSelector) Is(err error) bool

Is implements the errors interface, so the errors.Is() function can be used.

type UnsupportedTypeForSelector

type UnsupportedTypeForSelector struct {
	Selector Selector
	Value    reflect.Value
}

UnsupportedTypeForSelector is returned when a selector attempts to handle a data type it can't handle.

func (UnsupportedTypeForSelector) Error

func (e UnsupportedTypeForSelector) Error() string

Error returns the error message.

func (UnsupportedTypeForSelector) Is

func (e UnsupportedTypeForSelector) Is(err error) bool

Is implements the errors interface, so the errors.Is() function can be used.

type ValueNotFound

type ValueNotFound struct {
	Selector      string
	PreviousValue reflect.Value
}

ValueNotFound is returned when a selector string cannot be fully resolved.

func (ValueNotFound) Error

func (e ValueNotFound) Error() string

Error returns the error message.

func (ValueNotFound) Is

func (e ValueNotFound) Is(err error) bool

Is implements the errors interface, so the errors.Is() function can be used.