create

Documentation

Index

• Variables
• func MibToBytes(mib float64) int
• type Command
  • func (c *Command) Run(args []string) error

Variables

var Create = &charm.Spec{
	Name:  "create",
	Usage: "create [-coltresh thresh] [-skewthresh thesh] -o file files...",
	Short: "create a zst columnar object from a zng file or stream",
	Long: `
The create command generates a columnar zst object from a zng input stream,
which may be stdin or one or more zng storage objects (local files or s3 objects).
The output can be a local file or an s3 URI.

The -colthresh flag specifies the byte threshold (in MiB) at which chunks
of column data are written to disk.

The -skewthresh flag specifies a rough byte threshold (in MiB) that controls
how much column data is collectively buffered in memory before being entirely
flushed to disk.  This parameter controls the amount of buffering "skew" required
keep rows in alignment so that a reader should not have to use more than
this (approximate) memory footprint.

Unlike parquet, zst column data may be laid out any way a client so chooses
and is not constrained to the "row group" concept.  Thus, care should be
taken here to control the amount of row skew that can arise.`,
	New: newCommand,
}

Functions

func MibToBytes

func MibToBytes(mib float64) int

Types

type Command

type Command struct {
	*devzst.Command
	// contains filtered or unexported fields
}

func (*Command) Run

func (c *Command) Run(args []string) error