phew

README

phew

A hex-encoded, gzipped encoder/decoder for spew.

When debugging go programs spew offers a wealth of information, but it can also be useful in production for capturing critical information about the execution of programs under real-world conditions.

The latter becomes somewhat challenging, however, since the spew pretty-printer deviates from the standard log format where everything fits on a single line. When printing large structs, the output can span several hundred or thousands of lines, obliterating the readability of the production logs.

Consider the following struct:

type VeryLargeThing struct {
    Number      uint32
    LotsOfBytes []byte
}

Calling spew.Sdump on an instance of VeryLargeThing might resemble:

(main.VeryLargeThing) {
 Number: (uint32) 42,
 LotsOfBytes: ([]uint8) (len=8 cap=8) {
  00000000  62 69 67 20 64 61 74 61                           |big data|
 }
}

Often times we don't necessarily need to observe the spewed output, but rather we would like the information to be availabe if need to investigate an issue. However, embedding the above Sdump can cause production logs to become bloated, even if there is no active investigation.

Enter phew: instead of logging the pretty-printed struct directly, we log hex-encoded, gzipped output of spew so that the output can be unpacked later for evaluation.

As a drop in replacement, encoding the same struct with phew.Sdump produces the following output, which can be easily embedded in a single log line:

1f8b08000000000000ffd2c84dccccd30b4b2daaf4492c4a4f0dc9c8cc4bd754a8e652f02bcd4d4a2db252d028cdcc2b3136d2543031d2e152f0c92f29f64f73aa2c492db652d0888e05495a682a68e4a4e6d95a28242716d85a80752b184081828299918299a58299b98291818299898299a1823998c40d6a9232d31552124b126bb8146ab96a01010000ffff2ee62bb7a3000000

To inspect a phewed object at a later point, one can use the phew decode CLI command, which yields the original, spewed output:

$ phew decode 1f8b08000000000000ffd2c84dccccd30b4b2daaf4492c4a4f0dc9c8cc4bd754a8e652f02bcd4d4a2db252d028cdcc2b3136d2543031d2e152f0c92f29f64f73aa2c492db652d0888e05495a682a68e4a4e6d95a28242716d85a80752b184081828299918299a58299b98291818299898299a1823998c40d6a9232d31552124b126bb8146ab96a01010000ffff2ee62bb7a3000000
(main.VeryLargeThing) {
 Number: (uint32) 42,
 LotsOfBytes: ([]uint8) (len=8 cap=8) {
  00000000  62 69 67 20 64 61 74 61                           |big data|
 }
}

CLI Installation

Using go get:

go get github.com/cfromknecht/phew

From source:

git clone git@github.com:cfromknecht/phew.git
cd phew
go install ./cmd/phew

Documentation

Index

• func Sdump(v ...interface{}) string
• type Reader
  • func NewReader(r io.Reader) (*Reader, error)
  • func (z *Reader) Read(p []byte) (int, error)
  • func (z *Reader) Reset(r io.Reader) error
• type Writer
  • func NewWriter(w io.Writer) *Writer
  • func (z *Writer) Close() error
  • func (z *Writer) Flush() error
  • func (z *Writer) Reset(w io.Writer)
  • func (w *Writer) Write(p []byte) (int, error)

Functions

func Sdump

func Sdump(v ...interface{}) string

Sdump spews a varidic number of objects into a phew stream.

Types

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Hzip files store a length and checksum of the uncompressed data. The Reader will return an ErrChecksum when Read reaches the end of the uncompressed data if it does not have the expected length or checksum. Clients should treat data returned by Read as tentative until they receive the io.EOF marking the end of the data.

func NewReader

func NewReader(r io.Reader) (*Reader, error)

NewReader creates a new Reader reading the given reader. If r does not also implement io.ByteReader, the decompressor may read more data than necessary from r.

It is the caller's responsibility to call Close on the Reader when done.

func (*Reader) Read

func (z *Reader) Read(p []byte) (int, error)

Read implements io.Reader, reading uncompressed bytes from its underlying Reader.

func (*Reader) Reset

func (z *Reader) Reset(r io.Reader) error

Reset discards the Reader z's state and makes it equivalent to the result of its original state from NewReader, but reading from r instead. This permits reusing a Reader rather than allocating a new one.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

A Writer is an io.WriteCloser used to encode binary data into a gzipped, hex-encoded binary stream.

func NewWriter

func NewWriter(w io.Writer) *Writer

It is the caller's responsibility to call Close on the Writer when done. Writes may be buffered and not flushed until Close.

func (*Writer) Close

func (z *Writer) Close() error

Close closes the Writer by flushing any unwritten data to the underlying io.Writer and writing the hex-encoded, GZIP footer. It does not close the underlying io.Writer.

func (*Writer) Flush

func (z *Writer) Flush() error

It is useful mainly in compressed network protocols, to ensure that a remote reader has enough data to reconstruct a packet. Flush does not return until the data has been written. If the underlying writer returns an error, Flush returns that error.

In the terminology of the zlib library, Flush is equivalent to Z_SYNC_FLUSH.

func (*Writer) Reset

func (z *Writer) Reset(w io.Writer)

Reset discards the Writer z's state and makes it equivalent to the result of its original state from NewWriter, but writing to w instead. This permits reusing a Writer rather than allocating a new one.

func (*Writer) Write

func (w *Writer) Write(p []byte) (int, error)

Write writes a compressed, hex-encoded form of p to the underlying io.Writer. The compressed bytes are not necessarily flushed until the Writer is closed.