bindata

README

go-bindata (forked by tmthrgd)

This is a rewrite of go-bindata that started life as fork of a fork. It was forked by lestrrat as jteeuwen seemed to have abandoned the project.

Since that fork, go-bindata has been largely rewritten and has become a standalone project. While the generated code has changed, the generated API remains backwards compatible. The package API is not backwards compatible. The CLI remains backwards compatible, but may not be as feature complete as the package API.

The suggested way of using go-bindata is from a single .go file with an ignore build tag (// +build ignore) run with //go:generate go run path/to/generate.go. (See issue #2 for reference).

Nota bene: Most of the README that follows has not been updated to match the changes made since rewriting go-bindata and likely does not accurately represent the state of go-bindata.

bindata

This package converts any file into manageable Go source code. Useful for embedding binary data into a go program. The file data is optionally gzip compressed before being converted to a raw byte slice.

Installation

To install the library, use the following:

go get -u github.com/tmthrgd/go-bindata/...

Accessing an asset

To access asset data, we use the Asset(string) ([]byte, error) function which is included in the generated output.

data, err := Asset("pub/style/foo.css")
if err != nil {
	// Asset was not found.
}

// use asset data

Debug vs Release builds

When invoking the program with the -debug flag, the generated code does not actually include the asset data. Instead, it generates function stubs which load the data from the original file on disk. The asset API remains identical between debug and release builds, so your code will not have to change.

This is useful during development when you expect the assets to change often. The host application using these assets uses the same API in both cases and will not have to care where the actual data comes from.

An example is a Go webserver with some embedded, static web content like HTML, JS and CSS files. While developing it, you do not want to rebuild the whole server and restart it every time you make a change to a bit of javascript. You just want to build and launch the server once. Then just press refresh in the browser to see those changes. Embedding the assets with the debug flag allows you to do just that. When you are finished developing and ready for deployment, just re-invoke go-bindata without the -debug flag. It will now embed the latest version of the assets.

Lower memory footprint

Using the -nomemcopy flag, will alter the way the output file is generated. It will employ a hack that allows us to read the file data directly from the compiled program's .rodata section. This ensures that when we call our generated function, we omit unnecessary memcopies.

The downside of this, is that it requires dependencies on the reflect and unsafe packages. These may be restricted on platforms like AppEngine and thus prevent you from using this mode.

Another disadvantage is that the byte slice we create, is strictly read-only. For most use-cases this is not a problem, but if you ever try to alter the returned byte slice, a runtime panic is thrown. Use this mode only on target platforms where memory constraints are an issue.

The default behaviour is to use the old code generation method. This prevents the two previously mentioned issues, but will employ at least one extra memcopy and thus increase memory requirements.

For instance, consider the following two examples:

This would be the default mode, using an extra memcopy but gives a safe implementation without dependencies on reflect and unsafe:

func myfile() []byte {
    return []byte{0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a}
}

Here is the same functionality, but uses the .rodata hack. The byte slice returned from this example can not be written to without generating a runtime error.

var _myfile = "\x89\x50\x4e\x47\x0d\x0a\x1a"

func myfile() []byte {
    var empty [0]byte
    sx := (*reflect.StringHeader)(unsafe.Pointer(&_myfile))
    b := empty[:]
    bx := (*reflect.SliceHeader)(unsafe.Pointer(&b))
    bx.Data = sx.Data
    bx.Len = len(_myfile)
    bx.Cap = bx.Len
    return b
}

Optional compression

When the -nocompress flag is given, the supplied resource is not GZIP compressed before being turned into Go code. The data should still be accessed through a function call, so nothing changes in the usage of the generated file.

This feature is useful if you do not care for compression, or the supplied resource is already compressed. Doing it again would not add any value and may even increase the size of the data.

The default behaviour of the program is to use compression.

Path prefix stripping

The keys used in the _bindata map, are the same as the input file name passed to go-bindata. This includes the path. In most cases, this is not desirable, as it puts potentially sensitive information in your code base. For this purpose, the tool supplies another command line flag -prefix. This accepts a portion of a path name, which should be stripped off from the map keys and function names.

For example, running without the -prefix flag, we get:

$ go-bindata /path/to/templates/

_bindata["/path/to/templates/foo.html"] = path_to_templates_foo_html

Running with the -prefix flag, we get:

$ go-bindata -prefix "/path/to/" /path/to/templates/

_bindata["templates/foo.html"] = templates_foo_html

Build tags

With the optional -tags flag, you can specify any go build tags that must be fulfilled for the output file to be included in a build. This is useful when including binary data in multiple formats, where the desired format is specified at build time with the appropriate tags.

The tags are appended to a // +build line in the beginning of the output file and must follow the build tags syntax specified by the go tool.

Testing

To execute the test case, run the following commands:

go get -t -u github.com/tmthrgd/go-bindata
go test github.com/tmthrgd/go-bindata

Test corpus

To generate the corpus-sha256sums needed for travis, run the following commands:

[ -d .testcorpus ] && rm -r .testcorpus
go test -run TestCorpus -randtests 50 -corpus .testcorpus -gencorpus .
cd .testcorpus && sha256sum * > ../corpus-sha256sums; cd ..

This must be done every time the generated code changes, but can be skipped while working on a pull request until it is ready to merge.

Documentation

Overview

Package bindata converts any file into manageable Go source code. Useful for embedding binary data into a go program. The file data is optionally gzip compressed before being converted to a raw byte slice.

The following paragraphs cover some of the customization options which can be specified in the Config struct, which must be passed into the Translate() call.

Debug vs Release builds

When used with the `Debug` option, the generated code does not actually include the asset data. Instead, it generates function stubs which load the data from the original file on disk. The asset API remains identical between debug and release builds, so your code will not have to change.

This is useful during development when you expect the assets to change often. The host application using these assets uses the same API in both cases and will not have to care where the actual data comes from.

An example is a Go webserver with some embedded, static web content like HTML, JS and CSS files. While developing it, you do not want to rebuild the whole server and restart it every time you make a change to a bit of javascript. You just want to build and launch the server once. Then just press refresh in the browser to see those changes. Embedding the assets with the `debug` flag allows you to do just that. When you are finished developing and ready for deployment, just re-invoke `go-bindata` without the `-debug` flag. It will now embed the latest version of the assets.

Lower memory footprint

The `MemCopy` option will alter the way the output file is generated. If false, it will employ a hack that allows us to read the file data directly from the compiled program's `.rodata` section. This ensures that when we call call our generated function, we omit unnecessary memcopies.

The downside of this, is that it requires dependencies on the `reflect` and `unsafe` packages. These may be restricted on platforms like AppEngine and thus prevent you from using this mode.

Another disadvantage is that the byte slice we create, is strictly read-only. For most use-cases this is not a problem, but if you ever try to alter the returned byte slice, a runtime panic is thrown. Use this mode only on target platforms where memory constraints are an issue.

The default behaviour is to use the old code generation method. This prevents the two previously mentioned issues, but will employ at least one extra memcopy and thus increase memory requirements.

For instance, consider the following two examples:

This would be the default mode, using an extra memcopy but gives a safe implementation without dependencies on `reflect` and `unsafe`:

func myfile() []byte {
	return []byte{0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a}
}

Here is the same functionality, but uses the `.rodata` hack. The byte slice returned from this example can not be written to without generating a runtime error.

var _myfile = "\x89\x50\x4e\x47\x0d\x0a\x1a"

func myfile() []byte {
	var empty [0]byte
	sx := (*reflect.StringHeader)(unsafe.Pointer(&_myfile))
	b := empty[:]
	bx := (*reflect.SliceHeader)(unsafe.Pointer(&b))
	bx.Data = sx.Data
	bx.Len = len(_myfile)
	bx.Cap = bx.Len
	return b
}

Optional compression

The Compress option indicates that the supplied assets are GZIP compressed before being turned into Go code. The data should still be accessed through a function call, so nothing changes in the API.

This feature is useful if you do not care for compression, or the supplied resource is already compressed. Doing it again would not add any value and may even increase the size of the data.

The default behaviour of the program is to use compression.

Path prefix stripping

The keys used in the `_bindata` map are the same as the input file name passed to `go-bindata`. This includes the path. In most cases, this is not desirable, as it puts potentially sensitive information in your code base. For this purpose, the tool supplies another command line flag `-prefix`. This accepts a portion of a path name, which should be stripped off from the map keys and function names.

For example, running without the `-prefix` flag, we get:

$ go-bindata /path/to/templates/

_bindata["/path/to/templates/foo.html"] = path_to_templates_foo_html

Running with the `-prefix` flag, we get:

$ go-bindata -prefix "/path/to/" /path/to/templates/

_bindata["templates/foo.html"] = templates_foo_html

Build tags

With the optional Tags field, you can specify any go build tags that must be fulfilled for the output file to be included in a build. This is useful when including binary data in multiple formats, where the desired format is specified at build time with the appropriate tags.

The tags are appended to a `// +build` line in the beginning of the output file and must follow the build tags syntax specified by the go tool.

Index

• type File
• type Files
  • func FindFiles(path string, opts *FindFilesOptions) (files Files, err error)
  • func (f Files) Generate(w io.Writer, opts *GenerateOptions) error
• type FindFilesOptions
• type GenerateOptions
• type HashEncoding
  • func (he HashEncoding) String() string
• type HashFormat
  • func (hf HashFormat) String() string

Types

type File

type File interface {
	// Name returns the name by which asset is referenced.
	Name() string
	// Path returns the relative path to the file.
	Path() string
	// AbsolutePath returns the absolute path to the file.
	AbsolutePath() string

	// Open returns an io.ReadCloser for reading the file.
	Open() (io.ReadCloser, error)
	// Stat returns an os.FileInfo interface representing the file.
	Stat() (os.FileInfo, error)
}

File represents a single asset file.

type Files

type Files []File

Files represents a collection of asset files.

func FindFiles

func FindFiles(path string, opts *FindFilesOptions) (files Files, err error)

FindFiles adds all files inside a directory to the generated output.

func (Files) Generate

func (f Files) Generate(w io.Writer, opts *GenerateOptions) error

Generate writes the generated Go code to w.

type FindFilesOptions

type FindFilesOptions struct {
	// Prefix defines a path prefix which should be stripped from all
	// file names when generating the keys in the table of contents.
	// For example, running without the `-prefix` flag, we get:
	//
	// 	$ go-bindata /path/to/templates
	// 	go_bindata["/path/to/templates/foo.html"] = _path_to_templates_foo_html
	//
	// Running with the `-prefix` flag, we get:
	//
	// 	$ go-bindata -prefix "/path/to/" /path/to/templates/foo.html
	// 	go_bindata["templates/foo.html"] = templates_foo_html
	Prefix string

	// Recursive defines whether subdirectories of Path
	// should be recursively included in the conversion.
	Recursive bool

	// Ignores any filenames matching the regex pattern specified, e.g.
	// path/to/file.ext will ignore only that file, or \\.gitignore
	// will match any .gitignore file.
	//
	// This parameter can be provided multiple times.
	Ignore []*regexp.Regexp
}

FindFilesOptions defines a set of options to use when searching for files.

type GenerateOptions

type GenerateOptions struct {
	// Name of the package to use.
	Package string

	// Tags specify a set of optional build tags, which should be
	// included in the generated output. The tags are appended to a
	// `// +build` line in the beginning of the output file
	// and must follow the build tags syntax specified by the go tool.
	Tags string

	// MemCopy will alter the way the output file is generated.
	//
	// If false, it will employ a hack that allows us to read the file data directly
	// from the compiled program's `.rodata` section. This ensures that when we call
	// call our generated function, we omit unnecessary mem copies.
	//
	// The downside of this, is that it requires dependencies on the `reflect` and
	// `unsafe` packages. These may be restricted on platforms like AppEngine and
	// thus prevent you from using this mode.
	//
	// Another disadvantage is that the byte slice we create, is strictly read-only.
	// For most use-cases this is not a problem, but if you ever try to alter the
	// returned byte slice, a runtime panic is thrown. Use this mode only on target
	// platforms where memory constraints are an issue.
	//
	// The default behaviour is to use the old code generation method. This
	// prevents the two previously mentioned issues, but will employ at least one
	// extra memcopy and thus increase memory requirements.
	//
	// For instance, consider the following two examples:
	//
	// This would be the default mode, using an extra memcopy but gives a safe
	// implementation without dependencies on `reflect` and `unsafe`:
	//
	// 	func myfile() []byte {
	// 		return []byte{0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a}
	// 	}
	//
	// Here is the same functionality, but uses the `.rodata` hack.
	// The byte slice returned from this example can not be written to without
	// generating a runtime error.
	//
	// 	var _myfile = "\x89\x50\x4e\x47\x0d\x0a\x1a"
	//
	// 	func myfile() []byte {
	// 		var empty [0]byte
	// 		sx := (*reflect.StringHeader)(unsafe.Pointer(&_myfile))
	// 		b := empty[:]
	// 		bx := (*reflect.SliceHeader)(unsafe.Pointer(&b))
	// 		bx.Data = sx.Data
	// 		bx.Len = len(_myfile)
	// 		bx.Cap = bx.Len
	// 		return b
	// 	}
	MemCopy bool

	// Compress means the assets are GZIP compressed before being turned into
	// Go code. The generated function will automatically unzip the file data
	// when called. Defaults to true.
	Compress bool

	// Perform a debug build. This generates an asset file, which
	// loads the asset contents directly from disk at their original
	// location, instead of embedding the contents in the code.
	//
	// This is mostly useful if you anticipate that the assets are
	// going to change during your development cycle. You will always
	// want your code to access the latest version of the asset.
	// Only in release mode, will the assets actually be embedded
	// in the code. The default behaviour is Release mode.
	Debug bool

	// Perform a dev build, which is nearly identical to the debug option. The
	// only difference is that instead of absolute file paths in generated code,
	// it expects a variable, `rootDir`, to be set in the generated code's
	// package (the author needs to do this manually), which it then prepends to
	// an asset's name to construct the file path on disk.
	//
	// This is mainly so you can push the generated code file to a shared
	// repository.
	Dev bool

	// When true, the AssetDir API will be provided.
	AssetDir bool

	// When true, only gzip decompress the data on first use.
	DecompressOnce bool

	// [Deprecated]: use github.com/tmthrgd/go-bindata/restore.
	Restore bool

	// When false, size, mode and modtime are not preserved from files
	Metadata bool
	// When nonzero, use this as mode for all files.
	Mode os.FileMode
	// When nonzero, use this as unix timestamp for all files.
	ModTime int64

	// Hash is used to produce a hash of the file.
	Hash hash.Hash
	// Which of the given name hashing formats to use.
	HashFormat HashFormat
	// The length of the hash to use, defaults to 16 characters.
	HashLength uint
	// The encoding to use to encode the name hash.
	HashEncoding HashEncoding
}

GenerateOptions defines a set of options to use when generating the Go code.

type HashEncoding

type HashEncoding int

HashEncoding specifies which encoding to use when hashing names.

const (
	// HexHash uses hexadecimal encoding.
	HexHash HashEncoding = iota
	// Base32Hash uses unpadded, lowercase standard base32
	// encoding (see RFC 4648).
	Base32Hash
	// Base64Hash uses an unpadded URL-safe base64 encoding
	// defined in RFC 4648.
	Base64Hash
)

func (HashEncoding) String

func (he HashEncoding) String() string

type HashFormat

type HashFormat int

HashFormat specifies which format to use when hashing names.

const (
	// NameUnchanged leaves the file name unchanged.
	NameUnchanged HashFormat = iota
	// DirHash formats names like path/to/hash/name.ext.
	DirHash
	// NameHashSuffix formats names like path/to/name-hash.ext.
	NameHashSuffix
	// HashWithExt formats names like path/to/hash.ext.
	HashWithExt
)

func (HashFormat) String

func (hf HashFormat) String() string