zipgo

README

zipgo

Command line tool to zip-encode a file or directory and write a Go file that can unpack the embedded zip file.

Usage: zipgo [options] <path>

Options:
  -d, --data            Write only the zip data constant to the source file.
  -m, --digest <file>   Use MD5 digest in named file to determine if source has changed.
  -h, --help            Print this help text and exit
  -l, --log             Log the files as they are added to the zip archive.
  -x, --omit <files>    Comma-separated list of files names to omit from the zip archive.
  -o, --output <file>   Write output to <file> (default: unzip.go)
  -p, --package <name>  Specify Go package name (default: main)
  -v, --version         Print version and exit

The zipgo command will read a single file or a directory tree (identified as the path in the command line invocation). The file is compressed using the standard zip algorithm, and then expressed as a base64 character string constant. This constant is written to a Go source file, and is suitable to be included in a Go workspace that wishes to unzip the static archive of the file or directory at runtime.

Options

The command line options may be expressed in any order. Any command line item that is not recognized as a command line option is assumed to be the path to the file or directory to be archived.

--data

When --data is added to the command line, the output source file generated by zipgo contains only the Go const string that holds the archive. It does not contain the function that will extract the archive back to disk. This is useful if you separate the unzip function into its own source file. Then, when you need to regenerate the archive data, you need only generate the source file with the archive constant.

If --data is not on the command line, then the resulting source file contains a function named Unzip in addition to the archive data. This function can be called by the Go program and is passed a location path in which to extract the archive. See the section later on using the resulting source program.

--digest

When --digest followed by a filename is specified, zipgo will calculate an MD5 checksum of the input file or directory tree, and compare it to the checksum stored in the named file (if it exists).

If the file exists and the checkums match, the program does not generate a new source file. This is used when zipgo is part of an automated build to prevent it from regenerating each time the build is done when the source file or tree has not changed.

If the file does not exist, or the checksums do not match, the output file is generated and the revised checksum is stored into the named file.

--help

Using the --help option displays a text message similar to the one at the top of this document on the user's console, describing the command line usage and the command line options.

The zipgo command then exits without doing any further work.

--log

The --log command line option causes the command to print a message to the stdout listing each directory and file that is recursively processed to create the archive. By default, no log messages are generated.

--omit

The --omit option specifies one or more files that are to be omitted from the archive as it is created. You can specify this option as many times as you wish on the command line, or you can express the list as a comma-separated string enclosed in quotes. The list should not include directory path information, only the filename. For example,

zipgo lib --omit "server.key,server.crt"

This generates an archive for the directory ./lib but omits any files named "server.key" or "server.crt" found anywhere in the tree. All other files not specified in the omit list will be included in the archive.

--output

The --output option specifies the name of the output file. If the option is not given, the output is assumed to be "unzip.go" in the current directory. This file contains the Go source for the string constant containing the entire generated archive, as well as the Unzip function that can extract the archive back to disk. See the --data option above for information on how to omit the Unzip function from the output source if desired.

--package

The --package option specifies the name of the Go package the output source file is a member of. If not specified, the default is "main". If you wish the resulting Go source file to be in a subdirectory of your workspace with a different package name, use this option to specify that name.

--version

This causes the zipgo command to display it's version number, and then exit.

Using zipgo

This tool is used by the Ego package to generate a skeleton ./lib directory when the Ego program determines that no library exists already. In this case, the Ego program extracts the stored archive and writes it to the library location.

To generate the archive, the zipgo command is invoked, which looks something like this:

zipgo  -l -p app -o app-cli/app/unzip.go ./lib -x "https-server.key,https-server.crt"

This generates a file called "unzip.go" in the app package within the Ego source tree. This archive will contain all the files in the ./lib directory tree, except the files "https-server.key" and "https-server.crt" if those files are found. It is not an error to omit files that do not exist.

The purpose of this is to allow the development build to generate the archive file but omit server-specific secrets that are stored in the lib directory.

Here is an example of the generated Go file:

package main

import (
	"archive/zip"
	"bytes"
	"encoding/base64"
	"io"
	"os"
	"path/filepath"
)

const zipdata = `UEsDBBQACAAIAAAAAAAAAAAAAAAAAAAAAAAXAAAAdGVzdC5kYXRhL3N1YjE
vc3ViMS50eHQKSa0oUUgsUSjJSFVIyywqLlEoLk1SyEktS83hMuQCBAAA//
9QSwcIrKwkqCQAAAAeAAAAUEsDBBQACAAIAAAAAAAAAAAAAAAAAAAAAAAcA
AAAdGVzdC5kYXRhL3N1YjEvc3ViMi9zdWIyLnR4dApJrShRSCxRKMlIVShO
Tc7PS1EoLk1SyEktS83hMuICBAAA//9QSwcIbb466CUAAAAfAAAAUEsDBBQ
ACAAIAAAAAAAAAAAAAAAAAAAAAAATAAAAdGVzdC5kYXRhL3Jvb3QxLnR4dA
pJrShRSCxRKMlIVSjKzy9RyEktS83hMuQCBAAA//9QSwcI7yHnNh8AAAAZA
AAAUEsDBBQACAAIAAAAAAAAAAAAAAAAAAAAAAATAAAAdGVzdC5kYXRhL3Jv
b3QyLnR4dApJrShRSCxRKMlIVSjKzy9RyEktS83hMuICBAAA//9QSwcILHL
KHR8AAAAZAAAAUEsBAhQAFAAIAAgAAAAAAKysJKgkAAAAHgAAABcAAAAAAA
AAAAAAAAAAAAAAAHRlc3QuZGF0YS9zdWIxL3N1YjEudHh0UEsBAhQAFAAIA
AgAAAAAAG2+OuglAAAAHwAAABwAAAAAAAAAAAAAAAAAaQAAAHRlc3QuZGF0
YS9zdWIxL3N1YjIvc3ViMi50eHRQSwECFAAUAAgACAAAAAAA7yHnNh8AAAA
ZAAAAEwAAAAAAAAAAAAAAAADYAAAAdGVzdC5kYXRhL3Jvb3QxLnR4dFBLAQ
IUABQACAAIAAAAAAAscsodHwAAABkAAAATAAAAAAAAAAAAAAAAADgBAAB0Z
XN0LmRhdGEvcm9vdDIudHh0UEsFBgAAAAAEAAQAEQEAAJgBAAAAAA==`



// Unzip extracts the zip data to the file system.
func Unzip(path string) error {
	// Decode the zip data.
	data, err := base64.StdEncoding.DecodeString(zipdata)
	if err != nil {
		return err
	}

	// Open the zip archive.
	r, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
	if err != nil {
		return err
	}

	// Extract the files in the archive.
	for _, f := range r.File {
		if err := extractFile(f, path); err != nil {
			return err
		}
	}

	return nil
}

// extractFile extracts a single file from the zip archive.
func extractFile(f *zip.File, path string) error {
	// Open the file in the archive.
	rc, err := f.Open()
	if err != nil {
		return err
	}

	defer rc.Close()

	// Create the file in the file system.
	path = filepath.Join(path, f.Name)
	if f.FileInfo().IsDir() {
		if err := os.MkdirAll(path, 0755); err != nil {
			return err
		}
	} else {
		if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
			return err
		}

		f, err := os.Create(path)
		if err != nil {
			return err
		}
		defer f.Close()

		// Copy the file contents.
		if _, err := io.Copy(f, rc); err != nil {
			return err
		}
	}

	return nil
}