ar

README

ar

The Unix ar archive library for Go

This library provides a reader and a writer for Unix ar archives. The API heavily inspired by the tar module from Go's standard library. The following features set the library apart from other Go ar libraries such as github.com/blakesmith/ar:

• Automatic file size determination: Add files without knowing their size beforehand. This is useful when compressing the files on the fly while writing them to ar archive by stacking multiple io.Writer.
• Support for long file names: The traditional ar format has a file name size limit of 16 characters. Multiple extensions have been created to work around this. io.Writer writes long archive names in BSD style and io.Reader supports BSD and System V/Gnu style.
• Robust io.Reader/io.Writer APIs: Files can be writting in multiple Write calls and they can be read in multiple Read calls.

Reader

The ar Reader can read traditional, BSD-style and System V/Gnu-style archives. Symbol lookup tables are currently not supported.

arReader, err := ar.NewReader(file)
if err != nil {
  return err
}

// read file header and setup reader to provide the file content
header, err := arReader.Next()
if err != nil {
  return err
}
// read the file content that belongs to the header that was returned by the
// previous Next() call (io.ReadAll calls arReader.Read)
content, err := io.ReadAll(arReader)
if err != nil {
  return err
}

Writer

The ar writer supports automatic file size determination. Consider the following example, where a file is gzip compressed on the fly. Since the final size of the compressed file is not known until after it is written, the Size field of the ar header is set to ar.UnknownSize to enable automatic file size determination.

arWriter := NewWriter(file)

// write a file header for the compressed data while signaling that the file
// size needs to be automatically determined using the UnknownSize constant
err := arWriter.WriteHeader(&Header{
  Name:    "data.gz",
  ModTime: time.Now(),
  Mode:    0o644,
  Size:    UnknownSize,
})
if err != nil {
  panic(err)
}

// write compressed data
gzipWriter := gzip.NewWriter(arWriter)

_, err = io.Copy(gzipWriter, largeFile)
if err != nil {
  return err
}

err = gzipWriter.Close()
if err != nil {
  return err
}

err = arWriter.Close()
if err != nil {
  return err
}

Tips

When adding actual files to the ar archive, the header can be easily populated using the helper functions NewHeaderFromFile and NewHeaderFromFileInfo.

Caveats for automatic file size determination:

• If the underlying io.Writer is an *os.File or if it at least implements io.WriterAt and io.Seek, automatic file size determination has a negligible performance impact.
• If the underlying io.Writer is not an *os.File and if it does not implement WriteAt and io.Seek, the header and file content will be buffered in memory until the next ar entry is started or Close is called if automatic file size determination is enabled.

Long file names

If a file name has more than 16 characters, it will be written in BSD-style which can also be read by the Gnu/binutils ar tool.

Documentation

Overview

Package ar provides both a reader and a writer for Unix ar archives. The writer allows for optional automatic file size determination. This way, entries can be written without knowing the final file size beforehand, for example when adding files that are being compressed simultaneously.

Index

• Constants
• Variables
• type Header
  • func NewHeaderFromFile(fileName string) (*Header, error)
  • func NewHeaderFromFileInfo(stat fs.FileInfo) *Header
• type Reader
  • func NewReader(r io.Reader) (*Reader, error)
  • func (r *Reader) Next() (*Header, error)
  • func (r *Reader) Read(buffer []byte) (n int, err error)
• type Type
• type Writer
  • func NewWriter(w io.Writer) Writer

Examples

• NewReader
• NewWriter

Constants

const (
	// GlobalHeader contains the magic bytes that are written at the beginning
	// of an ar file (also known as armag).
	GlobalHeader = "!<arch>\n"
	// ThinArchiveGlobalHeader contains the magic bytes that are written at the
	// beginning of a thin ar archive, which is currently not supported.
	ThinArchiveGlobalHeader = "!<thin>\n"
	// HeaderTerminator contains the byte sequence with which file headers are
	// terminated (also known as ar_fmag).
	HeaderTerminator = "`\n"
	// HeaderSize holds the size of an ar header in bytes.
	HeaderSize = 60
	// UnknownSize signals that an ar entry's size is not known before writing
	// it. If Header.Size holds UnknownSize, Writer will automatically determine
	// the size and correct the header's size value when finalizing the entry.
	// If the underlying writer is not an *os.File (or does not implement Write,
	// WriteAt and Seek), this requires the Writer to buffer the file content in
	// memory.
	UnknownSize = math.MaxInt64
)

Variables

var (
	// ErrWriteTooLong is returned when more bytes are written than advertized
	// in the file header. ErrWriteTooLong is not returned in auto-correcting
	// mode (when Header.Size == UnknownSize) where the header size is
	// retroactively corrected instead.
	ErrWriteTooLong = fmt.Errorf("write too long")

	// ErrWriteTooShort is returned when a new header is appended or if the file
	// is closed before writing as many bytes for the previous file entry as
	// advertized in the corresponding header. ErrWriteTooShort is not returned
	// in auto-correcting mode (when Header.Size == UnknownSize) where the
	// header size is retroactively corrected instead.
	ErrWriteTooShort = fmt.Errorf("write too short")

	// ErrInvalidGlobalHeader is returned by NewReader if the provided data does
	// not start with the correct ar global header.
	ErrInvalidGlobalHeader = fmt.Errorf("invalid global header")
)

Types

type Header

type Header struct {
	// Name holds the file name. If the name has more than 16 characters, it is
	// considered an extended name which will be written in BSD style.
	Name string
	// ModTime holds the time stamp of the last file modification.
	ModTime time.Time
	// UID holds the file owner's UID.
	UID int64
	// GID holds the fole owner's GID.
	GID int64
	// Mode holds the file's mode.
	Mode uint32
	// Size holds the file size of up to 9999999999 bytes. If Size is
	// UnknownSize, the file size will be automatically determined by the actual
	// bytes written for the entry.
	Size int64
}

Header holds an ar entry's metadata.

func NewHeaderFromFile

func NewHeaderFromFile(fileName string) (*Header, error)

NewHeaderFromFile creates an ar header based on the stat information of a file.

func NewHeaderFromFileInfo

func NewHeaderFromFileInfo(stat fs.FileInfo) *Header

NewHeaderFromFileInfo creates an ar header based on the provided fs.FileInfo as returned for examply by os.Stat.

type Reader

type Reader struct {
	// DisableBSDExtensions disables parsing the BSD file format extensions. If
	// this is disabled, the file names and data of ar files written with BSD's
	// or macOS's ar program will be corrupt if a file name is longer than 16
	// characters.
	DisableBSDExtensions bool

	// DisableGnuExtensions disables parsing the System V/Gnu file format. If
	// this is disabled, the file names and data of ar files written with the
	// Gnu/binutils ar program will be corrupt if a file name is longer than 16
	// characters.
	DisableGnuExtensions bool
	// contains filtered or unexported fields
}

Reader allows to sequentially read ar files by first reading an entry's header using Next and then reading the full file content using Read.

func NewReader

func NewReader(r io.Reader) (*Reader, error)

NewReader creates a new ar file Reader.

Example

f, _ := os.Open("example.a")
defer f.Close()

arReader, err := NewReader(f)
if err != nil {
	panic(err)
}

// read file header and setup reader to provide the file content
header, err := arReader.Next()
if err != nil {
	panic(err)
}

fmt.Println(header.Name + ":")

// read the file content that belongs to the header that was returned by the
// previous Next() call (io.ReadAll calls arReader.Read)
content, err := io.ReadAll(arReader)
if err != nil {
	panic(err)
}

fmt.Println(string(content))

func (*Reader) Next

func (r *Reader) Next() (*Header, error)

Next returns the header of the next file entry in the ar file and enables reading the corresponding body in subsequent Read calls. If Next is called before the previous file is completely read, the remaining bytes of the previous file will be skipped automatically.

func (*Reader) Read

func (r *Reader) Read(buffer []byte) (n int, err error)

Read reads the file content of the entry whose header was previously read using Next.

type Type

type Type int

Type specifies the ar archive variant with the corresponding extensions.

var (
	// TypeBasic is a basic ar archive without any extensions that supports 16
	// character file names.
	TypeBasic Type = 0 //nolint:revive
	// TypeBSD is the ar archive type written by BSD's ar tool with support
	// for large file names.
	TypeBSD Type = 1
	// TypeGNU is the ar archive type written by GNU's or SYSTEM V's ar tool
	// with support for large file names.
	TypeGNU Type = 2
)

type Writer

type Writer interface {
	// WriteHeader writes the file entry header and if necessary the global ar
	// header. By setting Header.Size == UnknownSize, the file size will be
	// auto-corrected. File names longer than 16 characters will be written
	// using BSD file name extension.
	WriteHeader(*Header) error

	// Write writes the actual file content corresponding the file header that
	// was previously written using WriteHeader. An ar file can be written in
	// multiple consecutive Write calls.
	Write([]byte) (int, error)

	// Close ensures that the complete ar content is flushed.
	Close() error
}

Writer allows for sequential writing of an ar file by consecutively writing each file's header using WriteHeader and body using one or more calls to Write. Remember to call Close after finishing the last entry.

func NewWriter

func NewWriter(w io.Writer) Writer

NewWriter creates a new ar Writer that supports optional automatic file size correction (see Header.Size). If the provided writer is not an *os.File (or does not implement Seek, Write and WriteAt), auto-correcting the file size requires Writer to buffer the file content in memory.

Example

// setup file to be copied into the ar archive
f, _ := os.CreateTemp("", "ar_example")
defer f.Close()
defer os.RemoveAll(f.Name())

var buffer bytes.Buffer

arWriter := NewWriter(&buffer)

stat, _ := os.Stat(f.Name())

// write file header based on the file's stat
err := arWriter.WriteHeader(NewHeaderFromFileInfo(stat))
if err != nil {
	panic(err)
}

// write the file body (io.Copy uses arWriter.Write)
_, err = io.Copy(arWriter, f)
if err != nil {
	panic(err)
}

// make sure everything is written to the archive
err = arWriter.Close()
if err != nil {
	panic(err)
}

// print ar archive to stdout
fmt.Println(buffer.String())