partialzip

README

partialzip

Partial Implementation of PartialZip in Go

---

Why?

If you need to download a VERY large zip, but you only need some small file from it, this is a nice optimization to allow you to only extract what you need and nothing else. Saving time, bandwidth and space.

Install

go get github.com/blacktop/partialzip

Example

import (
    "fmt"
    "log"

    "github.com/blacktop/partialzip"
)

func main() {
    pzip, err := partialzip.New("https://apple.com/ipsw/download/link")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(pzip.List())

    n, err := pzip.Download("kernelcache.release.iphone11")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("extracting kernelcache.release.iphone11, wrote %d bytes\n", n)
}

extracting "kernelcache.release.iphone11", wrote 17842148 bytes

CLI

Install

Download punzip from releases

Usage

List zipped files

$ punzip list http://updates-http.cdn-apple.com/download/ipsw

NAME                                                         |SIZE
Firmware/                                                    |0 B
...SNIP...
kernelcache.release.iphone11                                 |18 MB
Firmware/all_flash/LLB.d321.RELEASE.im4p.plist               |331 B
048-16246-211.dmg                                            |107 MB
048-15811-213.dmg                                            |106 MB
Firmware/ICE18-1.00.08.Release.bbfw                          |39 MB

Download a file from the remote zip

$ punzip get --path kernelcache.release.iphone11 http://updates-http.cdn-apple.com/download/ipsw

Successfully downloaded "kernelcache.release.iphone11"

Credits

• planetbeing/partial-zip (written in C)
• marcograss/partialzip (written in Rust)

License

MIT Copyright (c) 2018 blacktop

Documentation

Overview

Package partialzip is a Go package to pull single files our of large remote zip archives.

Index

• type File
• type FileHeader
• type PartialZip
  • func New(url string) (*PartialZip, error)
  • func (p *PartialZip) Download(path string) (int, error)
  • func (p *PartialZip) Get(path string) (io.ReadCloser, error)
  • func (p *PartialZip) List() []string

Types

type File

type File struct {
	FileHeader

	HeaderOffset int64
	DataOffset   int64 // Offsetr to data start
	// contains filtered or unexported fields
}

File is a file object embedded in a remote zip

type FileHeader

type FileHeader struct {
	// Name is the name of the file.
	//
	// It must be a relative path, not start with a drive letter (such as "C:"),
	// and must use forward slashes instead of back slashes. A trailing slash
	// indicates that this file is a directory and should have no data.
	//
	// When reading zip files, the Name field is populated from
	// the zip file directly and is not validated for correctness.
	// It is the caller's responsibility to sanitize it as
	// appropriate, including canonicalizing slash directions,
	// validating that paths are relative, and preventing path
	// traversal through filenames ("../../../").
	Name string

	// Comment is any arbitrary user-defined string shorter than 64KiB.
	Comment string

	// NonUTF8 indicates that Name and Comment are not encoded in UTF-8.
	//
	// By specification, the only other encoding permitted should be CP-437,
	// but historically many ZIP readers interpret Name and Comment as whatever
	// the system's local character encoding happens to be.
	//
	// This flag should only be set if the user intends to encode a non-portable
	// ZIP file for a specific localized region. Otherwise, the Writer
	// automatically sets the ZIP format's UTF-8 flag for valid UTF-8 strings.
	NonUTF8 bool

	CreatorVersion uint16
	ReaderVersion  uint16
	Flags          uint16

	// Method is the compression method. If zero, Store is used.
	Method uint16

	// Modified is the modified time of the file.
	//
	// When reading, an extended timestamp is preferred over the legacy MS-DOS
	// date field, and the offset between the times is used as the timezone.
	// If only the MS-DOS date is present, the timezone is assumed to be UTC.
	//
	// When writing, an extended timestamp (which is timezone-agnostic) is
	// always emitted. The legacy MS-DOS date field is encoded according to the
	// location of the Modified time.
	Modified     time.Time
	ModifiedTime uint16 // Deprecated: Legacy MS-DOS date; use Modified instead.
	ModifiedDate uint16 // Deprecated: Legacy MS-DOS time; use Modified instead.

	CRC32              uint32
	CompressedSize     uint32 // Deprecated: Use CompressedSize64 instead.
	UncompressedSize   uint32 // Deprecated: Use UncompressedSize64 instead.
	CompressedSize64   uint64
	UncompressedSize64 uint64
	Extra              []byte
	ExternalAttrs      uint32 // Meaning depends on CreatorVersion
}

FileHeader is the file header object for a file embedded in a remote zip

type PartialZip

type PartialZip struct {
	URL   string
	Size  int64
	Files []*File
}

PartialZip defines a custom partialzip object

func New

func New(url string) (*PartialZip, error)

New returns a newly created partialzip object.

func (*PartialZip) Download

func (p *PartialZip) Download(path string) (int, error)

Download downloads a file from the remote zip. It returns the number of bytes written and an error, if any.

func (*PartialZip) Get

func (p *PartialZip) Get(path string) (io.ReadCloser, error)

Get gets a handle to a file from the remote zip. It returns an io.ReadCloser and an error, if any.

func (*PartialZip) List

func (p *PartialZip) List() []string

List lists the files in the remote zip. It returns a string array of file paths.