zipper

package module
v1.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 9, 2019 License: MIT Imports: 33 Imported by: 2

README

Zipper Build Status GoDoc

Library to create a zip file from different kind of source, this sources can be:

It also provides a fast way to know when your source change by using signature mechanism. (signatures are not made to verify validity they often try to only make a signature from a small chunk)

You can find a a cli tool directly on this project (see cli section).

Install

Run go get -u github.com/ArthurHlt/zipper/...

Usage

package main

import (
    "github.com/ArthurHlt/zipper"
    "os"
    "io"
)

func main(){
    s, _ := zipper.CreateSession("/a/local/dir", "local") // instead of local you can choose: git or http 
    
    // zipper can auto detect what you want
    s, _ = zipper.CreateSession("/a/local/dir")
    // http support zip (useful for signature), tar or tgz
    // when not any tar, tgz or zip file will be store as it is inside a zi[
    s, _ = zipper.CreateSession("http://user:password@url.com/anexecutable") 
    s, _ = zipper.CreateSession("http://user:password@url.com/afile.zip")
    s, _ = zipper.CreateSession("http://user:password@url.com/afile.tar")
    s, _ = zipper.CreateSession("http://user:password@url.com/afile.tar.gz")
    
    // git support is powerful to target different branch or tag or a commit
    s, _ = zipper.CreateSession("http://user:password@github.com/ArthurHlt/zipper.git")
    s, _ = zipper.CreateSession("http://user:password@github.com/ArthurHlt/zipper.git#branch-or-tag-or-commit")
    
    // create the zip
    zipFile, _ := s.Zip() // zipFile implement io.ReadCloser
    defer zipFile.Close()
    
    zipFile.Size() // get the zip size
    
    // let's create the zip file on your fs
    f, _ := os.Create("myfile.zip")
    defer f.Close()
    io.Copy(f, zipFile)
    
    // Create the signature and use it to see if file change
    sig, _ := s.Sha1()
    
    isDiff, sourceSig, _ := s.IsDiff(sig)
    if isDiff {
        // source files have changed
        // let's update current signature
        sig = sourceSig
    }
}

Source types

Local

Zip from a local directory

  • Type Name: local
  • Auto detection: on an existing folder
  • Valid path:
    • /path/to/a/folder
    • /path/to/a/file.zip
    • /path/to/a/file.tar
    • /path/to/a/file.tar.gz
    • /path/to/a/file.tgz
    • /path/to/a/file.tar.bz2
    • /path/to/a/file.jar
    • /path/to/a/file.war
  • Signature creation: Create signature from the first 5kb of the final zip file.

Tips:

  • Creating a .cfignore, .zipignore or/and .cloudignore in .gitignore style will make zipper ignoring files which match pattern when zipping.
  • Any valid zip content will be interpreted as zip (no need extension on a zip file to recognize it)
  • Any valid tar, tar.gz or tar.bz2 files will be converted as zip (no need extension recognize their types)
Http

Zip from a zip (will be a full http stream in this case), tar or tgz file.

  • Type Name: http
  • Auto detection: on an url with protocol http or https.
  • Valid path:
    • http://url.com/anyfile
    • http://url.com/afile.zip
    • http://url.com/afile.jar
    • http://url.com/afile.war
    • http://url.com/afile.tar
    • http://url.com/afile.tgz
    • http://url.com/afile.tar.gz
  • Signature creation: Create signature from the first 5kb on the remote file (this mean that when calling sha1 this will not download the entire file).

Tips:

  • You can pass user and password for basic auth.
  • Excutable file (elf - linux executable, windows executable, macho - osx executable or file containing shebang) will be store with executable permission
  • Any valid zip content will be interpreted as zip (no need extension on a zip file to recognize it)
  • Any valid tar, tar.gz or tar.bz2 files will be converted as zip (no need extension recognize their types)
Git

Zip from a git repository

  • Type Name: local
  • Auto detection: on an url with protocol http, https or ssh and path ending with .git
  • Valid path:
    • http://github.com/ArthurHlt/zipper.git
    • ssh://git@github.com:ArthurHlt/zipper-fixture.git
    • ssh://git@github.com:ArthurHlt/zipper-fixture.git/folder/in/repo
    • ssh://git@github.com:ArthurHlt/zipper-fixture.git/folder/in/repo?private-key=/pass/to/pem/key&password-key=
    • http://github.com/ArthurHlt/zipper.git#branch-or-tag-or-commit
    • ssh://git@github.com:ArthurHlt/zipper-fixture.git#branch-or-tag-or-commit
  • Signature creation: From commit sha1 (use a bare repo to retrieve it faster).

Tips:

  • Creating a .cfignore or/and a .zipignore in .gitignore style will make zipper ignoring files which match pattern when zipping.
  • You can pass user and password for basic auth.

Cli

You can use a command line to use zipper directly.

If you have set your PATH with $GOPATH/bin you should have zipper available (run zipper -h).

See doc:

NAME:
   zipper - use zipper in cli

USAGE:
   zipper [global options] command [command options] [arguments...]

VERSION:
   1.0.0

COMMANDS:
     zip, z   create zip from a source
     sha1, s  Get sha1 signature for the file from source
     diff, s  Check if file from source is different from your stored sha1
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --type value, -t value  Choose source type
   --insecure, -k          Ignore certificate validation
   --help, -h              show help
   --version, -v           print the version

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var BZ2_FILE_EXT []string = []string{
	".bz2",
}
View Source 
var GZIP_FILE_EXT []string = []string{
	".gz",
	".gzip",
}
View Source 
var TARGZ_FILE_EXT []string = []string{
	".tgz",
}
View Source 
var TAR_FILE_EXT []string = []string{
	".tar",
}
View Source 
var ZIP_FILE_EXT []string = []string{
	".zip",
	".jar",
	".war",
}

Functions

func AddHandler 

func AddHandler(handler Handler) error

For default manager

Add new zip handler to manager

func AddHandlers 

func AddHandlers(handlers ...Handler) error

For default manager

Add new zip handlers to manager

func Chunk 

func Chunk(reader io.Reader, chunkSize int64) ([]byte, error)

func CtxHttpClient 

func CtxHttpClient(src *Source) *http.Client

Retrieve http client set in context This could be use for a zip handler

func GetSha1FromReader 

func GetSha1FromReader(reader io.Reader) (string, error)

Create sha1 from a reader by loading in maximum 5kb

func HasExtFile 

func HasExtFile(path string, extensions ...string) bool

check if file has one if extensions given

func IsExecutable 

func IsExecutable(reader io.Reader) bool

check if file is an executable file

func IsTarFile 

func IsTarFile(path string) bool

check if file is a tar file

func IsTarGzFile 

func IsTarGzFile(path string) bool

check if file is a tgz file

func IsWebURL 

func IsWebURL(path string) bool

check if path is a web url

func IsZipFile 

func IsZipFile(reader io.Reader) bool

check if file is a zip file

func IsZipFileExt 

func IsZipFileExt(path string) bool

check if file is a zip file by extension

func SetCtxHttpClient 

func SetCtxHttpClient(src *Source, client *http.Client)

Set http client in the context of a source This could be use for a zip handler

func SetHttpClient 

func SetHttpClient(httpClient *http.Client)

For default manager

Set a custom http client for zip handlers which need it

Types

type CompressProcessor added in v1.2.0 

type CompressProcessor struct {
	// contains filtered or unexported fields
}

func NewCompressProcessor added in v1.2.0 

func NewCompressProcessor(src *Source, readCloserFunc readCloserFunc) *CompressProcessor

func (CompressProcessor) ToZip added in v1.2.0 

func (p CompressProcessor) ToZip() (ZipReadCloser, error)

type GitHandler 

type GitHandler struct {
	// contains filtered or unexported fields
}

func NewGitHandler 

func NewGitHandler() *GitHandler

func (GitHandler) Detect 

func (h GitHandler) Detect(src *Source) bool

func (GitHandler) Name 

func (h GitHandler) Name() string

func (GitHandler) Sha1 

func (h GitHandler) Sha1(src *Source) (string, error)

func (GitHandler) Zip 

func (h GitHandler) Zip(src *Source) (ZipReadCloser, error)

type GitUtils 

type GitUtils struct {
	Folder     string
	Url        string
	RefName    string
	AuthMethod transport.AuthMethod
	SubPath    string
}

func (GitUtils) Clone 

func (g GitUtils) Clone() error

func (GitUtils) CommitSha1 

func (g GitUtils) CommitSha1() (string, error)

type Handler 

type Handler interface {
	Zip(src *Source) (zip ZipReadCloser, err error)
	Sha1(src *Source) (sha1 string, err error)
	Detect(src *Source) bool
	Name() string
}

func FindHandler 

func FindHandler(path string, handlerName string) (Handler, error)

For default manager

Find zip handler by its type if type is empty string this will use auto-detection

type HttpHandler 

type HttpHandler struct {
}

func (HttpHandler) Detect 

func (h HttpHandler) Detect(src *Source) bool

func (HttpHandler) Name 

func (h HttpHandler) Name() string

func (HttpHandler) Sha1 

func (h HttpHandler) Sha1(src *Source) (string, error)

func (HttpHandler) Zip 

func (h HttpHandler) Zip(src *Source) (ZipReadCloser, error)

type LocalHandler 

type LocalHandler struct {
}

func (LocalHandler) Detect 

func (h LocalHandler) Detect(src *Source) bool

func (LocalHandler) GetZipSize 

func (h LocalHandler) GetZipSize(zipFile *os.File) (int64, error)

func (LocalHandler) Name 

func (h LocalHandler) Name() string

func (LocalHandler) Sha1 

func (h LocalHandler) Sha1(src *Source) (string, error)

func (LocalHandler) Zip 

func (h LocalHandler) Zip(src *Source) (ZipReadCloser, error)

func (LocalHandler) ZipFiles 

func (h LocalHandler) ZipFiles(dirOrZipFilePath string, targetFile *os.File) error

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

func NewManager 

func NewManager(handlers ...Handler) (*Manager, error)

Create new manager with given zip handlers

func (*Manager) AddHandler 

func (m *Manager) AddHandler(handler Handler) error

Add new zip handler to manager

func (*Manager) AddHandlers 

func (m *Manager) AddHandlers(handlers ...Handler) error

Add new zip handlers to manager

func (*Manager) CreateSession 

func (m *Manager) CreateSession(path string, handlerNames ...string) (*Session, error)

Create a session for a given path with given handler type. Omitting handler type will use auto detection)

func (*Manager) FindHandler 

func (m *Manager) FindHandler(path string, handlerName string) (Handler, error)

Find zip handler by its type if type is empty string this will use auto-detection

func (*Manager) SetHttpClient 

func (m *Manager) SetHttpClient(httpClient *http.Client)

Set a custom http client for zip handlers which need it

type Session 

type Session struct {
	// contains filtered or unexported fields
}

func CreateSession 

func CreateSession(path string, handlerNames ...string) (*Session, error)

For default manager

Create a session for a given path with given handler type. Omitting handler type will use auto detection)

func NewSession 

func NewSession(src *Source, handler Handler) *Session

Create a new session

func (Session) Handler 

func (s Session) Handler() Handler

Retrieve handler use in the session

func (Session) IsDiff 

func (s Session) IsDiff(storedSha1 string) (bool, string, error)

Check if source signature is different from a previous signature If true, it's mean than files have changed

func (Session) Sha1 

func (s Session) Sha1() (string, error)

Retrieve signature

func (Session) Source 

func (s Session) Source() *Source

Retrieve source use in the session

func (Session) Zip 

func (s Session) Zip() (ZipReadCloser, error)

Create zip file

type Source 

type Source struct {
	// Path for zip handler
	Path string
	// contains filtered or unexported fields
}

func NewSource 

func NewSource(path string) *Source

Create a new source

func (*Source) Context 

func (s *Source) Context() context.Context

Context returns the request's context. To change the context, use WithContext.

The returned context is always non-nil; it defaults to the background context.

For outgoing client requests, the context controls cancelation.

For incoming server requests, the context is canceled when the client's connection closes, the request is canceled (with HTTP/2), or when the ServeHTTP method returns.

func (*Source) WithContext 

func (s *Source) WithContext(ctx context.Context) *Source

WithContext returns a shallow copy of r with its context changed to ctx. The provided ctx must be non-nil.

type SourceContextKey 

type SourceContextKey int
const (
	HttpClientContextKey SourceContextKey = iota
)

type ZipFile 

type ZipFile struct {
	// contains filtered or unexported fields
}

func NewZipFile 

func NewZipFile(file io.ReadCloser, size int64, clean func() error) *ZipFile

Create a new ZipFile

func (ZipFile) Close 

func (f ZipFile) Close() error

func (ZipFile) Read 

func (f ZipFile) Read(p []byte) (int, error)

func (ZipFile) Size 

func (f ZipFile) Size() int64

Retrieve size of the zip file

type ZipReadCloser 

type ZipReadCloser interface {
	io.ReadCloser
	Size() int64
}

Source Files

View all Source files

Directories

Path Synopsis
cli
zipper
come from
 come from
Code generated by counterfeiter.
 Code generated by counterfeiter.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.