dap

README

deterministic-archive-proxy

Always get the same archive from GitHub (and others).

Save cache misses that are solely due to

• filenames being in a different order
• timestamps of identical files being different
• users, groups or permissions changing

Usage

Start the proxy

go build cmd/proxy/proxy.go && PORT=8888 ./proxy

Update your http_archives

http_archive(
    name = "bazel_skylib",
    type = "tar.gz",
    url = "https://localhost:8080/github.com/bazelbuild/bazel-skylib/archive/1.0.3.tar.gz",
    sha256 = "868350c41188cda1b4923c0df62b06d7dbd977527b576eb1d2515da631709c40",
)

Documentation

Index

• type GitHubHandler
  • func NewGitHubHandler(opts ...Option) *GitHubHandler
  • func (ghh *GitHubHandler) DefaultSendFile(ctx context.Context, uri string, rw http.ResponseWriter, rep *http.Response) error
  • func (ghh *GitHubHandler) ServeHTTP(rw http.ResponseWriter, r *http.Request)
• type Option
  • func WithAuthToken(token string) Option
  • func WithHTTPClient(client *http.Client) Option
  • func WithSendFiler(fn SendFiler) Option
• type SendFiler

Types

type GitHubHandler

type GitHubHandler struct {
	// contains filtered or unexported fields
}

GitHubHandler proxies requests for GitHub archives following the pattern https://github.com/<user>/<repo>/archive/<hash>.zip or https://github.com/<user>/<repo>/archive/<hash>.tar.gz It returns deterministic archives or GitHub's own server error.

func NewGitHubHandler

func NewGitHubHandler(opts ...Option) *GitHubHandler

NewGitHubHandler creates a new GitHubHandler

func (*GitHubHandler) DefaultSendFile

func (ghh *GitHubHandler) DefaultSendFile(ctx context.Context, uri string, rw http.ResponseWriter, rep *http.Response) error

DefaultSendFile copies the response headers (except content-length, etag and authorization), then recomputes the archive so it is deterministic and streams the result.

func (*GitHubHandler) ServeHTTP

func (ghh *GitHubHandler) ServeHTTP(rw http.ResponseWriter, r *http.Request)

type Option

type Option func(*options) error

Option specializes a handler.

func WithAuthToken

func WithAuthToken(token string) Option

WithAuthToken sets the token to use when requesting tarballs from upstream.

func WithHTTPClient

func WithHTTPClient(client *http.Client) Option

WithHTTPClient sets the HTTP client to use when requesting the original archive. By default, http.DefaultClient is used which is safe for concurrent use by multiple goroutines.

func WithSendFiler

func WithSendFiler(fn SendFiler) Option

WithSendFiler controls how the HTTP response is written. Override this to use e.g. caching.

type SendFiler

type SendFiler func(ctx context.Context, uri string, rw http.ResponseWriter, rep *http.Response) error

SendFiler writes the HTTP response.