patcher

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 28, 2023 License: MIT Imports: 22 Imported by: 0

Documentation

Index

Constants

View Source 
const ManifestFilename = "ta-manifest.json"

Filename for the manifest under the root dir.

Variables

This section is empty.

Functions

func CreateWithSizeHint 

func CreateWithSizeHint(filename string, size int64) (file *os.File, err error)

CreateWithSizeHint creates a file, reserving space to grow it to size.

func DetermineFilesToMeasure 

func DetermineFilesToMeasure(
	instructions []Instruction,
	manifest *Manifest,
	existingFiles map[string]BasicFileInfo,
) ([]string, map[string]string)

DetermineFilesToVerify given the raw instructions, a manifest (empty if it doesn't exist yet) and metadata of existing files returns a list of files that should be measured (checksum taken) and hashes of existing files that match the manifest.

func DoInParallel 

func DoInParallel[TIn any](
	ctx context.Context,
	execute func(context.Context, TIn) error,
	input []TIn,
	numWorkers int,
) error

DoInParallel is like DoInParallelWithResult but without collecting results.

func DoInParallelWithResult 

func DoInParallelWithResult[TIn any, TOut any](
	ctx context.Context,
	execute func(context.Context, TIn) (TOut, error),
	input []TIn,
	numWorkers int,
) ([]TOut, error)

DoInParallelWithResult runs the execute function for each element in the input slice, processing at most numWorkers at a time. If one of the workers errors the context passed to the others is cancelled.

The execute func should return context.Canceled if it stops due to context being cancelled.

The error returned is an error of a failing worker. If the whole operation is cancelled through the ctx context the error is context.Canceled.

func HashBytes 

func HashBytes(data []byte) string

HashBytes generates a SHA256 hash of a byte slice.

func HashEqual 

func HashEqual(hash1 string, hash2 string) bool

HashEqual compares two hashes for equality.

func HashReader 

func HashReader(ctx context.Context, s io.Reader) (string, error)

HashReader reads data via a reader and computes a SHA256 hash of it.

func LogVerbose 

func LogVerbose(ctx context.Context, format string, args ...any)

LogVerbose logs a message only if the verbose flag is set.

func RunPatcher 

func RunPatcher(ctx context.Context, instructions []Instruction, config PatcherConfig) error

func ScanFiles 

func ScanFiles(rootDir string) (map[string]BasicFileInfo, error)

ScanFiles recursively determines all files in the directory and gets limited information such as modification time.

func SetVerbose 

func SetVerbose(ctx context.Context, verbose bool) context.Context

SetVerbose sets the verbose flag for logging.

Types

type Averager 

type Averager struct {
	// contains filtered or unexported fields
}

An Averager computes a running average of measurements.

func NewAverager 

func NewAverager(window int) *Averager

NewAverager creates an averager with a particular window.

func (*Averager) Add 

func (a *Averager) Add(measurement float64)

Add adds a measurement to the averager, overwriting the oldest measurement if there are more measurements than the window.

func (*Averager) Average 

func (a *Averager) Average() float64

Average computes the current running average. Returns 0 if there are no measurements.

type BasicFileInfo 

type BasicFileInfo struct {
	// When the file was last modified.
	ModTime time.Time
}

Limited information about a file.

type DeterminedActions 

type DeterminedActions struct {
	// Which patch files to download.
	ToDownload []DownloadInstr
	// Which game files to install/update.
	ToUpdate []UpdateInstr
	// Which game files to delete.
	ToDelete []string
}

DeterminedActions are the result of DetermineActions.

func DetermineActions 

func DetermineActions(
	instructions []Instruction,
	manifest *Manifest,
	existingFiles map[string]BasicFileInfo,
	fileChecksums map[string]string,
) DeterminedActions

DetermineActions determines what should be downloaded and what should be patched/deleted. It receives the same data as DetermineFilesToMeasure and additionally the combined results of result of file measurement and looking up files in the manifest.

type DownloadConfig 

type DownloadConfig struct {
	// Maximum number of attempts.
	MaxAttempts int

	// Minimum time between retries.
	RetryBaseDelay time.Duration

	// How much to increment the delay between retries (factor, so 2 = double every retry).
	RetryWaitIncrementFactor float64

	// How many seconds to average the download speed over.
	DownloadSpeedWindow int

	// How much time to allow to send a request and receive the start of a response.
	DownloadRequestTimeout time.Duration

	// How much time to allow between receiving any data in a download.
	DownloadStallTimeout time.Duration
}

A DownloadConfig is the configuration for a Downloader.

type DownloadInstr 

type DownloadInstr struct {
	// Location of the patch file relative to the directory containing instructions.json.
	RemotePath string

	// Filename for the patch file on disk.
	LocalPath string

	// Checksum the patch file should have.
	Checksum string

	// Size of the patch file in bytes.
	Size int64
}

A DownloadInstr indicates how to download a patch file.

type DownloadStats 

type DownloadStats struct {
	// Running average of download speed in bytes/second.
	Speed int64

	// Total number of bytes downloaded.
	TotalBytes int64
}

DownloadStats are current information about the download activity.

type Downloader 

type Downloader struct {
	// contains filtered or unexported fields
}

A Downloader manages downloads. Mainly it keeps track of progress and download speed.

func NewDownloader 

func NewDownloader(
	config DownloadConfig,
	tickFunc func(DownloadStats),
	tickFuncCtx context.Context,
) *Downloader

NewDownloader creates a new downloader. Pass configuration and a function that will receive the download stats every second. Will run the tick func every second until the context is canceled.

func (*Downloader) DownloadFile 

func (d *Downloader) DownloadFile(
	ctx context.Context,
	downloadUrl *url.URL,
	filename string,
	expectedChecksum string,
	expectedSize int64,
) error

DownloadFile downloads a file to disk. It also verifies a SHA256 hash.

The caller must guarantee that DownloadFile is never called twice for the same filename (this can happen if two output files in the instruction list have the same content).

type Instruction 

type Instruction struct {
	// Path relative to install dir.  This is backslash encoded in the JSON but
	// DecodeInstructions will transform backslashes to slashes.
	Path string
	// For a delta patch the hash that the existing file should have.
	OldHash string
	// The hash the file should have after patching. If nil the file should be deleted.
	NewHash *string
	// The hash of the full (not delta) patch file.
	CompressedHash *string
	// If a delta patch exists the hash of the delta patch file.
	DeltaHash *string
	// Size of the file. May be 0 as this is a new field and instructions.json might not have it yet.
	FileSize int64 `json:"FileSize"`
	// Size in bytes of the full patch file.
	FullReplaceSize int64
	// Size in bytes of the delta patch file. Zero if there is no delta patch file.
	DeltaSize int64
}

An Instruction contains the relevant part of an instruction from instructions.json with some minor processing.

func DecodeInstructions 

func DecodeInstructions(jsonData []byte) ([]Instruction, error)

DecodeInstructions decodes instructions.json and runs some basic sanity checks.

type Manifest 

type Manifest struct {
	// Identifier for whatever game is installed, something like "renx_alpha".
	Product string
	Entries map[string]ManifestEntry
}

A Manifest records the last recorded checksum and change time for files. It's used to bypass expensive computation of the checksum for a file. A Manifest is a map from relative filename (with OS specific separator) to manifest data. E.g. a key can be "Binaries\InstallData\dotNetFx40_Full_setup.exe" on Windows.

func NewManifest 

func NewManifest(product string) *Manifest

NewManifest creates a new empty manifest for the product.

func ReadManifest 

func ReadManifest(installDir string, product string) (*Manifest, error)

ReadManifest reads a manifest from the standard location in the installation dir. Verifies that the manifest has the correct product field set. Returns an empty manifest if there's no manifest file.

func (*Manifest) Add 

func (m *Manifest) Add(filename string, lastChange time.Time, checksum string)

Add adds a file along with last change info and known checksum to the manifest. Overwrites an existing entry for the file.

func (*Manifest) Check 

func (m *Manifest) Check(filename string, lastChange time.Time, checksum string) bool

Check returns true iff a file with the given name, last change time and checksum exists in the manifest. (I.e. if a file can be assumed to have the correct checksum.)

func (*Manifest) Get 

func (m *Manifest) Get(filename string, lastChange time.Time) (string, bool)

Get returns the checksum of a file if one exists with the given name and last change time.

func (*Manifest) WriteManifest 

func (m *Manifest) WriteManifest(installDir string) error

WriteManifest writes a manifest to the standard location in the installation dir.

type ManifestEntry 

type ManifestEntry struct {
	LastChange   time.Time `json:"last_change"`
	LastChecksum string    `json:"last_checksum"`
}

A ManifestEntry records the last recorded checksum and change time for a file.

type PatcherConfig 

type PatcherConfig struct {
	// URL containing instructions.json
	BaseUrl *url.URL

	// Directory where the game should be installed.
	InstallDir string

	// Product name that should be stored in the manifest.
	Product string

	// How many concurrent workers in verify phase.
	VerifyWorkers int

	// How many concurrent workers in download phase.
	DownloadWorkers int

	// How many concurrent workers in apply phase.
	ApplyWorkers int

	// Configuration of the download system.
	DownloadConfig DownloadConfig

	// Where to find the xdelta binary. If just a basename without directory
	// will look in PATH and also in the current directory.
	XDeltaBinPath string

	// A function that gets called every few seconds with the current progress
	// until the context passed to RunPatcher is canceled.
	ProgressFunc func(Progress)

	// How often to call ProgressFunc.
	ProgressInterval time.Duration
}

type Phase 

type Phase int
const (
	PhaseVerify   Phase = 0
	PhaseDownload Phase = 1
	PhaseApply    Phase = 2
)

type Progress 

type Progress struct {
	// Running average of download speed in bytes per second.
	DownloadSpeed int64 `json:"downloadSpeed"`

	// Total bytes downloaded.
	DownloadTotalBytes int64 `json:"downloadTotalBytes"`

	// Progress in the verify phase.
	Verify ProgressPhase `json:"verify"`

	// Progress in the download phase.
	Download ProgressPhase `json:"download"`

	// Progress in the apply phase.
	Apply ProgressPhase `json:"apply"`
}

Progress is current progress information. Beware that this gets directly serialized for JSON progress output,

func (*Progress) GetPhase 

func (p *Progress) GetPhase(phase Phase) *ProgressPhase

GetPhase returns a phase by number.

type ProgressPhase 

type ProgressPhase struct {
	// How many items are being processed.
	Processing int `json:"processing"`
	// How many items have been successfully processed.
	Completed int `json:"completed"`
	// How many items have errored.
	Errors int `json:"errors"`
	// How many items should be processed.
	Needed int `json:"needed"`
	// Whether the Needed value is known. If this is false the Needed value
	// will be 0 but doesn't mean anything yet.
	NeededKnown bool `json:"needed_known"`
	// Whether the phase is completed.
	// In the case of completed == 0 the phase might be not started yet
	// or completed.
	Done bool `json:"done"`
	// How much time was spent in the stage at the last time progress was reported, in seconds.
	Duration int `json:"duration"`
	// contains filtered or unexported fields
}

ProgressPhase contains the progress in a particular phase.

type ProgressTracker 

type ProgressTracker struct {
	// contains filtered or unexported fields
}

ProgressTracker is used to track the progress of the patching process.

func NewProgress 

func NewProgress() *ProgressTracker

NewProgress creates a progress tracker.

func (*ProgressTracker) Current 

func (p *ProgressTracker) Current() Progress

Current returns a copy of the current progress with duration calculated correctly.

func (*ProgressTracker) PhaseDone 

func (p *ProgressTracker) PhaseDone(phase Phase)

PhaseDone marks a phase as finished.

func (*ProgressTracker) PhaseItemDone 

func (p *ProgressTracker) PhaseItemDone(phase Phase, err error)

PhaseItemDone increments the errors or completed value in a phase and decreases the processing value.

func (*ProgressTracker) PhaseItemStarted 

func (p *ProgressTracker) PhaseItemStarted(phase Phase)

PhaseItemStarted increments the processing value in a phase.

func (*ProgressTracker) PhaseItemsSkipped 

func (p *ProgressTracker) PhaseItemsSkipped(phase Phase, count int)

PhaseItemsSkipped increases the completed count for a phase without putting items in processing.

func (*ProgressTracker) PhaseSetNeeded 

func (p *ProgressTracker) PhaseSetNeeded(phase Phase, needed int)

PhaseSetNeeded sets the needed value for a phase.

func (*ProgressTracker) PhaseStarted 

func (p *ProgressTracker) PhaseStarted(phase Phase)

PhaseStarted marks a phase as started.

func (*ProgressTracker) UpdateDownloadStats 

func (p *ProgressTracker) UpdateDownloadStats(stats DownloadStats)

UpdateDownloadStats updates the download related statistics.

type ResolvedInstructions 

type ResolvedInstructions struct {
	Instructions []Instruction
	BaseUrl      *url.URL
	VersionName  string
}

func ResolveInstructions 

func ResolveInstructions(productsUrl *url.URL, product string) (*ResolvedInstructions, error)

ResolveInstructions finds the instructions and URL containing the patch files by looking up a product through the root products.json file.

type UpdateInstr 

type UpdateInstr struct {
	// Filename for the patch file on disk.
	PatchPath string

	// Filename for the file to patch.
	FilePath string

	// Temporary filename for the new file.
	TempFilename string

	// Whether the patch should be applied as a delta patch.
	IsDelta bool

	// The final hash the file should have.
	Checksum string

	// The size the file is expected to have. It may be 0 if this is unknown.
	Size int64
}

An UpdateInstr indicates how to apply a patch.

type XDelta 

type XDelta struct {
	// contains filtered or unexported fields
}

An XDelta instance provides helpers for invoking the xdelta program.

func NewXDelta 

func NewXDelta(binPath string) (*XDelta, error)

Create an XDelta instance.

If the binPath is just a basename without directory it will be looked up in PATH. To use binary in the current directory use something like './xdelta3'.

func (XDelta) ApplyPatch 

func (x XDelta) ApplyPatch(
	ctx context.Context,
	oldPath *string,
	patchPath string,
	newPath string,
	expectedChecksum string,
	expectedSize int64,
) error

ApplyPatch runs the xdelta binary, outputting to newPath, validating the checksum at the same time. If oldPath is not nil it's a delta patch, otherwise it's a full patch.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.