README
¶
Kopia
n.
Kopia is a fast and secure open-source backup/restore tool that allows you to create encrypted snapshots of your data and save the snapshots to remote or cloud storage of your choice, to network-attached storage or server, or locally on your machine. Kopia does not 'image' your whole machine. Rather, Kopia allows you to backup/restore any and all files/directories that you deem are important or critical.
Kopia has both CLI (command-line interface) and GUI (graphical user interface) versions, making it the perfect tool for both advanced and regular users. You can read more about Kopia's unique features -- which include compression, deduplication, end-to-end 'zero knowledge' encryption, and error correction -- to get a better understanding of how Kopia works.
When ready, head to the installation page to download and install Kopia, and make sure to read the Getting Started Guide for a step-by-step walkthrough of how to use Kopia.
Pick the Cloud Storage Provider You Want
Kopia supports saving your encrypted and compressed snapshots to all of the following storage locations:
- Amazon S3 and any cloud storage that is compatible with S3
- Azure Blob Storage
- Backblaze B2
- Google Cloud Storage
- Any remote server or cloud storage that supports WebDAV
- Any remote server or cloud storage that supports SFTP
- Some of the cloud storage options supported by Rclone
- Requires you to download and setup Rclone in addition to Kopia, but after that Kopia manages/runs Rclone for you
- Rclone support is experimental: not all the cloud storage products supported by Rclone have been tested to work with Kopia, and some may not work with Kopia; Kopia has been tested to work with Dropbox, OneDrive, and Google Drive through Rclone
- Your local machine and any network-attached storage or server
- Your own server by setting up a Kopia Repository Server
And Kopia uses data deduplication to save you money! Read the repositories help page for more information on supported storage locations.
With Kopia you are in full control of where to store your snapshots, that is, you pick the storage provider you want to use. You must provision and pay for the storage provider for whatever storage locations you want to use, and then tell Kopia what those storage locations are. You can even use multiple storage locations for different backup repositories if you want. Kopia also supports backing up multiple machines to the same storage location.
Kopia in Action
Using Kopia via command-line interface:
Using Kopia via graphical user interface (note: the video is of an older version of Kopia and the interface is different in the current version of Kopia, but the main principles of the interface are the same):
Getting Started
See Kopia Documentation for more information.
Building Kopia
See Build Infrastructure for more information on building Kopia and working with the source code.
Licensing
Kopia is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.
Contribution Guidelines
Kopia is open source and contributions are welcome. For more information on how to contribute see the Contribution Guidelines.
Reporting Security Issues
If you find a security issue you'd like to disclose privately, please contact security@kopia.io or via direct message to maintainers on Slack.
Documentation
¶
Overview ¶
Command-line tool for creating and accessing backups.
Usage:
$ kopia [<flags>] <subcommand> [<args> ...]
Use 'kopia help' to see more details.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cli implements command-line commands for the Kopia.
|
Package cli implements command-line commands for the Kopia. |
|
Package fs defines virtual filesystem abstractions.
|
Package fs defines virtual filesystem abstractions. |
|
cachefs
Package cachefs implements a wrapper that caches filesystem actions.
|
Package cachefs implements a wrapper that caches filesystem actions. |
|
ignorefs
Package ignorefs implements a wrapper that hides ignored files listed in '.kopiaignore' and in policies attached to directories.
|
Package ignorefs implements a wrapper that hides ignored files listed in '.kopiaignore' and in policies attached to directories. |
|
localfs
Package localfs implements virtual filesystem abstraction for a local filesystem.
|
Package localfs implements virtual filesystem abstraction for a local filesystem. |
|
loggingfs
Package loggingfs implements a wrapper that logs all filesystem actions.
|
Package loggingfs implements a wrapper that logs all filesystem actions. |
|
virtualfs
Package virtualfs implements an in-memory abstraction of fs.Directory and fs.StreamingFile.
|
Package virtualfs implements an in-memory abstraction of fs.Directory and fs.StreamingFile. |
|
internal
|
|
|
acl
Package acl provides management of ACLs that define permissions granted to repository users.
|
Package acl provides management of ACLs that define permissions granted to repository users. |
|
apiclient
Package apiclient implements a client for connecting to Kopia HTTP API server.
|
Package apiclient implements a client for connecting to Kopia HTTP API server. |
|
atomicfile
Package atomicfile provides wrappers for atomically writing files in a manner compatible with long filenames.
|
Package atomicfile provides wrappers for atomically writing files in a manner compatible with long filenames. |
|
auth
Package auth provides authentication and authorization constructs.
|
Package auth provides authentication and authorization constructs. |
|
bigmap
Package bigmap implements a custom hashmap data structure where keys and values are binary and keys are meant to be well-distributed hashes, such as content IDs, object IDs, etc.
|
Package bigmap implements a custom hashmap data structure where keys and values are binary and keys are meant to be well-distributed hashes, such as content IDs, object IDs, etc. |
|
bigmap/bigmapbench
Command bigmapbench provides a benchmark for the bigmap implementation.
|
Command bigmapbench provides a benchmark for the bigmap implementation. |
|
blobcrypto
Package blobcrypto performs whole-blob crypto operations.
|
Package blobcrypto performs whole-blob crypto operations. |
|
blobtesting
Package blobtesting is used for testing BLOB storage implementations.
|
Package blobtesting is used for testing BLOB storage implementations. |
|
cache
Package cache implements durable on-disk cache with LRU expiration.
|
Package cache implements durable on-disk cache with LRU expiration. |
|
cachedir
Package cachedir contains utilities for manipulating cache directories.
|
Package cachedir contains utilities for manipulating cache directories. |
|
cacheprot
Package cacheprot provides utilities for protection of cache entries.
|
Package cacheprot provides utilities for protection of cache entries. |
|
clock
Package clock provides indirection for accessing current wall clock time.
|
Package clock provides indirection for accessing current wall clock time. |
|
completeset
Package completeset manages complete set of blob metadata.
|
Package completeset manages complete set of blob metadata. |
|
connection
Package connection manages (abstract) connections with retrying and reconnection.
|
Package connection manages (abstract) connections with retrying and reconnection. |
|
crypto
Package crypto implements common symmetric-encryption and key-derivation functions.
|
Package crypto implements common symmetric-encryption and key-derivation functions. |
|
diff
Package diff implements helpers for comparing two filesystems.
|
Package diff implements helpers for comparing two filesystems. |
|
dirutil
Package dirutil contains directory manipulation helpers.
|
Package dirutil contains directory manipulation helpers. |
|
editor
Package editor encapsulates working with external text editor.
|
Package editor encapsulates working with external text editor. |
|
epoch
Package epoch manages repository epochs.
|
Package epoch manages repository epochs. |
|
faketime
Package faketime fakes time for tests
|
Package faketime fakes time for tests |
|
fault
Package fault defines fault behaviors.
|
Package fault defines fault behaviors. |
|
feature
Package feature tracks features that are supported by Kopia client to ensure forwards and backwards compatibility.
|
Package feature tracks features that are supported by Kopia client to ensure forwards and backwards compatibility. |
|
freepool
Package freepool manages a free pool of objects that are expensive to create.
|
Package freepool manages a free pool of objects that are expensive to create. |
|
fshasher
Package fshasher computes a fingerprint for an FS tree for testing purposes
|
Package fshasher computes a fingerprint for an FS tree for testing purposes |
|
fusemount
Package fusemount implements FUSE filesystem nodes for mounting contents of filesystem stored in repository.
|
Package fusemount implements FUSE filesystem nodes for mounting contents of filesystem stored in repository. |
|
gather
Package gather implements data structures storing binary data organized in a series of byte slices of fixed size that only gathered together by the user.
|
Package gather implements data structures storing binary data organized in a series of byte slices of fixed size that only gathered together by the user. |
|
hmac
Package hmac contains utilities for dealing with HMAC checksums.
|
Package hmac contains utilities for dealing with HMAC checksums. |
|
impossible
Package impossible provides PanicOnError which panics on impossible conditions.
|
Package impossible provides PanicOnError which panics on impossible conditions. |
|
indextest
Package indextest provides utilities for testing content index.
|
Package indextest provides utilities for testing content index. |
|
iocopy
Package iocopy is a wrapper around io.Copy() that recycles shared buffers.
|
Package iocopy is a wrapper around io.Copy() that recycles shared buffers. |
|
jsonencoding
Package jsonencoding defines common types with JSON marshalers.
|
Package jsonencoding defines common types with JSON marshalers. |
|
listcache
Package listcache defines a blob.Storage wrapper that caches results of list calls for short duration of time.
|
Package listcache defines a blob.Storage wrapper that caches results of list calls for short duration of time. |
|
logfile
Package logfile manages log files.
|
Package logfile manages log files. |
|
metricid
Package metricid provides mapping between metric names and persistent IDs.
|
Package metricid provides mapping between metric names and persistent IDs. |
|
metrics
Package metrics provides unified way of emitting metrics inside Kopia.
|
Package metrics provides unified way of emitting metrics inside Kopia. |
|
mockfs
Package mockfs implements in-memory filesystem for testing.
|
Package mockfs implements in-memory filesystem for testing. |
|
mount
Package mount manages creating operating system mount points for directory snapshots.
|
Package mount manages creating operating system mount points for directory snapshots. |
|
osexec
Package osexec provides OS-dependent utilities for running child processes.
|
Package osexec provides OS-dependent utilities for running child processes. |
|
ospath
Package ospath provides discovery of OS-dependent paths.
|
Package ospath provides discovery of OS-dependent paths. |
|
ownwrites
Package ownwrites defines a blob.Storage wrapper that ensures that recently-written blobs show up in ListBlob() results, if the underlying provider is eventually consistent when it comes to list results.
|
Package ownwrites defines a blob.Storage wrapper that ensures that recently-written blobs show up in ListBlob() results, if the underlying provider is eventually consistent when it comes to list results. |
|
parallelwork
Package parallelwork implements parallel work queue with fixed number of workers that concurrently process and add work items to the queue.
|
Package parallelwork implements parallel work queue with fixed number of workers that concurrently process and add work items to the queue. |
|
passwordpersist
Package passwordpersist manages password persistence.
|
Package passwordpersist manages password persistence. |
|
pproflogging
Package pproflogging for pproflogging helper functions.
|
Package pproflogging for pproflogging helper functions. |
|
providervalidation
Package providervalidation implements validation to ensure the blob storage is compatible with Kopia requirements.
|
Package providervalidation implements validation to ensure the blob storage is compatible with Kopia requirements. |
|
releasable
Package releasable allows process-wide tracking of objects that need to be released.
|
Package releasable allows process-wide tracking of objects that need to be released. |
|
repodiag
Package repodiag manages logs and metrics in the repository.
|
Package repodiag manages logs and metrics in the repository. |
|
repotesting
Package repotesting contains test utilities for working with repositories.
|
Package repotesting contains test utilities for working with repositories. |
|
retry
Package retry implements exponential retry policy.
|
Package retry implements exponential retry policy. |
|
scheduler
Package scheduler implements a simple scheduler that triggers the next item when its due time is reached based on the list of upcoming items.
|
Package scheduler implements a simple scheduler that triggers the next item when its due time is reached based on the list of upcoming items. |
|
scrubber
Package scrubber contains helpers that remove sensitive information from Go structs before it's presented to users.
|
Package scrubber contains helpers that remove sensitive information from Go structs before it's presented to users. |
|
server
Package server implements Kopia API server handlers.
|
Package server implements Kopia API server handlers. |
|
serverapi
Package serverapi contains GO types corresponding to Kopia server API.
|
Package serverapi contains GO types corresponding to Kopia server API. |
|
servertesting
Package servertesting provides helpers for launching and testing Kopia server.
|
Package servertesting provides helpers for launching and testing Kopia server. |
|
sparsefile
Package sparsefile provides wrappers for handling the writing of sparse files (files with holes).
|
Package sparsefile provides wrappers for handling the writing of sparse files (files with holes). |
|
stat
Package stat provides a cross-platform abstraction for common stat commands.
|
Package stat provides a cross-platform abstraction for common stat commands. |
|
stats
Package stats provides helpers for simple stats
|
Package stats provides helpers for simple stats |
|
tempfile
Package tempfile provides a cross-platform abstraction for creating private read-write temporary files which are automatically deleted when closed.
|
Package tempfile provides a cross-platform abstraction for creating private read-write temporary files which are automatically deleted when closed. |
|
testlogging
Package testlogging implements logger that writes to testing.T log.
|
Package testlogging implements logger that writes to testing.T log. |
|
testutil
Package testutil contains test utilities.
|
Package testutil contains test utilities. |
|
timestampmeta
Package timestampmeta provides utilities for preserving timestamps using per-blob key-value-pairs (metadata, tags, etc.)
|
Package timestampmeta provides utilities for preserving timestamps using per-blob key-value-pairs (metadata, tags, etc.) |
|
timetrack
Package timetrack tracks the progress and estimates completion of a task.
|
Package timetrack tracks the progress and estimates completion of a task. |
|
tlsutil
Package tlsutil contains TLS utilities.
|
Package tlsutil contains TLS utilities. |
|
uitask
Package uitask provided management of in-process long-running tasks that are exposed to the UI.
|
Package uitask provided management of in-process long-running tasks that are exposed to the UI. |
|
units
Package units contains helpers to convert sizes to human-readable strings.
|
Package units contains helpers to convert sizes to human-readable strings. |
|
user
Package user provides management of user accounts.
|
Package user provides management of user accounts. |
|
volumesizeinfo
Package volumesizeinfo contains helpers to obtain information about volume.
|
Package volumesizeinfo contains helpers to obtain information about volume. |
|
wcmatch
Package wcmatch implements wildcard matching files using .gitignore syntax.
|
Package wcmatch implements wildcard matching files using .gitignore syntax. |
|
webdavmount
Package webdavmount implements webdav filesystem for serving snapshots.
|
Package webdavmount implements webdav filesystem for serving snapshots. |
|
workshare
Package workshare implements work sharing worker pool.
|
Package workshare implements work sharing worker pool. |
|
zaplogutil
Package zaplogutil provides reusable utilities for working with ZAP logger.
|
Package zaplogutil provides reusable utilities for working with ZAP logger. |
|
Package notification provides a mechanism to send notifications for various events.
|
Package notification provides a mechanism to send notifications for various events. |
|
notifydata
Package notifydata contains the data structures used by the notification package.
|
Package notifydata contains the data structures used by the notification package. |
|
notifyprofile
Package notifyprofile notification profile management.
|
Package notifyprofile notification profile management. |
|
notifytemplate
Package notifytemplate provides a way to access notification templates.
|
Package notifytemplate provides a way to access notification templates. |
|
sender
Package sender provides a common interface for sending notifications.
|
Package sender provides a common interface for sending notifications. |
|
sender/email
Package email provides email notification support.
|
Package email provides email notification support. |
|
sender/jsonsender
Package jsonsender provides a notification sender that writes messages in JSON format to the provided writer.
|
Package jsonsender provides a notification sender that writes messages in JSON format to the provided writer. |
|
sender/pushover
Package pushover provides pushover notification support.
|
Package pushover provides pushover notification support. |
|
sender/testsender
Package testsender provides notification sender testing support.
|
Package testsender provides notification sender testing support. |
|
sender/webhook
Package webhook provides webhook notification support.
|
Package webhook provides webhook notification support. |
|
Package repo implements content-addressable Repository on top of BLOB storage.
|
Package repo implements content-addressable Repository on top of BLOB storage. |
|
blob
Package blob implements simple storage of immutable, unstructured binary large objects (BLOBs).
|
Package blob implements simple storage of immutable, unstructured binary large objects (BLOBs). |
|
blob/azure
Package azure implements Azure Blob Storage.
|
Package azure implements Azure Blob Storage. |
|
blob/b2
Package b2 implements Storage based on an Backblaze B2 bucket.
|
Package b2 implements Storage based on an Backblaze B2 bucket. |
|
blob/beforeop
Package beforeop implements wrapper around blob.Storage that run a given callback before all operations.
|
Package beforeop implements wrapper around blob.Storage that run a given callback before all operations. |
|
blob/filesystem
Package filesystem implements filesystem-based Storage.
|
Package filesystem implements filesystem-based Storage. |
|
blob/gcs
Package gcs implements Storage based on Google Cloud Storage bucket.
|
Package gcs implements Storage based on Google Cloud Storage bucket. |
|
blob/gdrive
Package gdrive implements Storage based on Google Drive.
|
Package gdrive implements Storage based on Google Drive. |
|
blob/logging
Package logging implements wrapper around Storage that logs all activity.
|
Package logging implements wrapper around Storage that logs all activity. |
|
blob/rclone
Package rclone implements blob storage provider proxied by rclone (http://rclone.org)
|
Package rclone implements blob storage provider proxied by rclone (http://rclone.org) |
|
blob/readonly
Package readonly implements wrapper around readonlyStorage that prevents all mutations.
|
Package readonly implements wrapper around readonlyStorage that prevents all mutations. |
|
blob/retrying
Package retrying implements wrapper around blob.Storage that adds retry loop around all operations in case they return unexpected errors.
|
Package retrying implements wrapper around blob.Storage that adds retry loop around all operations in case they return unexpected errors. |
|
blob/s3
Package s3 implements Storage based on an S3 bucket.
|
Package s3 implements Storage based on an S3 bucket. |
|
blob/sftp
Package sftp implements blob storage provided for SFTP/SSH.
|
Package sftp implements blob storage provided for SFTP/SSH. |
|
blob/sharded
Package sharded implements common support for sharded blob providers, such as filesystem or webdav.
|
Package sharded implements common support for sharded blob providers, such as filesystem or webdav. |
|
blob/storagemetrics
Package storagemetrics implements wrapper around Storage that adds metrics around all activity.
|
Package storagemetrics implements wrapper around Storage that adds metrics around all activity. |
|
blob/throttling
Package throttling implements wrapper around blob.Storage that adds throttling to all calls.
|
Package throttling implements wrapper around blob.Storage that adds throttling to all calls. |
|
blob/webdav
Package webdav implements WebDAV-based Storage.
|
Package webdav implements WebDAV-based Storage. |
|
compression
Package compression manages compression algorithm implementations.
|
Package compression manages compression algorithm implementations. |
|
content
Package content implements repository support for content-addressable storage.
|
Package content implements repository support for content-addressable storage. |
|
content/index
Package index manages content indices.
|
Package index manages content indices. |
|
content/indexblob
Package indexblob manages sets of active index blobs.
|
Package indexblob manages sets of active index blobs. |
|
ecc
Package ecc implements common support for error correction in sharded blob providers
|
Package ecc implements common support for error correction in sharded blob providers |
|
encryption
Package encryption manages content encryption algorithms.
|
Package encryption manages content encryption algorithms. |
|
format
Package format manages kopia.repository and other central format blobs.
|
Package format manages kopia.repository and other central format blobs. |
|
hashing
Package hashing encapsulates all keyed hashing algorithms.
|
Package hashing encapsulates all keyed hashing algorithms. |
|
logging
Package logging provides loggers for Kopia.
|
Package logging provides loggers for Kopia. |
|
maintenance
Package maintenance manages automatic repository maintenance.
|
Package maintenance manages automatic repository maintenance. |
|
manifest
Package manifest implements support for managing JSON-based manifests in repository.
|
Package manifest implements support for managing JSON-based manifests in repository. |
|
object
Package object implements repository support for content-addressable objects of arbitrary size.
|
Package object implements repository support for content-addressable objects of arbitrary size. |
|
splitter
Package splitter manages splitting of object data into chunks.
|
Package splitter manages splitting of object data into chunks. |
|
Package snapshot manages metadata about snapshots stored in repository.
|
Package snapshot manages metadata about snapshots stored in repository. |
|
policy
Package policy implements management of snapshot policies.
|
Package policy implements management of snapshot policies. |
|
restore
Package restore manages restoring filesystem snapshots.
|
Package restore manages restoring filesystem snapshots. |
|
snapshotfs
Package snapshotfs implements virtual filesystem on top of snapshots in repo.Repository.
|
Package snapshotfs implements virtual filesystem on top of snapshots in repo.Repository. |
|
snapshotgc
Package snapshotgc implements garbage collection of contents that are no longer referenced through snapshots.
|
Package snapshotgc implements garbage collection of contents that are no longer referenced through snapshots. |
|
snapshotmaintenance
Package snapshotmaintenance provides helpers to run snapshot GC and low-level repository snapshotmaintenance.
|
Package snapshotmaintenance provides helpers to run snapshot GC and low-level repository snapshotmaintenance. |
|
tests
|
|
|
clitestutil
Package clitestutil contains utilities for
|
Package clitestutil contains utilities for |
|
recovery/blobmanipulator
Package blobmanipulator provides the framework for snapshot fix testing.
|
Package blobmanipulator provides the framework for snapshot fix testing. |
|
repository_stress_test/repomodel
Package repomodel provides simplified model of repository operation.
|
Package repomodel provides simplified model of repository operation. |
|
robustness
Package robustness contains tests that validate data stability over time.
|
Package robustness contains tests that validate data stability over time. |
|
robustness/checker
Package checker defines the framework for creating and restoring snapshots with a data integrity check
|
Package checker defines the framework for creating and restoring snapshots with a data integrity check |
|
robustness/engine
Package engine provides the framework for a snapshot repository testing engine
|
Package engine provides the framework for a snapshot repository testing engine |
|
robustness/fiofilewriter
Package fiofilewriter provides a FileWriter based on FIO.
|
Package fiofilewriter provides a FileWriter based on FIO. |
|
robustness/multiclient_test/framework
Package framework contains tools to enable multiple clients to connect to a central repository server and run robustness tests concurrently.
|
Package framework contains tools to enable multiple clients to connect to a central repository server and run robustness tests concurrently. |
|
robustness/multiclient_test/storagestats
Package storagestats contains logging mechanism log disk space consumed by directories created by robustness test framework before and after the test run.
|
Package storagestats contains logging mechanism log disk space consumed by directories created by robustness test framework before and after the test run. |
|
robustness/pathlock
Package pathlock defines a PathLocker interface and an implementation that will synchronize based on filepath.
|
Package pathlock defines a PathLocker interface and an implementation that will synchronize based on filepath. |
|
robustness/snapmeta
Package snapmeta provides Kopia implementations of Persister and Snapshotter.
|
Package snapmeta provides Kopia implementations of Persister and Snapshotter. |
|
testdirtree
Package testdirtree provides utilities for creating test directory trees for testing.
|
Package testdirtree provides utilities for creating test directory trees for testing. |
|
testenv
Package testenv contains Environment for use in testing.
|
Package testenv contains Environment for use in testing. |
|
testingaction
Command testingaction implements a action that is used in various tests.
|
Command testingaction implements a action that is used in various tests. |
|
tools/fio
Package fio wraps calls to the fio tool.
|
Package fio wraps calls to the fio tool. |
|
tools/fswalker
Package fswalker provides the checker.Comparer interface using FSWalker walker and reporter.
|
Package fswalker provides the checker.Comparer interface using FSWalker walker and reporter. |
|
tools/fswalker/protofile
Package protofile contains helper functions common across multiple fswalker tool wrappers
|
Package protofile contains helper functions common across multiple fswalker tool wrappers |
|
tools/fswalker/reporter
Package reporter wraps calls to the fswalker Reporter
|
Package reporter wraps calls to the fswalker Reporter |
|
tools/fswalker/walker
Package walker wraps calls to the fswalker Walker
|
Package walker wraps calls to the fswalker Walker |
|
tools/kopiaclient
Package kopiaclient provides a client to interact with a Kopia repo.
|
Package kopiaclient provides a client to interact with a Kopia repo. |
|
tools/kopiarunner
Package kopiarunner wraps the execution of the kopia binary.
|
Package kopiarunner wraps the execution of the kopia binary. |
|
tools
|
|
|
cli2md
Command cli2md generates documentation pages from CLI flags.
|
Command cli2md generates documentation pages from CLI flags. |
|
gettool
Package gettool combines and replaces curl, tar and gunzip, sha256sum and a bunch of Makefile scripts to quickly download, verify and install OS-specific version of tools (typically from GitHub) in a platform-agnostic manner without external tooling.
|
Package gettool combines and replaces curl, tar and gunzip, sha256sum and a bunch of Makefile scripts to quickly download, verify and install OS-specific version of tools (typically from GitHub) in a platform-agnostic manner without external tooling. |
|
gettool/autodownload
Package autodownload combines and replaces curl, tar and gunzip and sha256sum and allows downloading, verifying and extracting the archive (zip, tar, tar.gz) to a local directory without using external tools.
|
Package autodownload combines and replaces curl, tar and gunzip and sha256sum and allows downloading, verifying and extracting the archive (zip, tar, tar.gz) to a local directory without using external tools. |
Click to show internal directories.
Click to hide internal directories.