README ¶
devicemapper - a storage backend based on Device Mapper
Theory of operation
The device mapper graphdriver uses the device mapper thin provisioning
module (dm-thinp) to implement CoW snapshots. For each devicemapper
graph location (typically /var/lib/docker/devicemapper
, $graph below)
a thin pool is created based on two block devices, one for data and
one for metadata. By default these block devices are created
automatically by using loopback mounts of automatically created sparse
files.
The default loopback files used are $graph/devicemapper/data
and
$graph/devicemapper/metadata
. Additional metadata required to map
from docker entities to the corresponding devicemapper volumes is
stored in the $graph/devicemapper/json
file (encoded as Json).
In order to support multiple devicemapper graphs on a system, the thin
pool will be named something like: docker-0:33-19478248-pool
, where
the 0:33
part is the minor/major device nr and 19478248
is the
inode number of the $graph directory.
On the thin pool, docker automatically creates a base thin device,
called something like docker-0:33-19478248-base
of a fixed
size. This is automatically formatted with an empty filesystem on
creation. This device is the base of all docker images and
containers. All base images are snapshots of this device and those
images are then in turn used as snapshots for other images and
eventually containers.
options
The devicemapper backend supports some options that you can specify
when starting the docker daemon using the --storage-opt
flags.
This uses the dm
prefix and would be used something like docker -d --storage-opt dm.foo=bar
.
Here is the list of supported options:
-
dm.basesize
Specifies the size to use when creating the base device, which limits the size of images and containers. The default value is 10G. Note, thin devices are inherently "sparse", so a 10G device which is mostly empty doesn't use 10 GB of space on the pool. However, the filesystem will use more space for the empty case the larger the device is. Warning: This value affects the system-wide "base" empty filesystem that may already be initialized and inherited by pulled images. Typically, a change to this value will require additional steps to take effect: 1) stop
docker -d
, 2)rm -rf /var/lib/docker
, 3) startdocker -d
.Example use:
docker -d --storage-opt dm.basesize=20G
-
dm.loopdatasize
Specifies the size to use when creating the loopback file for the "data" device which is used for the thin pool. The default size is 100G. Note that the file is sparse, so it will not initially take up this much space.
Example use:
docker -d --storage-opt dm.loopdatasize=200G
-
dm.loopmetadatasize
Specifies the size to use when creating the loopback file for the "metadadata" device which is used for the thin pool. The default size is 2G. Note that the file is sparse, so it will not initially take up this much space.
Example use:
docker -d --storage-opt dm.loopmetadatasize=4G
-
dm.fs
Specifies the filesystem type to use for the base device. The supported options are "ext4" and "xfs". The default is "ext4"
Example use:
docker -d --storage-opt dm.fs=xfs
-
dm.mkfsarg
Specifies extra mkfs arguments to be used when creating the base device.
Example use:
docker -d --storage-opt "dm.mkfsarg=-O ^has_journal"
-
dm.mountopt
Specifies extra mount options used when mounting the thin devices.
Example use:
docker -d --storage-opt dm.mountopt=nodiscard
-
dm.datadev
Specifies a custom blockdevice to use for data for the thin pool.
If using a block device for device mapper storage, ideally both datadev and metadatadev should be specified to completely avoid using the loopback device.
Example use:
docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1
-
dm.metadatadev
Specifies a custom blockdevice to use for metadata for the thin pool.
For best performance the metadata should be on a different spindle than the data, or even better on an SSD.
If setting up a new metadata pool it is required to be valid. This can be achieved by zeroing the first 4k to indicate empty metadata, like this:
``dd if=/dev/zero of=$metadata_dev bs=4096 count=1```
Example use:
docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1
-
dm.blocksize
Specifies a custom blocksize to use for the thin pool. The default blocksize is 64K.
Example use:
docker -d --storage-opt dm.blocksize=512K
-
dm.blkdiscard
Enables or disables the use of blkdiscard when removing devicemapper devices. This is enabled by default (only) if using loopback devices and is required to res-parsify the loopback file on image/container removal.
Disabling this on loopback can lead to much faster container removal times, but will make the space used in /var/lib/docker directory not be returned to the system for other use when containers are removed.
Example use:
docker -d --storage-opt dm.blkdiscard=false
Documentation ¶
Index ¶
- Constants
- Variables
- func BlockDeviceDiscard(path string) error
- func DevmapperLogCallback(level C.int, file *C.char, line C.int, dm_errno_or_class C.int, ...)
- func FindLoopDeviceFor(file *os.File) *os.File
- func GetBlockDeviceSize(file *os.File) (uint64, error)
- func GetLibraryVersion() (string, error)
- func Init(home string, options []string) (graphdriver.Driver, error)
- func LogInitVerbose(level int)
- func LoopbackSetCapacity(file *os.File) error
- func Mounted(mountpoint string) (bool, error)
- func ProbeFsType(device string) (string, error)
- func RemoveDevice(name string) error
- func SetDevDir(dir string) error
- func UdevWait(cookie uint) error
- type AddNodeType
- type CDmTask
- type CLoopInfo64
- type DevInfo
- type DevStatus
- type DeviceSet
- func (devices *DeviceSet) AddDevice(hash, baseHash string) error
- func (devices *DeviceSet) DeleteDevice(hash string) error
- func (devices *DeviceSet) GetDeviceStatus(hash string) (*DevStatus, error)
- func (devices *DeviceSet) HasActivatedDevice(hash string) bool
- func (devices *DeviceSet) HasDevice(hash string) bool
- func (devices *DeviceSet) List() []string
- func (devices *DeviceSet) MountDevice(hash, path, mountLabel string) error
- func (devices *DeviceSet) ResizePool(size int64) error
- func (devices *DeviceSet) Shutdown() error
- func (devices *DeviceSet) Status() *Status
- func (devices *DeviceSet) UnmountDevice(hash string) error
- type DevmapperLogger
- type DiskUsage
- type Driver
- func (d *Driver) Cleanup() error
- func (d *Driver) Create(id, parent string) error
- func (d *Driver) Exists(id string) bool
- func (d *Driver) Get(id, mountLabel string) (string, error)
- func (d *Driver) Put(id string)
- func (d *Driver) Remove(id string) error
- func (d *Driver) Status() [][2]string
- func (d *Driver) String() string
- type Info
- type LoopInfo64
- type MetaData
- type Status
- type Task
- func (t *Task) AddTarget(start, size uint64, ttype, params string) error
- func (t *Task) GetDriverVersion() (string, error)
- func (t *Task) GetInfo() (*Info, error)
- func (t *Task) GetNextTarget(next uintptr) (nextPtr uintptr, start uint64, length uint64, targetType string, params string)
- func (t *Task) Run() error
- func (t *Task) SetAddNode(addNode AddNodeType) error
- func (t *Task) SetCookie(cookie *uint, flags uint16) error
- func (t *Task) SetMessage(message string) error
- func (t *Task) SetName(name string) error
- func (t *Task) SetRo() error
- func (t *Task) SetSector(sector uint64) error
- type TaskType
Constants ¶
const ( BlkGetSize64 = C.BLKGETSIZE64 BlkDiscard = C.BLKDISCARD LoopSetFd = C.LOOP_SET_FD LoopCtlGetFree = C.LOOP_CTL_GET_FREE LoopGetStatus64 = C.LOOP_GET_STATUS64 LoopSetStatus64 = C.LOOP_SET_STATUS64 LoopClrFd = C.LOOP_CLR_FD LoopSetCapacity = C.LOOP_SET_CAPACITY )
IOCTL consts
const ( LoFlagsAutoClear = C.LO_FLAGS_AUTOCLEAR LoFlagsReadOnly = C.LO_FLAGS_READ_ONLY LoFlagsPartScan = C.LO_FLAGS_PARTSCAN LoKeySize = C.LO_KEY_SIZE LoNameSize = C.LO_NAME_SIZE )
Variables ¶
var ( DefaultDataLoopbackSize int64 = 100 * 1024 * 1024 * 1024 DefaultMetaDataLoopbackSize int64 = 2 * 1024 * 1024 * 1024 DefaultBaseFsSize uint64 = 10 * 1024 * 1024 * 1024 DefaultThinpBlockSize uint32 = 128 // 64K = 128 512b sectors )
var ( ErrTaskRun = errors.New("dm_task_run failed") ErrTaskSetName = errors.New("dm_task_set_name failed") ErrTaskSetMessage = errors.New("dm_task_set_message failed") ErrTaskSetAddNode = errors.New("dm_task_set_add_node failed") ErrTaskSetRo = errors.New("dm_task_set_ro failed") ErrTaskAddTarget = errors.New("dm_task_add_target failed") ErrTaskSetSector = errors.New("dm_task_set_sector failed") ErrTaskGetInfo = errors.New("dm_task_get_info failed") ErrTaskGetDriverVersion = errors.New("dm_task_get_driver_version failed") ErrTaskSetCookie = errors.New("dm_task_set_cookie failed") ErrNilCookie = errors.New("cookie ptr can't be nil") ErrAttachLoopbackDevice = errors.New("loopback mounting failed") ErrGetBlockSize = errors.New("Can't get block size") ErrUdevWait = errors.New("wait on udev cookie failed") ErrSetDevDir = errors.New("dm_set_dev_dir failed") ErrGetLibraryVersion = errors.New("dm_get_library_version failed") ErrCreateRemoveTask = errors.New("Can't create task of type DeviceRemove") ErrRunRemoveDevice = errors.New("running removeDevice failed") ErrInvalidAddNode = errors.New("Invalide AddNoce type") ErrGetLoopbackBackingFile = errors.New("Unable to get loopback backing file") ErrLoopbackSetCapacity = errors.New("Unable set loopback capacity") ErrBusy = errors.New("Device is Busy") )
var ( DmGetLibraryVersion = dmGetLibraryVersionFct DmGetNextTarget = dmGetNextTargetFct DmLogInitVerbose = dmLogInitVerboseFct DmSetDevDir = dmSetDevDirFct DmTaskAddTarget = dmTaskAddTargetFct DmTaskCreate = dmTaskCreateFct DmTaskDestroy = dmTaskDestroyFct DmTaskGetInfo = dmTaskGetInfoFct DmTaskGetDriverVersion = dmTaskGetDriverVersionFct DmTaskRun = dmTaskRunFct DmTaskSetAddNode = dmTaskSetAddNodeFct DmTaskSetCookie = dmTaskSetCookieFct DmTaskSetMessage = dmTaskSetMessageFct DmTaskSetName = dmTaskSetNameFct DmTaskSetRo = dmTaskSetRoFct DmTaskSetSector = dmTaskSetSectorFct DmUdevWait = dmUdevWaitFct LogWithErrnoInit = logWithErrnoInitFct )
Functions ¶
func BlockDeviceDiscard ¶
func DevmapperLogCallback ¶
func GetLibraryVersion ¶
func LogInitVerbose ¶
func LogInitVerbose(level int)
func LoopbackSetCapacity ¶
func ProbeFsType ¶ added in v0.12.0
Types ¶
type AddNodeType ¶
type AddNodeType int
const ( AddNodeOnResume AddNodeType = iota AddNodeOnCreate )
type CDmTask ¶
type CDmTask C.struct_dm_task
type CLoopInfo64 ¶
type CLoopInfo64 C.struct_loop_info64
type DevInfo ¶
type DeviceSet ¶
type DeviceSet struct { MetaData sync.Mutex // Protects Devices map and serializes calls into libdevmapper TransactionId uint64 NewTransactionId uint64 // contains filtered or unexported fields }
func NewDeviceSet ¶
func (*DeviceSet) DeleteDevice ¶
func (*DeviceSet) GetDeviceStatus ¶
func (*DeviceSet) HasActivatedDevice ¶
func (*DeviceSet) MountDevice ¶
func (*DeviceSet) ResizePool ¶
func (*DeviceSet) UnmountDevice ¶
type DevmapperLogger ¶
type DevmapperLogger interface {
// contains filtered or unexported methods
}
type LoopInfo64 ¶
type LoopInfo64 struct {
// contains filtered or unexported fields
}
type Task ¶
type Task struct {
// contains filtered or unexported fields
}
func TaskCreate ¶
func (*Task) GetDriverVersion ¶ added in v0.12.0
func (*Task) GetNextTarget ¶
func (*Task) SetAddNode ¶
func (t *Task) SetAddNode(addNode AddNodeType) error