bdev

package
v0.0.0-...-abc2165 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 10, 2025 License: NIST-PD-fallback Imports: 20 Imported by: 0

README

ndn-dpdk/dpdk/bdev

This package contains bindings of SPDK block device (bdev) layer. It is used by Disk-based Data Store, which is intended for extending the forwarder's Content Store with additional capacity.

Block Device Drivers

SPDK offers a range of block device drivers. This package supports a subset of these drivers.

Malloc type represents a memory-backed emulated block device. The storage space is reserved from hugepage memory, not on an actual disk drive. It is mainly for unit testing.

File type represents a file-backed virtual block device. It may use either io_uring or Linux AIO driver as backend. The storage space is a file located in the local filesystem. For best performance, you should keep the file on a local disk, not a network storage.

Nvme type represents a hardware NVMe drive. It contains one or more NvmeNamespaces, each is a block device. Although SPDK NVMe driver supports both local PCIe drives and NVMe over Fabrics, this implementation is limited to local drives only. SPDK would have exclusive control over the PCI device, which means you cannot use the NVMe driver on the same NVMe device where you have kernel filesystems. In most cases, the PCI device should be bound to vfio-pci or uio_pci_generic kernel driver. You can setup PCI binding with SPDK setup.sh script, example command: sudo NRHUGE=0 PCI_ALLOWED="0000:00:09.0" ./scripts/setup.sh. Importantly, once you assign an NVMe device to be used with NDN-DPDK via the SPDK NVMe driver, all existing data will be erased.

Delay type represents a virtual block device that wraps an existing block device, adding a delay to each I/O operation. ErrorInjection type represents a virtual block device that wraps an existing block device, injecting an error to certain I/O operations. They are mainly for unit testing.

All types described above implement the Device interface, which allows retrieving information about the block device.

Open Device and I/O Operations

Bdev type represents an Open()-ed block device. This package only supports block devices with 512-octet block size, which is used by the majority of known devices.

I/O operations may be submitted to an open block device. This package implements two I/O operations:

  • Bdev_WritePacket writes a packet mbuf to a block offset.
  • Bdev_ReadPacket reads the packet at a block offset to an mbuf.

All I/O operations are asynchronous. If multiple I/O operations are accessing the same block offset, it's caller's responsible to ensure proper sequencing.

Memory Alignment

Certain SPDK drivers have specific memory alignment requirements. Bdev_WritePacket and Bdev_ReadPacket can handle these requirements.

Some NVMe devices require every buffers to have dword (32-bit) alignment, and every buffer length to be a multiple of 4. To satisfy this requirement without copying, the content being written to the block device may contain part of the headroom and tailroom in the mbuf. The amount of excess writing is captured in a BdevStoredPacket struct, which must be saved by the caller. During reading, the original packet is recovered from the saved information, by either adjusting headroom or memmove-ing. This design chooses to incur less overhead during writing and more overhead during reading, because the primary use case is packet caching, in which only a subset of written packets would generate cache hits and need to be read back, and each packet reenters in-memory cache after being read.

Both file-backed drivers expect the buffer to be aligned at 512-octet boundary, otherwise the driver will incur copying overhead. During writing, such copying is mostly unavoidable, because the excess writing method described above would greatly amplify storage usage. During reading, the mbuf must have sufficient dataroom to cover the packet itself plus twice the block size, so that the read function can find a properly aligned memory address within the dataroom to read the packet.

Documentation

Overview

Package bdev contains bindings of SPDK block device layer.

Index

Constants

View Source 
const RequiredBlockSize = C.BdevBlockSize

RequiredBlockSize is the expected block size.

Variables

This section is empty.

Functions

func TruncateFile 

func TruncateFile(filename string, size int64) error

TruncateFile creates and truncates a file.

Types

type Bdev 

type Bdev struct {
	Device
	// contains filtered or unexported fields
}

Bdev represents an open block device descriptor.

func Open 

func Open(device Device, mode Mode) (bd *Bdev, e error)

Open opens a block device.

func (*Bdev) Close 

func (bd *Bdev) Close() error

Close closes the block device.

func (*Bdev) CopyToC 

func (bd *Bdev) CopyToC(ptr unsafe.Pointer)

CopyToC copies to *C.Bdev.

func (*Bdev) ReadPacket 

func (bd *Bdev) ReadPacket(blockOffset int64, pkt *pktmbuf.Packet, sp StoredPacket) error

ReadPacket reads blocks via scatter gather list.

func (*Bdev) UnmapBlocks 

func (bd *Bdev) UnmapBlocks(blockOffset, blockCount int64) error

UnmapBlocks notifies the device that the data in the blocks are no longer needed.

func (*Bdev) WritePacket 

func (bd *Bdev) WritePacket(blockOffset int64, pkt *pktmbuf.Packet) (sp StoredPacket, e error)

WritePacket writes blocks via scatter gather list.

type Delay 

type Delay struct {
	*Info
}

Delay represents a delay block device.

func NewDelay 

func NewDelay(inner Device, cfg DelayConfig) (device *Delay, e error)

NewDelay creates a delay block device.

func (*Delay) Close 

func (device *Delay) Close() error

Close destroys this block device. The inner device is not closed.

type DelayConfig 

type DelayConfig struct {
	AvgReadLatency  time.Duration
	P99ReadLatency  time.Duration
	AvgWriteLatency time.Duration
	P99WriteLatency time.Duration
}

DelayConfig configures Delay bdev.

type Device 

type Device interface {
	DevInfo() *Info
}

Device interface represents a device.

func OverrideWriteMode 

func OverrideWriteMode(device Device, wm WriteMode) Device

OverrideWriteMode forces a specific WriteMode for unit testing.

type DeviceCloser 

type DeviceCloser interface {
	Device
	io.Closer
}

DeviceCloser interface is a device that can be closed.

type ErrorInjection 

type ErrorInjection struct {
	*Info
}

ErrorInjection represents an error injection block device.

func NewErrorInjection 

func NewErrorInjection(inner Device) (device *ErrorInjection, e error)

NewErrorInjection creates an error injection block device.

func (*ErrorInjection) Close 

func (device *ErrorInjection) Close() error

Close destroys this block device. The inner device is not closed.

func (*ErrorInjection) Inject 

func (device *ErrorInjection) Inject(ioType IOType, count int) error

Inject injects some errors.

type File 

type File struct {
	*Info
	// contains filtered or unexported fields
}

File represents a file-backed block device. This may use either AIO or Uring driver.

func NewFile 

func NewFile(filename string) (*File, error)

NewFile opens a file-backed block device.

func NewFileWithDriver 

func NewFileWithDriver(driver FileDriver, filename string) (device *File, e error)

NewFileWithDriver opens a file-backed block device with specified driver.

func (*File) Close 

func (device *File) Close() error

Close destroys this block device. The underlying file is not deleted.

func (*File) Filename 

func (device *File) Filename() string

Filename returns the filename.

type FileDriver 

type FileDriver string

FileDriver indicates a file-backed block device driver. 

const (
	FileAio   FileDriver = "aio"
	FileUring FileDriver = "uring"
)

FileDriver values.

type IOType 

type IOType int

IOType represents an I/O type. 

const (
	IORead      IOType = C.SPDK_BDEV_IO_TYPE_READ
	IOWrite     IOType = C.SPDK_BDEV_IO_TYPE_WRITE
	IOUnmap     IOType = C.SPDK_BDEV_IO_TYPE_UNMAP
	IONvmeAdmin IOType = C.SPDK_BDEV_IO_TYPE_NVME_ADMIN
	IONvmeIO    IOType = C.SPDK_BDEV_IO_TYPE_NVME_IO
)

type Info 

type Info C.struct_spdk_bdev

Info provides information about a block device.

func Find 

func Find(name string) *Info

Find finds a block device by name.

func List 

func List() (list []*Info)

List returns a list of existing block devices.

func (*Info) BlockSize 

func (bdi *Info) BlockSize() int

BlockSize returns logical block size in octets.

func (*Info) BufAlign 

func (bdi *Info) BufAlign() int

BufAlign returns minimum I/O buffer address alignment in octets.

func (*Info) CountBlocks 

func (bdi *Info) CountBlocks() int64

CountBlocks returns size of block device in logical blocks.

func (*Info) DevInfo 

func (bdi *Info) DevInfo() *Info

DevInfo implements Device interface.

func (*Info) DriverInfo 

func (bdi *Info) DriverInfo() (value any)

DriverInfo returns driver-specific information.

func (*Info) HasIOType 

func (bdi *Info) HasIOType(ioType IOType) bool

HasIOType determines whether the I/O type is supported.

func (*Info) HasWriteCache 

func (bdi *Info) HasWriteCache() bool

HasWriteCache returns whether write cache is enabled.

func (*Info) MarshalLogObject 

func (bdi *Info) MarshalLogObject(enc zapcore.ObjectEncoder) error

MarshalLogObject implements zapcore.ObjectMarshaler interface.

func (*Info) Name 

func (bdi *Info) Name() string

Name returns device name.

func (*Info) OptimalIOBoundary 

func (bdi *Info) OptimalIOBoundary() (boundary int, mandatory bool)

OptimalIOBoundary returns optimal I/O boundary in logical blocks and whether it's mandatory.

func (*Info) ProductName 

func (bdi *Info) ProductName() string

ProductName returns product name.

func (*Info) WriteUnitSize 

func (bdi *Info) WriteUnitSize() int

WriteUnitSize returns write unit size in logical blocks.

type Locator 

type Locator struct {
	// Malloc=true creates a simulated block device in hugepages memory.
	Malloc bool `json:"malloc,omitempty"`

	// File, if not empty, specifies a filename and creates a block device backed by this file.
	// The file is automatically created and truncated to the required size.
	File string `json:"file,omitempty"`
	// FileDriver customizes the SPDK driver for the file-backed block device.
	FileDriver *FileDriver `json:"fileDriver,omitempty"`

	// PCIAddr, if not nil, attaches an NVMe device.
	// The first NVMe namespace that has the expected block size and block count is used.
	PCIAddr *pciaddr.PCIAddress `json:"pciAddr,omitempty"`
}

Locator describes where to create or attach a block device.

func (Locator) Create 

func (loc Locator) Create(minBlocks int64) (Device, io.Closer, error)

Create creates a block device. It ensures block size is as expected, and a minimum number of blocks are present.

type Malloc 

type Malloc struct {
	*Info
}

Malloc represents a memory-backed block device.

func NewMalloc 

func NewMalloc(nBlocks int64) (device *Malloc, e error)

NewMalloc creates a memory-backed block device.

func (*Malloc) Close 

func (device *Malloc) Close() error

Close destroys this block device.

type Mode 

type Mode bool

Mode indicates mode of opening a block device. 

const (
	ReadOnly  Mode = false
	ReadWrite Mode = true
)

Mode values.

type Nvme 

type Nvme struct {
	// Namespaces is a list of NVMe namespaces as block devices.
	Namespaces []*NvmeNamespace
	// contains filtered or unexported fields
}

Nvme represents an NVMe controller.

func AttachNvme 

func AttachNvme(pciAddr pciaddr.PCIAddress) (nvme *Nvme, e error)

AttachNvme attaches block devices on an NVMe controller.

func (*Nvme) Close 

func (nvme *Nvme) Close() error

Close detaches the NVMe controller.

func (*Nvme) ControllerName 

func (nvme *Nvme) ControllerName() string

ControllerName returns NVMe controller name.

func (*Nvme) SglSupport 

func (nvme *Nvme) SglSupport() (supported, dwordAlign bool)

SglSupport reports whether NVMe controller supports scatter-gather lists and whether it requires dword alignment.

type NvmeNamespace 

type NvmeNamespace struct {
	// contains filtered or unexported fields
}

NvmeNamespace represents an NVMe namespace.

func (*NvmeNamespace) Controller 

func (nn *NvmeNamespace) Controller() *Nvme

Controller returns NVMe controller.

func (*NvmeNamespace) DevInfo 

func (nn *NvmeNamespace) DevInfo() *Info

DevInfo implements Device interface.

type StoredPacket 

type StoredPacket C.BdevStoredPacket

StoredPacket describes length and alignment of a stored packet.

func StoredPacketFromPtr 

func StoredPacketFromPtr(ptr unsafe.Pointer) *StoredPacket

StoredPacketFromPtr converts *C.BdevStoredPacket pointer to StoredPacket.

func (*StoredPacket) Ptr 

func (sp *StoredPacket) Ptr() unsafe.Pointer

Ptr returns *C.BdevStoredPacket pointer.

type WriteMode 

type WriteMode int

WriteMode indicates memory alignment requirements of writev request. 

const (
	WriteModeSimple WriteMode = iota
	WriteModeDwordAlign
	WriteModeContiguous
)

WriteMode values.

func (WriteMode) String 

func (wm WriteMode) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.