nbd

README

Go implementation of the Linux Network Block Device protocol

This repository contains Go implementations of

• The NBD network protocol (both client and server).
• The Netlink API for NBD, to configure the in-kernel NBD client.
• A basic CLI tool that uses these implementations.

It's currently in beta and thus there are a bunch of known (and unknown) issues. Check the "beta" label for known problems. It is particularly interesting if you require features that are not yet supported. Please comment/vote on the corresponding issue.

Installation

Currently, this is pre-release, so the only way to install is from source. To do that, use

go get -u github.com/Merovius/nbd

Using the library

There are two packages:

• nbd, containing the client and server implementations of the network protocol, as well as some convenience functions for nbdnl. The network protocol is used as a handshake between client and server, to negotiate optional features and other options. Under Linux, there are also a couple of functions provided to easily hook up a Device implementation and use it as a block device.
• nbdnl, containing an implementation of the NBD generic netlink family, based on Matt Layher's genetlink package. This package can only be used on Linux; you should guard any usage with corresponding build tags.

The main usecase of this library is fuzzing code that tries to provide durable filesystem-operations. It allows you to implement aribtrary failure modes of a block device and then create any filesystem you'd like to test on it. For example, to fuzz for crash-resistence, you can have the block device return errors on any write-operations after an arbitrary point in time and then repeatedly mount a filesystem, run a bunch of application code, simulate a crash and then check invariants (after re-mounting). This can provide some confidence (though no guarantees) that your code works with actual filesystem-implementations.

Note, that any code that wants to configure the in-kernel NBD client has to be privileged (the process needs to have CAP_SYS_ADMIN).

NBD tool

This repo contains basic CLI tool to configure/serve/connect to NBD devices. You can install it via

go get -u github.com/Merovius/nbd/cmd/nbd

To see what it can do, use nbd help. Note, that most of the useful commands require root (or, more specifically, CAP_SYS_ADMIN) to work.

One of the most useful subcommands is lo, which can be used to use a file as a block device (similarly to losetup). It also supports toggling write-only mode of the device via a unix signal, though, which can be used to test the durability of software not written in Go. Refer to nbd help lo for details.

License

Copyright 2018 Axel Wagner

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Documentation

Overview

Package nbd implements the NBD network protocol.

You can find a full description of the protocol at https://sourceforge.net/p/nbd/code/ci/master/tree/doc/proto.md

This package implements both the client and the server side of the protocol, as well as (on Linux) utilities to hook up the kernel NBD client to use a server as a block device. The protocol is split into two phases: The handshake phase, which allows the client and server to negotiate their respective capabilities and what export to use. And the transmission phase, for actually reading/writing to the block device.

The client side of the handshake is done with the Client type. Its methods can be used to list the exports a server provides and their respective capabilities. Its Go method enters transmission phase. The returned Export can then be passed to Configure (linux only) to hook it up to an NBD device (/dev/nbdX).

The server side combines both handshake and transmission phase into the Serve or ListenAndServe functions. The user is expected to implement the Device interface to serve actual reads/writes. Under linux, the Loopback function serves as a convenient way to use a given Device as a block device.

Index

• func Configure(e Export, socks ...*os.File) (uint32, error)
• func ListenAndServe(ctx context.Context, network, addr string, exp ...Export) error
• func Loopback(ctx context.Context, d Device, size uint64) (idx uint32, wait func() error, err error)
• func Serve(ctx context.Context, c net.Conn, exp ...Export) error
• type BlockSizeConstraints
• type Client
  • func ClientHandshake(ctx context.Context, c net.Conn) (*Client, error)
  • func (c *Client) Abort() error
  • func (c *Client) Go(exportName string) (Export, error)
  • func (c *Client) Info(exportName string) (Export, error)
  • func (c *Client) List() ([]string, error)
• type Device
• type Errno
  • func (e Errno) Errno() Errno
  • func (e Errno) Error() string
• type Error
  • func Errorf(code Errno, msg string, v ...interface{}) Error
• type Export
• Bugs

Functions

func Configure

func Configure(e Export, socks ...*os.File) (uint32, error)

Configure passes the given set of sockets to the kernel to provide them as an NBD device. socks must be connected to the same server (which must support multiple connections) and be in transmission phase. It returns the device-numbers that was chosen by the kernel or any error. You can then use /dev/nbdX as a block device. Use nbdnl.Disconnect to disconnect the device once you're done with it.

This is a Linux-only API.

func ListenAndServe

func ListenAndServe(ctx context.Context, network, addr string, exp ...Export) error

ListenAndServe starts listening on the given network/address and serves the given exports, the first of which will serve as the default. It starts a new goroutine for each connection. ListenAndServe only returns when ctx is cancelled or an unrecoverable error occurs. Either way, it will wait for all connections to terminate first.

func Loopback

func Loopback(ctx context.Context, d Device, size uint64) (idx uint32, wait func() error, err error)

Loopback serves d on a private socket, passing the other end to the kernel to connect to an NBD device. It returns the device-number that the kernel chose. wait should be called to check for errors from serving the device. It blocks until ctx is cancelled or an error occurs (so it behaves like Serve). When ctx is cancelled, the device will be disconnected, and any error encountered while disconnecting will be returned by wait.

This is a Linux-only API.

func Serve

func Serve(ctx context.Context, c net.Conn, exp ...Export) error

Serve serves the given exports on c. The first export is used as a default. Serve returns after ctx is cancelled or an error occurs.

Types

type BlockSizeConstraints

type BlockSizeConstraints struct {
	Min       uint32
	Preferred uint32
	Max       uint32
}

BlockSizeConstraints optionally specifies possible block sizes for a given export.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client performs the client-side of the NBD network protocol handshake and can be used to query information about the exports from a server.

func ClientHandshake

func ClientHandshake(ctx context.Context, c net.Conn) (*Client, error)

ClientHandshake starts the client-side of the NBD handshake over c.

func (*Client) Abort

func (c *Client) Abort() error

Abort aborts the handshake. c should not be used after Abort returns.

func (*Client) Go

func (c *Client) Go(exportName string) (Export, error)

Go terminates the handshake phase of the NBD protocol, opening the export identified by exportName. If exportName is the empty string, the default export will be used. c should not be used after Go returns.

func (*Client) Info

func (c *Client) Info(exportName string) (Export, error)

Info requests information about the export identified by exportName. If exportName is the empty string, the default export will be queried.

func (*Client) List

func (c *Client) List() ([]string, error)

List returns the names of exports the server is providing.

type Device

type Device interface {
	io.ReaderAt
	io.WriterAt
	// Sync should block until all previous writes where written to persistent
	// storage and return any errors that occured.
	Sync() error
}

Device is the interface that should be implemented to expose an NBD device to the network or the kernel. Errors returned should implement Error - otherwise, EIO is assumed as the error number.

type Errno

type Errno uint32

Errno is an error code suitable to be sent over the wire. It mostly corresponds to syscall.Errno, though the constants in this package are specified and so are the only ones to be safe to be sent over the wire and understood by all NBD servers/clients.

const (
	EPERM     Errno = 1
	EIO       Errno = 5
	ENOMEM    Errno = 12
	EINVAL    Errno = 22
	ENOSPC    Errno = 28
	EOVERFLOW Errno = 75
	ESHUTDOWN Errno = 108
)

See https://manpages.debian.org/stretch/manpages-dev/errno.3.en.html for a description of error numbers.

func (Errno) Errno

func (e Errno) Errno() Errno

Errno returns e.

func (Errno) Error

func (e Errno) Error() string

type Error

type Error interface {
	Error() string
	Errno() Errno
}

Error combines the normal error interface with an Errno method, that returns an NBD error number. All of Device's methods should return an Error - otherwise, EIO is assumed as the error number.

func Errorf

func Errorf(code Errno, msg string, v ...interface{}) Error

Errorf returns an error implementing Error, returning code from Errno.

type Export

type Export struct {
	Name        string
	Description string
	Size        uint64
	Flags       uint16 // TODO: Determine Flags from Device.
	BlockSizes  *BlockSizeConstraints
	Device      Device
}

Export specifies the data needed for the NBD network protocol.

Notes

Bugs

• BlockSizeConstraints are not yet enforced by the server.

• The server does not yet support FUA for direct IO.

• StartTLS is not supported yet.

• There is no way to declare a preferred block size for Loopback yet.

• Server flags are not yet set (or used) correctly.

• Structured replies are not yet supported.

• CMD_TRIM is not yet supported.

• Lame-duck mode (ESHUTDOWN) is not yet implemented.

• CMD_WRITE_ZEROES is not yet supported.

• Metadata querying is not yet supported.

• FLAG_ROTATIONAL is not yet supported.

• CMD_CACHE is not yet supported.