topolvm

README

TopoLVM

TopoLVM is a CSI plugin using LVM for Kubernetes. It can be considered as a specific implementation of local persistent volumes using CSI and LVM.

• Conformed CSI version: 1.5.0

Our supported platform are:

• Kubernetes: 1.28, 1.27, 1.26
• Node OS: Linux with LVM2 (*1)
• CPU Architecture: x86 (*2), arm64 (*3), ppc64le (*3)
• Filesystems: ext4, xfs, btrfs(experimental)
• lvm version 2.02.163 or later (adds JSON output support)

*1 The host's Linux Kernel must be v4.9 or later which supports rmapbt and reflink, if you use xfs filesystem with an official docker image.
*2 Tier1 support. The official docker images are provided and all functionalities are tested by CI.
*3 Tier2 support. The official docker images are provided, but no tests run by CI.

Docker images are available on ghcr.io.

Getting Started

If you want to try TopoLVM on your local machine, see example directory which provides scripts to run TopoLVM on kind (Kubernetes IN Docker).

To use TopoLVM on your real kubernetes cluster, read documents.

Deprecated: If you want to use TopoLVM on Rancher/RKE, see docs/deprecated/rancher/README.md.

Contributing

TopoLVM project welcomes contributions from any member of our community. To get started contributing, please see our Contributing Guide.

Scope

In Scope

• Dynamic provisioning: Volumes are created dynamically when PersistentVolumeClaim objects are created.
• Raw block volume: Volumes are available as block devices inside containers.
• Topology: TopoLVM uses CSI topology feature to schedule Pod to Node where LVM volume exists.
• Extended scheduler: TopoLVM extends the general Pod scheduler to prioritize Nodes having larger storage capacity.
• Volume metrics: Usage stats are exported as Prometheus metrics from kubelet.
• Volume Expansion: Volumes can be expanded by editing PersistentVolumeClaim objects.
• Storage capacity tracking: You can enable Storage Capacity Tracking mode instead of using topolvm-scheduler.
• Snapshot: Snapshots can be taken when using thin provisioning.

Planned Features

• Prioritize based on volume capacity without using the extended scheduler.

Communications

If you have any questions or ideas, please use discussions.

Resources

docs directory contains the user manual, designs and specifications, and so on.

A diagram of components is available in docs/design.md.

TopoLVM maintainers presented the motivation and implementation of TopoLVM at KubeCon Europe 2020: https://kccnceu20.sched.com/event/ZerD

License

This project is licensed under Apache License 2.0.

Documentation

Index

• Constants
• Variables
• func GetCapacityKeyPrefix() string
• func GetCapacityResource() corev1.ResourceName
• func GetDeviceClassKey() string
• func GetLVPendingDeletionKey() string
• func GetLogicalVolumeFinalizer() string
• func GetLvcreateOptionClassKey() string
• func GetNodeFinalizer() string
• func GetPluginName() string
• func GetResizeRequestedAtKey() string
• func GetTopologyNodeKey() string
• func UseLegacy() bool

Constants

const CreatedbyLabelKey = "app.kubernetes.io/created-by"

Label key that indicates The controller/user who created this resource https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels

const CreatedbyLabelValue = "topolvm-controller"

Label value that indicates The controller/user who created this resource

const DefaultCSISocket = "/run/topolvm/csi-topolvm.sock"

DefaultCSISocket is the default path of the CSI socket file.

const DefaultDeviceClassAnnotationName = "00default"

DefaultDeviceClassAnnotationName is the part of annotation name for the default device-class.

const DefaultDeviceClassName = ""

DefaultDeviceClassName is the name for the default device-class.

const DefaultLVMdSocket = "/run/topolvm/lvmd.sock"

DefaultLVMdSocket is the default path of the lvmd socket file.

const DefaultSize = int64(DefaultSizeGb << 30)

DefaultSize is DefaultSizeGb in bytes

const DefaultSizeGb = 1

DefaultSizeGb is the default size in GiB for volumes (PVC or generic ephemeral volumes) w/o capacity requests.

const DeviceDirectory = "/dev/topolvm"

DeviceDirectory is a directory where TopoLVM Node service creates device files.

const LegacyPVCFinalizer = legacyPluginName + "/pvc"

LegacyPVCFinalizer is a legacy finalizer of PVC.

const MinimumSectorSize = int64(4096)

MinimumSectorSize is the minimum size in bytes for volumes (PVC or generic ephemeral volumes). It is derived from the usual sector size of 512,1024 or 4096 bytes for logical volumes. While Sector Sizes of 512 are common, using 4096 is safe As it also aligns with 512 and 1024 byte sectors, and is the default for most modern disks. Going lower than this size will cause validation issues on volume creation for the user.

const PVCFinalizer = pluginName + "/pvc"

PVCFinalizer is a finalizer of PVC.

Variables

var (
	// Version is the topolvm version, this value will be injected on building.
	Version = "devel"
)

Functions

func GetCapacityKeyPrefix

func GetCapacityKeyPrefix() string

GetCapacityKeyPrefix returns the key prefix of Node annotation that represents VG free space.

func GetCapacityResource

func GetCapacityResource() corev1.ResourceName

GetCapacityResource returns the resource name of topolvm capacity.

func GetDeviceClassKey

func GetDeviceClassKey() string

GetDeviceClassKey returns the key used in CSI volume create requests to specify a device-class.

func GetLVPendingDeletionKey

func GetLVPendingDeletionKey() string

GetPendingDeletionKey returns the name of the pending-deletion annotation

func GetLogicalVolumeFinalizer

func GetLogicalVolumeFinalizer() string

GetLogicalVolumeFinalizer returns the name of LogicalVolume finalizer

func GetLvcreateOptionClassKey

func GetLvcreateOptionClassKey() string

GetDeviceClassKey returns the key used in CSI volume create requests to specify a lvcreate-option-class.

func GetNodeFinalizer

func GetNodeFinalizer() string

GetNodeFinalizer returns the name of Node finalizer of TopoLVM

func GetPluginName

func GetPluginName() string

GetPluginName returns the name of the CSI plugin.

func GetResizeRequestedAtKey

func GetResizeRequestedAtKey() string

GetResizeRequestedAtKey returns the key of LogicalVolume that represents the timestamp of the resize request.

func GetTopologyNodeKey

func GetTopologyNodeKey() string

TopologyNodeKey returns the key of topology that represents node name.

func UseLegacy

func UseLegacy() bool