pvci

README

PVCi: Persistent Volume Claim (S3 Object) Injector

PVCI runs as a web service in a Kubernetes cluster and exposes an API for creating PersistentVolumeClaims populated with objects from an S3 compatible (Minio or AWS) storage system.

API

POST body for /size:

{
    "s3_ssl": false,
    "s3_endpoint": "obj-service.data:9000",
    "s3_bucket": "datasets",
    "s3_prefix": "testset",
    "s3_key": "{{DEV_OBJ_KEY}}",
    "s3_secret": "{{DEV_OBJ_SECRET}}"
}

POST body for /create:

{
    "s3_ssl": false,
    "s3_endpoint": "obj-service.data:9000",
    "s3_bucket": "datasets",
    "s3_prefix": "testset",
    "s3_key": "{{DEV_OBJ_KEY}}",
    "s3_secret": "{{DEV_OBJ_SECRET}}",
    "namespace": "default",
    "storage_class": "rook-ceph-block",
    "name": "test-dataset-1"
}

POST body for /status:

{
    "namespace": "default",
    "name": "test-dataset-1"
}

Kubernetes Deployment

RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  name: pvci
  namespace: namespace_a
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: pvci
  namespace: namespace_b
rules:
  - apiGroups:
      - ""
      - batch
    resources:
      - jobs
      - pods
      - persistentvolumeclaims
    verbs:
      - create
      - delete
      - deletecollection
      - get
      - list
      - patch
      - update
      - watch
---
# create a binding in namespace_a
# between the pvci service account in namespace_a
# and the pvci role in namespace_b
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pvci-lab
  namespace: namespace_b
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: pvci
subjects:
  - kind: ServiceAccount
    name: pvci
    namespace: namespace_a

Service

apiVersion: v1
kind: Service
metadata:
  name: pvci
  namespace: namespace_a
  labels:
    app: pvci
    component: api
spec:
  selector:
    app: pvci
  ports:
    - name: http-int
      protocol: "TCP"
      port: 8070
      targetPort: http-api
  type: ClusterIP

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: pvci
  namespace: namespace_a
  labels:
    app: pvci
    component: api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pvci
  template:
    metadata:
      labels:
        app: pvci
        component: api
    spec:
      serviceAccountName: pvci
      containers:
        - name: pvci
          image: txn2/pvci:0.4.0
          imagePullPolicy: IfNotPresent
          env:
            - name: IP
              value: "0.0.0.0"
            - name: PORT
              value: "8070"
            - name: MODE
              value: "release" # "release" for prod
          ports:
            - name: http-api
              containerPort: 8070
            - name: http-mtx
              containerPort: 2112

Development

Release

goreleaser --skip-publish --rm-dist --skip-validate

GITHUB_TOKEN=$GITHUB_TOKEN goreleaser --rm-dist

Documentation

Index

• Constants
• type API
  • func NewApi(cfg *Config) (*API, error)
  • func (a *API) CreatePVC(pvcRequestConfig PVCRequestConfig) error
  • func (a *API) CreatePVCAsyncHandler() gin.HandlerFunc
  • func (a *API) CreatePVCHandler() gin.HandlerFunc
  • func (a *API) Delete(pvcRequestConfig PVCRequestConfig) error
  • func (a *API) DeleteHandler() gin.HandlerFunc
  • func (a *API) GetSize(pvcRequestConfig PVCRequestConfig) (int64, int64, error)
  • func (a *API) GetSizeHandler() gin.HandlerFunc
  • func (a *API) GetStatus(pvcRequestConfig PVCRequestConfig) (StatusReport, error)
  • func (a *API) GetStatusHandler() gin.HandlerFunc
  • func (a *API) OkHandler(version string, mode string, service string) gin.HandlerFunc
• type Config
• type PVCRequestConfig
• type PatchOperation
• type PatchOperations
• type S3Config
• type StatusReport
• type VolConfig

Constants

const JobAttemptInterval = 5

Types

type API

type API struct {
	*Config
	LogErrors prometheus.Counter
}

API is primary object implementing the core API methods and HTTP handlers

func NewApi

func NewApi(cfg *Config) (*API, error)

NewApi constructs an API object and populates it with configuration along with setting defaults where required.

func (*API) CreatePVC

func (a *API) CreatePVC(pvcRequestConfig PVCRequestConfig) error

CreatePVC is the core purpose of PVCI, to create PVCs and inject them with files. CreatePVC takes a PVCRequestConfig object and creates a Kubernetes PVC, followed by a Kubernetes Job used to populate it.

func (*API) CreatePVCAsyncHandler

func (a *API) CreatePVCAsyncHandler() gin.HandlerFunc

func (*API) CreatePVCHandler

func (a *API) CreatePVCHandler() gin.HandlerFunc

CreatePVCHandler used by the HTTP POST /create endpoint. CreatePVCHandler is the core purpose of PVCI, to create PVCs and inject them with files. This handler expects a JSON object representing a PVCRequestConfig.

func (*API) Delete

func (a *API) Delete(pvcRequestConfig PVCRequestConfig) error

Delete a PVC. @TODO limit to pvc created by PCI by looking at labels

func (*API) DeleteHandler

func (a *API) DeleteHandler() gin.HandlerFunc

DeleteHandler used for the /delete HTTP endpoint to delete a PVC

func (*API) GetSize

func (a *API) GetSize(pvcRequestConfig PVCRequestConfig) (int64, int64, error)

GetSize gets the size of a list of S3/MinIO objects (files) based on bucket and prefix specified in a PVCRequestConfig object.

func (*API) GetSizeHandler

func (a *API) GetSizeHandler() gin.HandlerFunc

GetSizeHandler used by the HTTP POST endpoint /size to get the size of a list of S3/MinIO objects (files) based on bucket and prefix.

func (*API) GetStatus

func (a *API) GetStatus(pvcRequestConfig PVCRequestConfig) (StatusReport, error)

GetStatus returns a StatusReport representing the state of PVCI created Jobs and PVCs.

func (*API) GetStatusHandler

func (a *API) GetStatusHandler() gin.HandlerFunc

GetStatusHandler is used by the HTTP POST /status endpoint and returns a StatusReport object as JSON.

func (*API) OkHandler

func (a *API) OkHandler(version string, mode string, service string) gin.HandlerFunc

OkHandler is provided for created a default slash route for the HTTP API and returns basic version, node and service name.

type Config

type Config struct {
	Service              string
	Version              string
	VolumeOveragePercent int
	AvgMPS               int
	MCImage              string
	Log                  *zap.Logger
	Cs                   *kubernetes.Clientset
}

Config configures the API

type PVCRequestConfig

type PVCRequestConfig struct {
	S3Config
	VolConfig
}

PVCRequestConfig is the primary configuration structure for describing the S3/MinIO cluster to pull objects from and kubernetes pvc to create and place the objects in.

type PatchOperation

type PatchOperation struct {
	Op    string      `json:"op"`
	Path  string      `json:"path"`
	Value interface{} `json:"value,omitempty"`
}

PatchOperation see: http://jsonpatch.com/

type PatchOperations

type PatchOperations []PatchOperation

type S3Config

type S3Config struct {
	S3Endpoint string `json:"s3_endpoint"`
	S3SSL      bool   `json:"s3_ssl"`
	S3Bucket   string `json:"s3_bucket"`
	S3Prefix   string `json:"s3_prefix"`
	S3Key      string `json:"s3_key"`
	S3Secret   string `json:"s3_secret"`
}

S3Config structures authentication, bucket and prefix configuration used to pull objects from an S3/MinIO object cluster.

type StatusReport

type StatusReport struct {
	InjectorHasError bool
	InjectorError    string
	InjectorState    string
	PVCHasError      bool
	PVCError         string
	PVCStatus        coreV1.PersistentVolumeClaimStatus
}

StatusReport structures data returned by the /status endpoint using the GetStatusHandler() and implementing the GetStatus() method in this package.

type VolConfig

type VolConfig struct {
	Namespace    string `json:"namespace"`
	Name         string `json:"name"`
	StorageClass string `json:"storage_class"`
}

VolConfig is part of the PVCRequestConfig and used to specify the name of the volume to create the the Kubernetes storage class. run `kubectl get StorageClass` to see a list of available storage classed for a cluster.