fleetlock
fleetlock
is a reboot coordinator for Fedora CoreOS nodes in Kubernetes clusters. It implements the FleetLock protocol for use as a Zincati lock strategy backend.
Usage
Zincati runs on-host (zincati.service
). Declare a Zincati fleet_lock
strategy when provisioning Fedora CoreOS nodes. Set base_url
for host nodes to access the in-cluster fleetlock
Service (e.g. known ClusterIP).
variant: fcos
version: 1.4.0
storage:
files:
- path: /etc/zincati/config.d/55-update-strategy.toml
contents:
inline: |
[updates]
strategy = "fleet_lock"
[updates.fleet_lock]
base_url = "http://10.3.0.15/"
Apply the fleetlock
Deployment, Service (with ClusterIP), and ServiceAccount.
kubectl apply -f examples/k8s
Inspect the fleetlock Lease object.
$ kubectl get leases -n default
NAME HOLDER AGE
fleetlock-default 049ad0f57ade4723a48692b7b692c318 4m50s
Configuration
Configure the server via flags.
flag |
description |
default |
-address |
HTTP listen address |
0.0.0.0:8080 |
-log-level |
Logger level |
info |
-version |
Show version |
NA |
-help |
Show help |
NA |
Or via environment variables.
variable |
description |
default |
NAMESPACE |
Kubernetes Namespace |
"default" |
KUBECONFIG |
Development Kubeconfig |
NA |
Typhoon
For Typhoon clusters, add the Zincati config a snippet.
module "nemo" {
...
controller_snippets = [
file("./snippets/zincati-strategy.yaml"),
]
worker_snippets = [
file("./snippets/zincati-strategy.yaml"),
]
}
Manual Intervention
fleetlock
coordinates OS auto-updates to avoid concurrent node updates or a potential bad auto-update continuing. Zincati obtains a reboot lease lock before finalization (i.e reboot).
If an auto-update fails, the lease continues to be held by design. An admin should investigate the node failure and decide whether it is safe to remove the lease.
$ kubectl get leases
$ kubectl delete lease fleetlock-default
Metrics
fleetlock
serves Prometheus /metrics
from Go, process, and custom collectors.
name |
description |
fleetlock_lock_state |
State of the fleetlock lease (0 unlocked, 1 locked) |
fleetlock_lock_transition_count |
Number of fleetlock lease transitions |
fleetlock_lock_request_count |
Number of lock requests |
fleetlock_unlock_request_count |
Number of unlock requests |
Development
To develop locally, build and run the executable.
Static Binary
Build the static binary.
make build
Container Image
Build the container image.
make image
Run
Run the executable.
export KUBECONFIG=some-dev-kubeconfig
./bin/fleetlock
Use curl to emulate a Zincati FleetLock client.
{
"client_params": {
"id": "c988d2509fdf5cdcbed39037c56406fb",
"group": "default"
}
}
Request a reboot lock.
curl -H "fleet-lock-protocol: true" -d @examples/body.json http://127.0.0.1:8080/v1/pre-reboot
Release a reboot lock.
curl -H "fleet-lock-protocol: true" -d @examples/body.json http://127.0.0.1:8080/v1/steady-state