debos

package module
v0.0.0-...-8155755 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 4, 2024 License: Apache-2.0 Imports: 14 Imported by: 0

README

debos - Debian OS image builder

Synopsis

debos [options] <recipe file in YAML>
debos [--help]

Application Options:

  -b, --fakemachine-backend=   Fakemachine backend to use (default: auto)
      --artifactdir=           Directory for packed archives and ostree repositories (default: current directory)
  -t, --template-var=          Template variables (use -t VARIABLE:VALUE syntax)
      --debug-shell            Fall into interactive shell on error
  -s, --shell=                 Redefine interactive shell binary (default: bash) (default: /bin/bash)
      --scratchsize=           Size of disk backed scratch space
  -c, --cpus=                  Number of CPUs to use for build VM (default: 2)
  -m, --memory=                Amount of memory for build VM (default: 2048MB)
      --show-boot              Show boot/console messages from the fake machine
  -e, --environ-var=           Environment variables (use -e VARIABLE:VALUE syntax)
  -v, --verbose                Verbose output
      --print-recipe           Print final recipe
      --dry-run                Compose final recipe to build but without any real work started
      --disable-fakemachine    Do not use fakemachine.

Description

debos is a tool to make the creation of various Debian-based OS images simpler. While most other tools focus on specific use-cases, debos is designed to be a toolchain making common actions trivial while providing enough rope to do whatever tweaking which might be required behind the scenes.

debos expects a YAML file as input and runs the actions listed in the file sequentially. These actions should be self-contained and independent of each other.

Some of the actions provided by debos to customise and produce images are:

  • apt: install packages and their dependencies with 'apt'
  • debootstrap: construct the target rootfs with debootstrap
  • download: download a single file from the internet
  • filesystem-deploy: deploy a root filesystem to an image previously created
  • image-partition: create an image file, make partitions and format them
  • ostree-commit: create an OSTree commit from rootfs
  • ostree-deploy: deploy an OSTree branch to the image
  • overlay: do a recursive copy of directories or files to the target filesystem
  • pack: create a tarball with the target filesystem
  • pacman: install packages and their dependencies with pacman
  • pacstrap: construct the target rootfs with pacstrap
  • raw: directly write a file to the output image at a given offset
  • recipe: includes the recipe actions at the given path
  • run: allows to run a command or script in the filesystem or in the host
  • unpack: unpack files from archive in the filesystem

A full syntax description of all the debos actions can be found at: https://godoc.org/github.com/go-debos/debos/actions

Installation (Docker container)

An official debos container is available:

docker pull godebos/debos

See docker/README.md for usage.

Installation from source (under Debian)

sudo apt install golang git libglib2.0-dev libostree-dev qemu-system-x86 \
     qemu-user-static debootstrap systemd-container
export GOPATH=/opt/src/gocode # or whatever suits your needs
go install -v github.com/go-debos/debos/cmd/debos@latest
/opt/src/gocode/bin/debos --help

Simple example

The following example will create an arm64 image, install several packages in it, change the file /etc/hostname to "debian" and finally make a tarball of the complete system.

{{- $image := or .image "debian.tgz" -}}

architecture: arm64

actions:
  - action: debootstrap
    suite: bookworm
    components:
      - main
      - non-free-firmware
    mirror: https://deb.debian.org/debian
    variant: minbase

  - action: apt
    packages: [ sudo, openssh-server, adduser, systemd-sysv, firmware-linux ]

  - action: run
    chroot: true
    command: echo debian > /etc/hostname

  - action: pack
    file: {{ $image }}
    compression: gz

To run it, create a file named example.yaml and run:

debos example.yaml

The final tarball will be named "debian.tgz" if you would like to modify the fileame, you can provided a different name for the variable image like this:

debos -t image:"debian-arm64.tgz" example.yaml

Other examples

Example recipes are collected in a separate repository:

https://github.com/go-debos/debos-recipes

Environment variables

debos reads a predefined list of environment variables from the host and propagates them to the fakemachine build environemnt. The set of environment variables is defined by environ_vars in cmd/debos/debos.go. Currently the list of environment variables includes the proxy environment variables documented at:

https://wiki.archlinux.org/index.php/proxy_settings

The list of environment variables currently exported to fakemachine is:

http_proxy, https_proxy, ftp_proxy, rsync_proxy, all_proxy, no_proxy

While the elements of environ_vars are in lower case, for each element both lower and upper case variants are probed on the host and if found propagated to fakemachine. So if the host has the environment variables HTTP_PROXY and no_proxy defined, both will be propagated to fakemachine respecting the case.

The command line options --environ-var and -e can be used to specify, overwrite and unset environment variables for fakemachine with the syntax:

$ debos -e ENVIRONVAR:VALUE ...

To unset an environment variable, or in other words, to prevent an environment variable being propagated to fakemachine, use the same syntax without a value. debos accepts multiple -e simultaneously.

Proxy configuration

While the proxy related environment variables are exported from the host to fakemachine, there are two known sources of issues:

  • Using localhost will not work from fakemachine. Use an address which is valid on your network. debos will warn if the environment variables contain localhost.

  • In case you are running applications and/or scripts inside fakemachine you may need to check which are the proxy environment variables they use. Different apps are known to use different environment variable names and different case for environment variable names.

Fakemachine Backend

debos (unless running debos with the --disable-fakemachine argument) creates and spawns a virtual machine using fakemachine and executes the actions defined by the recipe inside the virtual machine. This helps ensure recipes are reproducible no matter the host environment.

Fakemachine can use different virtualisation backends to spawn the virtual machine, for more information see the fakemachine documentation.

By default the backend will automatically be selected based on what is supported by the host machine, but this can be overridden using the --fakemachine-backend / -b option. If no backends are supported, debos reverts to running the recipe on the host without creating a fakemachine.

Performance of the backends is roughly as follows: kvm is faster than uml is faster than qemu. Using --disable-fakemachine is slightly faster than kvm, but requires root permissions.

Benchmark times for running pine-a64-plus/debian.yaml on an Intel Pentium G4560T with SSD:

Backend Wall Time Prerequisites
--disable-fakemachine 8 min root permissions
-b kvm 9 min access to /dev/kvm
-b uml 18 min package user-mode-linux installed
-b qemu 166 min none

Documentation

Index

Constants

View Source 
const (
	CHROOT_METHOD_NONE = iota // No chroot in use
	//CHROOT_METHOD_NSPAWN        // use nspawn to create the chroot environment
	CHROOT_METHOD_CHROOT // use chroot to create the chroot environment
)

Variables

This section is empty.

Functions

func CleanPath 

func CleanPath(path string) string

func CleanPathAt 

func CleanPathAt(path, at string) string

func CopyFile 

func CopyFile(src, dst string, mode os.FileMode) error

func CopyTree 

func CopyTree(sourcetree, desttree string) error

func DebugShell 

func DebugShell(context DebosContext)

DebugShell function launches an interactive shell for debug and problems investigation.

func DownloadHttpUrl 

func DownloadHttpUrl(url, filename string) error

Function for downloading single file object with http(s) protocol

func RealPath 

func RealPath(path string) (string, error)

func RestrictedPath 

func RestrictedPath(prefix, dest string) (string, error)

Types

type Action 

type Action interface {
	/* FIXME verify should probably be prepare or somesuch */
	Verify(context *DebosContext) error
	PreMachine(context *DebosContext, m *fakemachine.Machine, args *[]string) error
	PreNoMachine(context *DebosContext) error
	Run(context *DebosContext) error
	// Cleanup() method gets called only if the Run for an action
	// was started and in the same machine (host or fake) as Run has run
	Cleanup(context *DebosContext) error
	PostMachine(context *DebosContext) error
	// PostMachineCleanup() gets called for all actions if Pre*Machine() method
	// has run for Action. This method is always executed on the host with user's permissions.
	PostMachineCleanup(context *DebosContext) error
	String() string
}

type Archive 

type Archive struct {
	Archiver
}

func NewArchive 

func NewArchive(file string, arcType ...ArchiveType) (Archive, error)

NewArchive associate correct structure and methods according to archive type. If ArchiveType is omitted -- trying to guess the type. Return ArchiveType or nil in case of error.

type ArchiveBase 

type ArchiveBase struct {
	// contains filtered or unexported fields
}

func (*ArchiveBase) AddOption 

func (arc *ArchiveBase) AddOption(key, value interface{}) error

func (*ArchiveBase) RelaxedUnpack 

func (arc *ArchiveBase) RelaxedUnpack(destination string) error

RelaxedUnpack unpack archive in relaxed mode allowing to ignore or avoid minor issues with unpacker tool or framework.

func (*ArchiveBase) Type 

func (arc *ArchiveBase) Type() ArchiveType

func (*ArchiveBase) Unpack 

func (arc *ArchiveBase) Unpack(destination string) error

Unpack archive as is

type ArchiveDeb 

type ArchiveDeb struct {
	ArchiveBase
}

func (*ArchiveDeb) RelaxedUnpack 

func (deb *ArchiveDeb) RelaxedUnpack(destination string) error

func (*ArchiveDeb) Unpack 

func (deb *ArchiveDeb) Unpack(destination string) error

type ArchiveTar 

type ArchiveTar struct {
	ArchiveBase
}

func (*ArchiveTar) AddOption 

func (tar *ArchiveTar) AddOption(key, value interface{}) error

func (*ArchiveTar) RelaxedUnpack 

func (tar *ArchiveTar) RelaxedUnpack(destination string) error

func (*ArchiveTar) Unpack 

func (tar *ArchiveTar) Unpack(destination string) error

type ArchiveType 

type ArchiveType int
const (
	Tar ArchiveType
	Zip
	Deb
)

Supported types

type ArchiveZip 

type ArchiveZip struct {
	ArchiveBase
}

func (*ArchiveZip) RelaxedUnpack 

func (zip *ArchiveZip) RelaxedUnpack(destination string) error

func (*ArchiveZip) Unpack 

func (zip *ArchiveZip) Unpack(destination string) error

type Archiver 

type Archiver interface {
	Type() ArchiveType
	AddOption(key, value interface{}) error
	Unpacker
}

type BaseAction 

type BaseAction struct {
	Action      string
	Description string
}

func (*BaseAction) Cleanup 

func (b *BaseAction) Cleanup(context *DebosContext) error

func (*BaseAction) PostMachine 

func (b *BaseAction) PostMachine(context *DebosContext) error

func (*BaseAction) PostMachineCleanup 

func (b *BaseAction) PostMachineCleanup(context *DebosContext) error

func (*BaseAction) PreMachine 

func (b *BaseAction) PreMachine(context *DebosContext,
	m *fakemachine.Machine,
	args *[]string) error

func (*BaseAction) PreNoMachine 

func (b *BaseAction) PreNoMachine(context *DebosContext) error

func (*BaseAction) Run 

func (b *BaseAction) Run(context *DebosContext) error

func (*BaseAction) String 

func (b *BaseAction) String() string

func (*BaseAction) Verify 

func (b *BaseAction) Verify(context *DebosContext) error

type ChrootEnterMethod 

type ChrootEnterMethod int

type Command 

type Command struct {
	Architecture string            // Architecture of the chroot, nil if same as host
	Dir          string            // Working dir to run command in
	Chroot       string            // Run in the chroot at path
	ChrootMethod ChrootEnterMethod // Method to enter the chroot
	// contains filtered or unexported fields
}

func NewChrootCommandForContext 

func NewChrootCommandForContext(context DebosContext) Command

func (*Command) AddBindMount 

func (cmd *Command) AddBindMount(source, target string)

func (*Command) AddEnv 

func (cmd *Command) AddEnv(env string)

func (*Command) AddEnvKey 

func (cmd *Command) AddEnvKey(key, value string)

func (Command) Run 

func (cmd Command) Run(label string, cmdline ...string) error

type CommonContext 

type CommonContext struct {
	Scratchdir      string
	Rootdir         string
	Artifactdir     string
	Downloaddir     string
	Image           string
	ImagePartitions []Partition
	ImageMntDir     string
	ImageFSTab      bytes.Buffer // Fstab as per partitioning
	ImageKernelRoot string       // Kernel cmdline root= snippet for the / of the image
	DebugShell      string
	Origins         map[string]string
	State           DebosState
	EnvironVars     map[string]string
	PrintRecipe     bool
	Verbose         bool
}

type DebosContext 

type DebosContext struct {
	*CommonContext
	RecipeDir    string
	Architecture string
}

func (*DebosContext) Origin 

func (c *DebosContext) Origin(o string) (string, bool)

type DebosState 

type DebosState int
const (
	Success DebosState = iota
	Failed
)

Represent the current state of Debos

type Partition 

type Partition struct {
	Name       string
	DevicePath string
}

Mapping from partition name as configured in the image-partition action to device path for usage by other actions

type ServiceHelper 

type ServiceHelper struct {
	Rootdir string
}

func (*ServiceHelper) Allow 

func (s *ServiceHelper) Allow() error

Allow() allows to start/stop services on OS level.

func (*ServiceHelper) Deny 

func (s *ServiceHelper) Deny() error

Deny() prohibits to start/stop services on OS level.

type ServicesManager 

type ServicesManager interface {
	Allow() error
	Deny() error
}

type Unpacker 

type Unpacker interface {
	Unpack(destination string) error
	RelaxedUnpack(destination string) error
}

Source Files

View all Source files

Directories

Path Synopsis
Package 'actions' implements 'debos' modules used for OS creation.
 Package 'actions' implements 'debos' modules used for OS creation.
cmd
debos

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.