internal

Documentation

Overview

Package internal houses code for wr's general utility functions.

It also implements the the config system used by the cmd package (see config.go).

import "github.com/VertebrateResequencing/wr/internal"
import "github.com/inconshreveable/log15"
deployment := internal.DefaultDeployment()
logger := log15.New()
logger.SetHandler(log15.LvlFilterHandler(log15.LvlWarn, log15.StderrHandler))
config := internal.ConfigLoad(deployment, false, logger)
port := config.ManagerPort

Index

• Constants
• func CheckCerts(serverPemFile string, serverKeyFile string) error
• func CurrentIP(cidr string) (string, error)
• func DefaultDeployment(logger log15.Logger) string
• func DefaultServer(logger log15.Logger) (server string)
• func DiskSize(dir string) int
• func DockerStats(dockerClient *docker.Client, containerID string) (memMB int, cpuSec int, err error)
• func FileMD5(path string, logger log15.Logger) (string, error)
• func FloatAdd(a, b float64) float64
• func FloatLessThan(a, b float64) bool
• func FloatSubtract(a, b float64) float64
• func GenerateCerts(caFile, serverPemFile, serverKeyFile, domain string) error
• func InS3(path string) bool
• func InfobloxSetDomainIP(domain, ip string) error
• func IsRemote(path string) bool
• func LogClose(logger log15.Logger, obj io.Closer, msg string, extra ...interface{})
• func LogPanic(logger log15.Logger, desc string, die bool)
• func NoDiskSpaceLeft(dir string) bool
• func ProcMeminfoMBs() (int, error)
• func RandomString() string
• func SortMapKeysByIntValue(imap map[string]int, reverse bool) []string
• func SortMapKeysByMapIntValue(imap map[string]map[string]int, criterion string, reverse bool) []string
• func TildaToHome(path string) string
• func Userid() (int, error)
• func Username() (string, error)
• func WaitForFile(file string, timeout time.Duration) bool
• func Which(exeName string) string
• type Config
  • func ConfigLoad(deployment string, useparentdir bool, logger log15.Logger) Config
  • func DefaultConfig(logger log15.Logger) Config
  • func (c Config) IsDevelopment() bool
  • func (c Config) IsProduction() bool

Constants

const (

	// S3Prefix is the prefix used by S3 paths
	S3Prefix = "s3://"

	// Production is the name of the main deployment
	Production = "production"

	// Development is the name of the development deployment, used during testing
	Development = "development"
)

Functions

func CheckCerts

func CheckCerts(serverPemFile string, serverKeyFile string) error

CheckCerts checks if the given cert and key file are readable. If one or both of them are not, returns an error.

func CurrentIP

func CurrentIP(cidr string) (string, error)

CurrentIP returns the IP address of the machine we're running on right now. The cidr argument can be an empty string, but if set to the CIDR of the machine's primary network, it helps us be sure of getting the correct IP address (for when there are multiple network interfaces on the machine).

func DefaultDeployment

func DefaultDeployment(logger log15.Logger) string

DefaultDeployment works out the default deployment.

func DefaultServer

func DefaultServer(logger log15.Logger) (server string)

DefaultServer works out the default server (we need this to be able to report this default before we know what deployment the user has actually chosen, ie. before we have a final config).

func DiskSize

func DiskSize(dir string) int

DiskSize returns the size of the disk (mounted at the given directory, "." for current) in GB.

func DockerStats

func DockerStats(dockerClient *docker.Client, containerID string) (memMB int, cpuSec int, err error)

DockerStats asks docker for the current memory usage (RSS) and total CPU usage of the given container.

func FileMD5

func FileMD5(path string, logger log15.Logger) (string, error)

FileMD5 calculates the MD5 hash checksum of a file, returned as HEX encoded.

func FloatAdd

func FloatAdd(a, b float64) float64

FloatAdd does a + b, treating both float64s rounded to 3 decimal places of precision.

func FloatLessThan

func FloatLessThan(a, b float64) bool

FloatLessThan tells you if a < b, treating both float64s rounded to 3 decimal places of precision.

func FloatSubtract

func FloatSubtract(a, b float64) float64

FloatSubtract does a - b, treating both float64s rounded to 3 decimal places of precision.

func GenerateCerts

func GenerateCerts(caFile, serverPemFile, serverKeyFile, domain string) error

GenerateCerts creates a CA certificate which is used to sign a created server certificate which will have a corresponding key, all saved as PEM files. An error is generated if any of the files already exist.

func InS3

func InS3(path string) bool

InS3 tells you if a path is to a file in S3.

func InfobloxSetDomainIP

func InfobloxSetDomainIP(domain, ip string) error

InfobloxSetDomainIP uses infoblox to set the IP of a domain. Returns an error if INFOBLOX_HOST, INFOBLOX_USER or INFOBLOX_PASS env vars are not set.

func IsRemote

func IsRemote(path string) bool

IsRemote tells you if a path is to a remote file system or object store, based on its URI.

func LogClose

func LogClose(logger log15.Logger, obj io.Closer, msg string, extra ...interface{})

LogClose is for use to Close() an object during a defer when you don't care if the Close() returns an error, but do want non-EOF errors logged. Extra args are passed as additional context for the logger.

func LogPanic

func LogPanic(logger log15.Logger, desc string, die bool)

LogPanic is for use in a go routines, deferred at the start of them, to figure out what is causing runtime panics. If the die bool is true, the program exits, otherwise it continues, after logging the error message and stack trace. Desc string should be used to describe briefly what the goroutine you call this in does.

func NoDiskSpaceLeft

func NoDiskSpaceLeft(dir string) bool

NoDiskSpaceLeft tells you if the disk (mounted at the given directory, "." for current) has no more space left (or is within 100MB of being full).

func ProcMeminfoMBs

func ProcMeminfoMBs() (int, error)

ProcMeminfoMBs uses gopsutil (amd64 freebsd, linux, windows, darwin, openbds only!) to find the total number of MBs of memory physically installed on the current system.

func RandomString

func RandomString() string

RandomString generates a random string of length 8 characters.

func SortMapKeysByIntValue

func SortMapKeysByIntValue(imap map[string]int, reverse bool) []string

SortMapKeysByIntValue sorts the keys of a map[string]int by its values, reversed if you supply true as the second arg.

func SortMapKeysByMapIntValue

func SortMapKeysByMapIntValue(imap map[string]map[string]int, criterion string, reverse bool) []string

SortMapKeysByMapIntValue sorts the keys of a map[string]map[string]int by a the values found at a given sub value, reversed if you supply true as the second arg.

func TildaToHome

func TildaToHome(path string) string

TildaToHome converts a path beginning with ~/ to the absolute path based in the current home directory (according to the environment variable $HOME).

func Userid

func Userid() (int, error)

Userid returns the user id of the current user. This avoids problems with static compilation as it avoids the use of os/user. It will only work on linux-like systems where 'id -u' works.

func Username

func Username() (string, error)

Username returns the username of the current user. This avoids problems with static compilation as it avoids the use of os/user. It will only work on linux-like systems where 'id -u -n' works.

func WaitForFile

func WaitForFile(file string, timeout time.Duration) bool

WaitForFile waits as long as timeout for the given file to exist. When it exists, returns true. Otherwise false.

func Which

func Which(exeName string) string

Which returns the full path to the executable with the given name that is found first in the set of $PATH directories, ignoring any path that is actually a symlink to ourselves.

Types

type Config

type Config struct {
	ManagerPort         string `default:""`
	ManagerWeb          string `default:""`
	ManagerHost         string `default:"localhost"`
	ManagerDir          string `default:"~/.wr"`
	ManagerPidFile      string `default:"pid"`
	ManagerLogFile      string `default:"log"`
	ManagerDbFile       string `default:"db"`
	ManagerDbBkFile     string `default:"db_bk"`
	ManagerTokenFile    string `default:"client.token"`
	ManagerUploadDir    string `default:"uploads"`
	ManagerUmask        int    `default:"007"`
	ManagerScheduler    string `default:"local"`
	ManagerCAFile       string `default:"ca.pem"`
	ManagerCertFile     string `default:"cert.pem"`
	ManagerKeyFile      string `default:"key.pem"`
	ManagerCertDomain   string `default:"localhost"`
	ManagerSetDomainIP  bool   `default:"false"`
	RunnerExecShell     string `default:"bash"`
	Deployment          string `default:"production"`
	CloudFlavor         string `default:""`
	CloudKeepAlive      int    `default:"120"`
	CloudServers        int    `default:"-1"`
	CloudCIDR           string `default:"192.168.0.0/18"`
	CloudGateway        string `default:"192.168.0.1"`
	CloudDNS            string `default:"8.8.4.4,8.8.8.8"`
	CloudOS             string `default:"Ubuntu Xenial"`
	CloudUser           string `default:"ubuntu"`
	CloudRAM            int    `default:"2048"`
	CloudDisk           int    `default:"1"`
	CloudScript         string `default:""`
	CloudConfigFiles    string `default:"~/.s3cfg,~/.aws/credentials,~/.aws/config"`
	DeploySuccessScript string `default:""`
}

Config holds the configuration options for jobqueue server and client

func ConfigLoad

func ConfigLoad(deployment string, useparentdir bool, logger log15.Logger) Config

ConfigLoad loads configuration settings from files and environment variables. Note, this function exits on error, since without config we can't do anything.

We prefer settings in config file in current dir (or the current dir's parent dir if the useparentdir option is true (used for test scripts)) over config file in home directory over config file in dir pointed to by WR_CONFIG_DIR.

The deployment argument determines if we read .wr_config.production.yml or .wr_config.development.yml; we always read .wr_config.yml. If the empty string is supplied, deployment is development if you're in the git repository directory. Otherwise, deployment is taken from the environment variable WR_DEPLOYMENT, and if that's not set it defaults to production.

Multiple of these files can be used to have settings that are common to multiple users and deployments, and settings specific to users or deployments.

Settings found in no file can be set with the environment variable WR_<setting name in caps>, eg. export WR_MANAGER_PORT="11301"

func DefaultConfig

func DefaultConfig(logger log15.Logger) Config

DefaultConfig works out the default config for when we need to be able to report the default before we know what deployment the user has actually chosen, ie. before we have a final config.

func (Config) IsDevelopment

func (c Config) IsDevelopment() bool

IsDevelopment tells you if we're in the development deployment.

func (Config) IsProduction

func (c Config) IsProduction() bool

IsProduction tells you if we're in the production deployment.