cbd

package module
v0.0.0-...-fd4ca52 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 30, 2015 License: BSD-3-Clause Imports: 24 Imported by: 0

README

cbd

A distributed build and caching engine for C and C++.

Usage

On your "build-server" host start the server listening on port 18000:

cbd server -port 18000

Start up workers on your various hosts and point them toward the server (they will be listening on port 17000):

export CBD_SERVER=build-server:18000
cbd worker -port 17000

Use the client program in place of gcc and g++:

export CBD_SERVER=build-server:18000
export CC='cbd gcc'
export CXX='cbd  g++'

Now run your build tool as normal. If you set CBD_LOGFILE to point to a file, cbdcc will write verbose debug logging statements their.

Roadmap

  • Introduce some kind of queuing behavior to handle a loaded cluster
  • Centralized object file cache
  • More monitoring
    • Maybe events for start of pre-process
    • Start/Stop of data send
  • Queueing jobs on the server
  • Embedded web browser for monitoring GUI

TODO

  • Use worker IP addresses to identify themselves not hostnames
  • Workers should chose their port automatically
  • Everywhere we use CBD_SERVER we should accept a command line argument
  • Worker updates to monitoring programs should be on-demand

Development

The shell script "test.sh" runs the unit tests as well as some integration tests. The standard "go test" will run the unit tests. The "build.sh" program will build the software and install the binaries on the GOPATH.

Help full tools to have installed, goimports will automatically fix up imports, and can replace the usage of gofmt:

go get -u code.google.com/p/go.tools/cmd/goimports

godef will help look up definitions, it integrates with go-mode.el:

go get -u code.google.com/p/rog-go/exp/cmd/godef

Emacs Specific, setup, get the main mode:

git clone https://github.com/dominikh/go-mode.el

Get flymake support with goflymake for as you type error checking:

git clone https://github.com/dougm/goflymake

go get -u github.com/dougm/goflymake

TODO: maybe gocode & oracle? as well?

Environment Variables

  • CBD_SERVER - of the form "1.2.3.4:4000", identifies the server, required for setting up workers (port optional).
  • CBD_POTENTIAL_HOST - of the form "1.2.3.4:4000", tells the worker to connect directly to that worker, instead of querying the server for a worker.
  • CBD_LOGFILE - path to the debug log file. If not present, no log is created.
  • CBD_NO_LOCAL - client error out if it can't build on a remote host, mostly used for testing.

Design

The main parts of the final system and their purpose:

Server

  • Tracks the load status of workers.
  • Responds to requests sending back an available worker

Worker

  • Accepts build jobs, compiles and returns the results
  • Sends updates containing load and CPU count to the server

Client

  • Stands in for the compiler turning commands into build jobs
  • Uses the server to find a worker to complete it's job

Documentation

Index

Constants

View Source 
const (
	// Default port used by the server
	DefaultServerPort = uint(15796)
	// Default port used by the worker
	DefaultWorkerPort = uint(15797)
	// Beginning of our worker listen port range
	StartPort = DefaultWorkerPort
	// End of our listen port range
	EndPort = 15900
	// UPD port used for auto discovery
	DiscoveryPort = 15932
)
View Source 
const (
	DISC_SERVER = uint8(iota)
	DISC_CLIENT = uint8(iota)
)

Packet types

Variables

View Source 
var (
	// Whether or not we have debug logging on
	DebugLogging = false
)

Functions

func Copyfile 

func Copyfile(dst, src string) error

copies dst to src location, no metadata is copied

func DebugPrint 

func DebugPrint(v ...interface{})

Make this log statement only when debugging logging is on

func DebugPrintf 

func DebugPrintf(format string, v ...interface{})

Printf style debug logging

func GetLoadAverage 

func GetLoadAverage() (float64, error)

GetLoadAverage returns the 1 minute system load average TODO: linux only, support more of unix by using cgo and getloadavg

func MakeCompileJob 

func MakeCompileJob(compiler string, b Build) (j CompileJob, results ExecResult, err error)

MakeCompileJob takes the requested Build, pre-processses the needed file and returns a CompileJob with code.

func NetListen 

func NetListen(iport int) (ln net.Listener, port int, err error)

Listen on the given port, if zero an open port is found

func TempFile 

func TempFile(dir, prefix string, suffix string) (f *os.File, err error)

Generates and opens a temporary file with a defined prefix and suffix This is the same api as ioutil.TempFile accept it accepts a suffix 

TODO: see if this is too slow

Types

type Build 

type Build struct {
	Args          []string // Command line arguments
	Oindex        int      // Index of argument *before* the output file
	Iindex        int      // Index of input file option
	Cindex        int      // Index of "type" flag
	Distributable bool     // A job we can distribute

}

func ParseArgs 

func ParseArgs(args []string) Build

Takes in all the compiler arguments, without the actual compiler command, so "gcc -c data/main.c -o main.o" -> {'-c', 'data/main.c', '-o', 'main.o'}

func (Build) Input 

func (b Build) Input() string

Return the input path for the build job

func (Build) Output 

func (b Build) Output() string

Returns the output path build job

type ByPrivateIPAddr 

type ByPrivateIPAddr []net.IPNet

Custom IP sortring (doesn't support IPv6), which puts private IP addresses first, 192, then 172, then 10, then public

func (ByPrivateIPAddr) Len 

func (a ByPrivateIPAddr) Len() int

func (ByPrivateIPAddr) Less 

func (a ByPrivateIPAddr) Less(i, j int) bool

func (ByPrivateIPAddr) Swap 

func (a ByPrivateIPAddr) Swap(i, j int)

type CompileJob 

type CompileJob struct {
	Host     string // The host requesting it
	Build    Build  // The commands to build it with
	Input    []byte // The data to build
	Compiler string // The compiler to run it with
}

A job to be farmed out to our cluster

func (CompileJob) Compile 

func (c CompileJob) Compile() (result CompileResult, err error)

Compile a job locally using temporary files and return the result

func (CompileJob) Validate 

func (c CompileJob) Validate() (err error)

Return an error if there is something wrong with the build job

type CompileResult 

type CompileResult struct {
	ExecResult        // Results of the compiler command
	ObjectCode []byte // The compiled object code
}

The result of a compile

func ClientBuildJob 

func ClientBuildJob(job CompileJob) (cresults CompileResult, err error)

TODO: this needs some tests

type CompletedJob 

type CompletedJob struct {
	Client       MachineName   // Machine that requested the job
	Worker       MachineName   // Worker that build the job
	InputSize    int           // Bytes of source code compiled
	OutputSize   int           // Bytes of object code produced
	CompileTime  time.Duration // How long the job took to complete
	CompileSpeed float64       // Speed rating used for the job
}

CompletedJob is one updated about a job completed on the cluster

type DeadlineReadWriter 

type DeadlineReadWriter interface {
	io.Reader
	io.Writer

	// SetReadDeadline sets the deadline for future Read calls.
	// A zero value for t means Read will not time out.
	SetReadDeadline(t time.Time) error

	// SetWriteDeadline sets the deadline for future Write calls.
	// Even if write times out, it may return n > 0, indicating that
	// some of the data was successfully written.
	// A zero value for t means Write will not time out.
	SetWriteDeadline(t time.Time) error
}

DeadlineReadWriter is an interface that lets you read and write with a possible timeut based on deadlines.

type ExecResult 

type ExecResult struct {
	Output []byte // Output of the command
	Return int    // Return code of program
}

The result of running a command

func Compile 

func Compile(compiler string, b Build, input string) (resultPath string, result ExecResult, err error)

Build the file at the temporary location, you must clean up the returned file.

func Preprocess 

func Preprocess(compiler string, b Build) (resultPath string, result ExecResult, err error)

Build the file at the temporary location, you must clean up the returned file.

func RunCmd 

func RunCmd(prog string, args []string) (result ExecResult, err error)

Executes, returning the stdout if the program fails (the return code is ignored)

type FifoScheduler 

type FifoScheduler struct {
	// contains filtered or unexported fields
}

type GUID 

type GUID [16]byte

Our UUID v4 globally unique id

func NewGUID 

func NewGUID() GUID

Generate a totally random UUID v4

func (*GUID) String 

func (g *GUID) String() string

Turn that UUID into a string of the format: 

cd67b045-0f86-42b4-c2be-be31003ee83d

type MachineID 

type MachineID string

String that unique identifies a machine

func GetMachineID 

func GetMachineID() (m MachineID, err error)

Returns the ID of the current machine

func (*MachineID) ToString 

func (m *MachineID) ToString() string

type MachineName 

type MachineName struct {
	ID   MachineID // The unique ID of the machine
	Host string    // The host name of the machine
}

Convenient wrapper for ID & Host

func (*MachineName) ToString 

func (mn *MachineName) ToString() string

type MessageConn 

type MessageConn struct {
	// contains filtered or unexported fields
}

A connection which you can send and receive messages over

func NewMessageConn 

func NewMessageConn(c DeadlineReadWriter, d time.Duration) *MessageConn

Create a message connection with the given buffer

func NewTCPMessageConn 

func NewTCPMessageConn(address string, d time.Duration) (*MessageConn, error)

Create a TCP based message conn

func (MessageConn) Read 

func (mc MessageConn) Read() (MessageHeader, interface{}, error)

Generic Read function makes it possible to read messages of different types on the same pipe

func (MessageConn) ReadCompileJob 

func (mc MessageConn) ReadCompileJob() (j CompileJob, err error)

func (MessageConn) ReadCompileResult 

func (mc MessageConn) ReadCompileResult() (r CompileResult, err error)

func (MessageConn) ReadCompletedJob 

func (mc MessageConn) ReadCompletedJob() (c CompletedJob, err error)

func (MessageConn) ReadType 

func (mc MessageConn) ReadType(eID MessageID) (interface{}, error)

func (MessageConn) ReadWorkerResponse 

func (mc MessageConn) ReadWorkerResponse() (r WorkerResponse, err error)

func (MessageConn) ReadWorkerState 

func (mc MessageConn) ReadWorkerState() (s WorkerState, err error)

func (MessageConn) Send 

func (mc MessageConn) Send(i interface{}) (err error)

Generic send function, makes it simpler to send messages

type MessageHeader 

type MessageHeader struct {
	ID MessageID
}

The MessageHeader procedes each message in our data stream, it lets us determine what exact type is in the nessage

type MessageID 

type MessageID int
const (
	CompileJobID MessageID = iota
	CompileResultID
	WorkerRequestID
	WorkerResponseID
	WorkerStateID
	MonitorRequestID
	CompletedJobID
	WorkerStateListID
)

Our list of message types, use to figure out what exactly each type of message is

func (MessageID) String 

func (mID MessageID) String() string

type Monitor 

type Monitor struct {
	// contains filtered or unexported fields
}

func NewMonitor 

func NewMonitor(saddr string) *Monitor

Creates a new connects to the server

func (*Monitor) BasicReport 

func (m *Monitor) BasicReport() error

Print out report data in raw form, connecting if needed

func (*Monitor) Connect 

func (m *Monitor) Connect() error

Connect to the server and sends monitoring request

type MonitorRequest 

type MonitorRequest struct {
	Host string
}

MonitorRequest is sent from a client that wishes to be sent information about the current jobs running on the build cluster.

type ResponseType 

type ResponseType int

Determine what kind of response the server sent 

const (
	Queued    ResponseType = iota // No data, we are queued
	NoWorkers                     // No workers at all available
	Valid                         // Valid response
)

type Scheduler 

type Scheduler interface {
	// contains filtered or unexported methods
}

Schedules jobs amongst a pool of workers

type SchedulerRequest 

type SchedulerRequest struct {
	// contains filtered or unexported fields
}

The information needed

func NewSchedulerRequest 

func NewSchedulerRequest(addrs []net.IPNet) *SchedulerRequest

type ServerState 

type ServerState struct {
	// contains filtered or unexported fields
}

ServerState is all the state of our server TODO: consider some kind of channel system instead of a mutex to get sync access to these data structures.

func NewServerState 

func NewServerState() *ServerState

func (*ServerState) Serve 

func (s *ServerState) Serve(ln net.Listener)

server accepts incoming connections

type Worker 

type Worker struct {
	// contains filtered or unexported fields
}

func NewWorker 

func NewWorker(port int, saddr string) (w *Worker, err error)

NewWorker initializes a Worker struct based on the given server and local address. The local address will be parsed to determine our local port for receiving connections.

func (*Worker) Serve 

func (w *Worker) Serve(ln net.Listener) error

Serve listens for incoming build requests connections and spawns goroutines to handle them as needed. If we have a server address it will send status updates there as well.

type WorkerRequest 

type WorkerRequest struct {
	Client string      // Host request a worker
	Addrs  []net.IPNet // IP addresses of the client
}

WorkerRequest is sent from the client to the server in order to find a worker to process a job

type WorkerResponse 

type WorkerResponse struct {
	Type    ResponseType // Valid or queue
	ID      MachineID    // Uniquely identifies machine
	Host    string       // Host of the worker (for debugging purposes)
	Address net.IPNet    // IP address of the worker
	Port    int          // Port the workers accepts connections on
}

type WorkerState 

type WorkerState struct {
	ID       MachineID   // Uniquely id for the worker machine
	Host     string      // Host the worker resides one
	Addrs    []net.IPNet // IP addresses of the worker
	Port     int         // Port the worker accepts jobs on
	Capacity int         // Number of available cores for building
	Load     int         // How many cores are current in use
	Updated  time.Time   // When the state was last updated
	Speed    float64     // The speed of the worker, computed on the server
}

WorkState represents the load and capacity of a worker

type WorkerStateList 

type WorkerStateList struct {
	Workers []WorkerState
}

List of all currently active works

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.