tracing

README

Tracing

A light-weight library for manual distributed system tracing.

Package tracing provides a tracing library, in order to allow for precise automatic grading in CPSC 416. A trace provides a precise, ordered representation of what your assignment code is doing (well, what it says it's doing), which can be used to assess some things that are unclear from either unit testing or code inspection. These include correct concurrency management, as well as properly following any sequencing/causality rules required by the protocol you are implementing.

The tracing library is split into two parts: the tracing server TracingServer, and the tracing client Tracer. You should have one instance of Tracer per network node, and you should first get access to a Trace and then you can record an action by calling Trace.RecordAction(action). A Trace is a set of recorded actions that are associated with a unique trace ID. With traces, actions are recorded as part of traces.

Each report will be defined as a struct type, whose fields will list the details of a given action. These reports generally double as logging statements, which can be turned off and on with Tracer.SetShouldPrint.

The TracingServer will aggregate all recorded actions and write them out to a JSON file, which can be used both for grading and for debugging via external processing. Moreover, the tracing server generates a ShiViz-compatible log that can be used with ShiViz to visualize the execution of the system.

Installation

Make sure to run go get -u github.com/DistributedClocks/tracing manually. Otherwise, your local go.mod will remain pinned to an old hash of the tracing library.

Documentation

See https://godoc.org/github.com/DistributedClocks/tracing for API-level documentation.

Documentation

Overview

Package tracing provides a tracing library, in order to allow for precise automatic grading in CPSC 416. A trace provides a precise, ordered representation of what your assignment code is doing (well, what it says it's doing), which can be used to assess some things that are unclear from either unit testing or code inspection. These include correct concurrency management, as well as properly following any sequencing/causality rules required by the protocol you are implementing.

The tracing library is split into two parts: the tracing server TracingServer, and the tracing client Tracer. You should have one instance of Tracer per network node, and you should first get access to a Trace and then you can record an action by calling Trace.RecordAction(action). A Trace is a set of recorded actions that are associated with a unique trace ID. With traces, actions are recorded as part of traces.

Each report will be defined as a struct type, whose fields will list the details of a given action. These reports generally double as logging statements, which can be turned off and on with Tracer.SetShouldPrint.

The TracingServer will aggregate all recorded actions and write them out to a JSON file, which can be used both for grading and for debugging via external processing. Moreover, tracing server generates a ShiViz-compatible log that can be used with ShiViz to visualize the execution of the system.

Index

• type CreateTrace
• type GenerateTokenTrace
• type GetLastVCArg
• type GetLastVCResult
• type PrepareTokenTrace
• type RPCProvider
  • func (rp *RPCProvider) GetLastVC(arg GetLastVCArg, result *GetLastVCResult) error
  • func (rp *RPCProvider) RecordAction(arg RecordActionArg, result *RecordActionResult) error
• type ReceiveTokenTrace
• type RecordActionArg
• type RecordActionResult
• type Trace
  • func (trace *Trace) GenerateToken() TracingToken
  • func (trace *Trace) RecordAction(record interface{})
• type TraceRecord
• type Tracer
  • func NewTracer(config TracerConfig) *Tracer
  • func NewTracerFromFile(configFile string) *Tracer
  • func NewTracerNonFatal(config TracerConfig) *Tracer
  • func (tracer *Tracer) Close() error
  • func (tracer *Tracer) CreateTrace() *Trace
  • func (tracer *Tracer) ReceiveToken(token TracingToken) *Trace
  • func (tracer *Tracer) SetShouldPrint(shouldPrint bool)
• type TracerConfig
• type TracingServer
  • func NewTracingServer(config TracingServerConfig) *TracingServer
  • func NewTracingServerFromFile(configFile string) *TracingServer
  • func (tracingServer *TracingServer) Accept()
  • func (tracingServer *TracingServer) Close() error
  • func (tracingServer *TracingServer) Open() error
• type TracingServerConfig
• type TracingToken

Types

type CreateTrace

type CreateTrace struct{}

CreateTrace is an action that indicates creation of a trace.

type GenerateTokenTrace

type GenerateTokenTrace struct {
	Token TracingToken // the generated tracing token
}

GenerateTokenTrace is an action that indicates generation of a tracing token.

type GetLastVCArg

type GetLastVCArg string

type GetLastVCResult

type GetLastVCResult vclock.VClock

type PrepareTokenTrace

type PrepareTokenTrace struct{}

PrepareTokenTrace is an action that indicates start of generating a tracing token.

type RPCProvider

type RPCProvider struct {
	// contains filtered or unexported fields
}

RPCProvider is an abstraction to prevent registering non-RPC functions in the RPC server. RPCProvider should be used with rpc.Register, as an RPC target.

func (*RPCProvider) GetLastVC

func (rp *RPCProvider) GetLastVC(arg GetLastVCArg, result *GetLastVCResult) error

func (*RPCProvider) RecordAction

func (rp *RPCProvider) RecordAction(arg RecordActionArg, result *RecordActionResult) error

RecordAction writes the Record field of the argument as a JSON-encoded record, tagging the record with its type name. It also tags the result with TracerIdentity, which tracks the identity given to the tracer reporting the event.

type ReceiveTokenTrace

type ReceiveTokenTrace struct {
	Token TracingToken // the token that was received.
}

ReceiveTokenTrace is an action that indicated receiption of a token.

type RecordActionArg

type RecordActionArg struct {
	TracerIdentity string
	TraceID        uint64
	RecordName     string
	Record         []byte
	VectorClock    vclock.VClock
}

RecordActionArg indicates RecordAction RPC argument.

type RecordActionResult

type RecordActionResult struct{}

RecordActionResult indicates RecordActionRPC output.

type Trace

type Trace struct {
	ID     uint64
	Tracer *Tracer
}

Trace is a set of recorded actions that are associated with a unique trace ID. You must now first get access to a trace and then you can record an action (Trace.RecordAction(action)). There are two ways in which your code can get access to a trace instance: (1) create a new trace with a unique ID. In this case you should use

tracer.CreateTrace()

(2) receive an existing trace from another node. In this case you should

use tracer.ReceiveToken(token), which token is generated by a previous
trace.GenerateToken() call.

func (*Trace) GenerateToken

func (trace *Trace) GenerateToken() TracingToken

GenerateToken produces a fresh TracingToken, and records the event via RecordAction. This allows analysis of the resulting trace to correlate token generation and token reception.

func (*Trace) RecordAction

func (trace *Trace) RecordAction(record interface{})

RecordAction ensures that the record is recorded by the tracing server, and optionally logs the record's contents. record can be any struct value; its contents will be extracted via reflection. RecordAction implementation is thread-safe.

For example, consider (with tracer id "id"):

struct MyRecord { Foo string; Bar string }

and the call:

RecordAction(MyRecord{ Foo: "foo", Bar: "bar" })

This will result in a log (and relevant tracing data) that contains the following:

[TracerID] TraceID=ID MyRecord Foo="foo", Bar="bar"

type TraceRecord

type TraceRecord struct {
	TracerIdentity string
	TraceID        uint64
	Tag            string
	Body           json.RawMessage
	VectorClock    vclock.VClock
}

TraceRecord indicates the structure of each recorded trace

type Tracer

type Tracer struct {
	// contains filtered or unexported fields
}

Tracer is the tracing client.

func NewTracer

func NewTracer(config TracerConfig) *Tracer

NewTracer instantiates a fresh tracer client.

func NewTracerFromFile

func NewTracerFromFile(configFile string) *Tracer

NewTracerFromFile instantiates a fresh tracer client from a configuration file.

Configuration is loaded from the JSON-formatted configFile, which should specify:

• ServerAddress, an ip:port pair identifying a tracing server, as one might pass to rpc.Dial
• TracerIdentity, a unique string giving the tracer an identity that tracks which tracer reported which action
• Secret [TODO]

Note that each instance of Tracer is thread-safe.

func NewTracerNonFatal

func NewTracerNonFatal(config TracerConfig) *Tracer

NewTracer instantiates a fresh tracer client. Not calling Log.Fatal when rpc connection fails

func (*Tracer) Close

func (tracer *Tracer) Close() error

Close cleans up the connection to the tracing server. To allow for tracing long-running processes and Ctrl^C, this call is unnecessary, as there is no connection state. After this call, the use of any previously generated local Trace instances leads to undefined behavior.

func (*Tracer) CreateTrace

func (tracer *Tracer) CreateTrace() *Trace

CreateTrace creates a new trace object with a unique ID. Also, it records a CreateTrace action.

func (*Tracer) ReceiveToken

func (tracer *Tracer) ReceiveToken(token TracingToken) *Trace

ReceiveToken records the token by calling RecordAction with ReceiveTokenTrace.

func (*Tracer) SetShouldPrint

func (tracer *Tracer) SetShouldPrint(shouldPrint bool)

SetShouldPrint determines whether RecordAction should log the action being recorded as it sends the action to the tracing server. In other words, it indicates that the Tracer instance should log (print to stdout) the recorded actions or not. For more complex applications which have long, involved traces, it may be helpful to silence trace logging.

type TracerConfig

type TracerConfig struct {
	ServerAddress  string // address of the server to send traces to
	TracerIdentity string // a unique string identifying the tracer
	Secret         []byte // TODO
}

TracerConfig contains the necessary configuration options for a tracer.

type TracingServer

type TracingServer struct {
	Listener net.Listener

	Config *TracingServerConfig
	// contains filtered or unexported fields
}

TracingServer should be used with rpc.Register, as an RPC target.

func NewTracingServer

func NewTracingServer(config TracingServerConfig) *TracingServer

NewTracingServer instantiates a new tracing server.

func NewTracingServerFromFile

func NewTracingServerFromFile(configFile string) *TracingServer

NewTracingServerFromFile instantiates a new tracing server from a configuration file.

Configuration is loaded from the JSON-formatted configFile, whose fields correspond to the TracingServerConfig struct.

Note that each instance of Tracer is thread-safe.

Note also that this function does not actually set up any RPC/server binding, it handles everything up to that point (opening output files, setting up internals).

func (*TracingServer) Accept

func (tracingServer *TracingServer) Accept()

Accept accepts connections on the listener and serves requests for each incoming connection. Accept blocks until the listener returns a non-nil error. This implementation matches exactly the implementation of `rpc.Accept` from https://golang.org/src/net/rpc/server.go?s=18334:18380#L613, except it does not log the listner.Accept error.

func (*TracingServer) Close

func (tracingServer *TracingServer) Close() error

Close closes the related opened files and the RPC server.

func (*TracingServer) Open

func (tracingServer *TracingServer) Open() error

Open creates the related files for the tracing server and starts an RPC server on the specified address.

type TracingServerConfig

type TracingServerConfig struct {
	ServerBind       string // the ip:port pair to which the server should bind, as one might pass to net.Listen
	Secret           []byte
	OutputFile       string // the output filename, where the tracing records JSON will be written
	ShivizOutputFile string // the shiviz-compatible output filename
}

TracingServerConfig contains the necessary configuration options for a tracing server.

type TracingToken

type TracingToken []byte

TracingToken is an abstract token to be used when tracing message passing between network nodes.

A one-time-use token can be retrieved using Trace.GenerateToken, and the "reception" of that token can be recorded using Tracer.ReceiveToken.