README
¶
Orca
*Echolocation of device with static nodes and network latency.
Orca is a reflector/ping generator service that is intended to measure network latency of gRPC requests from mobile devices (laptops) to storage locations in home and work networks. Orca was built to get some baseline metrics for latencies in distributed storage systems and is purely experimental, not intended for large scale use.
Orca specifies two primary commands: reflect and generate. The reflector is a Protocol Buffers service that accepts echo requests and returns replies. The generator is a service that sends a request to all known reflectors on a routine interval and logs the latency in a SQLite database.
Installation
In order to install Orca and get it running on your system, you have two options: build from source or download a prebuilt binary that may or may not exist for your system.
To download the prebuilt binary, visit the Current Release on GitHub and download the executable for your system. Extract the archive to reveal the executable, and put it on your path, making sure it has executable permissions.
Note: currently only Linux x64 and OS X (darwin) x64 binaries are prebuilt. My preference for path location is $HOME/bin but make sure that location is in your path. Otherwise, /usr/local/bin is a good choice for moving the binary to.
To build from source, make sure you have your Go environment setup and run:
$ go get github.com/bbengfort/orca/...
You can then use go install to install the binary to $GOBIN as follows:
$ go install cmd/orca.go
To contribute or modify the orca command, you'll want to go the source route; otherwise if you are using Linux or OS X, I recommend going the binary route. If there is an architecture that you'd like prebuilt, please let me know in a GitHub issue.
Getting Started
Once installed, some configuration is necessary. This is the recommended configuration, though other configuration options are available and can be inspected via the orca help command.
-
Create the directory where configuration and data will reside:
$ mkdir ~/.orca -
Copy the example configuration file to the orca directory
$ cp fixtures/orca-config-example.yml ~/.orca/config.ymlNote that the first path refers to the fixtures directory inside of the GitHub repository. If you downloaded the binary package from the releases page, then the example configuration will be in the archive you downloaded.
-
Edit the configuration as needed, noting the defaults and comments in the example configuration file.
-
Create the orca SQLite database used to track meta information about reflection and generators.
$ orca createdb
At this point orca is configured to begin reflecting, however generators require an extra configuration step.
Relectors
Running orca as a reflector runs a gRPC service that listens for echo requests on the current IP address and port, increments the request sequence (to detect out of order messages), and responds with an echo reply. This process means that there is a small database transaction before reply, so latency will not be exactly the same as using the ping utility. If debug: true in the configuration, then the server will also log all incoming echo requests to stdout, which can be redirected to a log file if required.
Run the reflector as follows:
$ orca reflect
To run in the background on a constant server:
$ nohup orca reflect &
Upstart and LaunchAgent scripts for managing the background process are forthcoming, though I'd be happy to accept a pull request for them!
Generators
Generators require a bit more configuration, since you'll have to add all of the reflectors that you want the generator to ping to the database. Do this via the orca devices --add command:
$ orca devices --add
Enter device name []: rogue
Enter device IP address []: 1.2.3.4:3265
Enter device domain []:
Do this for as many devices as you'd like to ping on each interval. To see the devices already added use orca devices --list. Run the generator as follows:
$ orca generate
Similar to the reflector, you'll have to nohup and background this in order to ensure it always runs. LaunchAgent and Upstart scripts are coming soon. The generator waits until the interval has passed, loads up the list of devices to ping, and sends an echo request to them, recording the request (in the case of non-connectivity) and sequence number in the database. On receipt of the reply, it measures latency and stores the information in the database.
Location Servicesd
Orca can provide location services for mobile devices via the MaxMind GeoIP2 Precision City Service. In order to enable location services, you need to register for a MaxMind developer account and include your API user id and license key in the YAML configuration file. Because MaxMind is a paid service, location lookups are only made when the current IP address of the machine changes.
Note also that the granularity for this service is limited; for example, a GeoIP2 lookup from my office in the A.V. Williams Building of the University of Maryland yielded the following location via latitude and longitude:
This location is not centered on my office, the building, or even in the center of the university. The level of granularity probably differs based on the type of network you're connected to. If a higher level of granularity is required, then the use of GPS is recommended. Additional location inaccuracy can come about when tethering to a mobile phone. Use specified locations with care!
Acknowledgements
Orca is an open source project built to obtain metrics about mobile distributed systems and various latencies. If you'd like to contribute, I'd love some help, but no current plans are underway for future development.
Attribution
The photo used in this README, “Orca (Design Exploration)” by Alberto Cerriteño is used under a CC BY-NC-ND 2.0 creative commons license.
Documentation
¶
Overview ¶
Package orca provides the library for a systems experiment that measures latency and uptime of mobile nodes against fixed responder nodes.
Index ¶
- Constants
- Variables
- func Asset(name string) ([]byte, error)
- func AssetDir(name string) ([]string, error)
- func AssetInfo(name string) (os.FileInfo, error)
- func AssetNames() []string
- func ExternalIP() (string, error)
- func MustAsset(name string) []byte
- func ResolveAddr(addr string) (string, error)
- func RestoreAsset(dir, name string) error
- func RestoreAssets(dir, name string) error
- type App
- func (app *App) ConnectDB() error
- func (app *App) CreateDB() error
- func (app *App) Echo(ctx context.Context, in *echo.Request) (*echo.Reply, error)
- func (app *App) FetchDevices() (Devices, error)
- func (app *App) FetchDevicesExcept(device *Device) (Devices, error)
- func (app *App) Generate() error
- func (app *App) GetDB() *sql.DB
- func (app *App) GetDevice() *Device
- func (app *App) GetListenAddr() (string, error)
- func (app *App) Ping(device *Device) error
- func (app *App) Reflect() error
- func (app *App) SendPing(device *Device) (*echo.Reply, error)
- func (app *App) SetLocation(loc *Location, save bool) error
- func (app *App) SyncLocation() error
- type Config
- type Device
- func (d *Device) Delete(db *sql.DB) (bool, error)
- func (d *Device) Echo() *echo.Device
- func (d *Device) Exists(id int64, db *sql.DB) (bool, error)
- func (d *Device) Get(id int64, db *sql.DB) error
- func (d *Device) GetByName(name string, db *sql.DB) error
- func (d *Device) Save(db *sql.DB) (bool, error)
- func (d *Device) String() string
- type Devices
- type Location
- func (loc *Location) Delete(db *sql.DB) (bool, error)
- func (loc *Location) Exists(id int64, db *sql.DB) (bool, error)
- func (loc *Location) Get(id int64, db *sql.DB) error
- func (loc *Location) IPExists(db *sql.DB) error
- func (loc *Location) Save(db *sql.DB) (bool, error)
- func (loc *Location) String() string
- type MaxMindClient
- type MaxMindConfig
- type Model
- type ModelMeta
- type Ping
Constants ¶
const DefaultPort = 3265
DefaultPort is used to compute the TCP address in the absense of one.
const Timeout = time.Duration(30) * time.Second
Timeout is the amount of time sonar will wait for a reply
const Version = "0.1"
Version specifies the current version of the Orca library.
Variables ¶
var ConfigPath = []string{ "/etc/orca.yml", filepath.Join(getUserDir(), ".orca", "config.yml"), filepath.Join(getCwd(), "orca.yml"), }
ConfigPath specifies the locations to look up configurations
Functions ¶
func Asset ¶
Asset loads and returns the asset for the given name. It returns an error if the asset could not be found or could not be loaded.
func AssetDir ¶
AssetDir returns the file names below a certain directory embedded in the file by go-bindata. For example if you run go-bindata on data/... and data contains the following hierarchy:
data/
foo.txt
img/
a.png
b.png
then AssetDir("data") would return []string{"foo.txt", "img"} AssetDir("data/img") would return []string{"a.png", "b.png"} AssetDir("foo.txt") and AssetDir("notexist") would return an error AssetDir("") will return []string{"data"}.
func AssetInfo ¶
AssetInfo loads and returns the asset info for the given name. It returns an error if the asset could not be found or could not be loaded.
func ExternalIP ¶
ExternalIP looks up an the first available external IP address.
func MustAsset ¶
MustAsset is like Asset but panics when Asset would return an error. It simplifies safe initialization of global variables.
func ResolveAddr ¶
ResolveAddr accepts an address as a string and if the IP address is missing it replaces it with the result from ExternalIP then returns the addr string. Likewise if the Port is missing, it returns an address with the DefaultPort appended to the address string. This function hopefully will make the network configuration easy for new scio apps.
func RestoreAsset ¶
RestoreAsset restores an asset under the given directory
func RestoreAssets ¶
RestoreAssets restores an asset under the given directory recursively
Types ¶
type App ¶
type App struct {
Config *Config // The configuration loaded from the YAML file
GeoIP *MaxMindClient // GeoIP Lookup API client
Device *Device // Descriptor for the device in the database
Location *Location // Current location of the application
ExternalIP string // Current external IP address of the machine
// contains filtered or unexported fields
}
App is the primary application that maintains references to the config and device details as well as initializes the environment and runs the reflect and generate commands.
func (*App) CreateDB ¶
CreateDB executes the create table statements in the schema.sql stored as binary data in the application (as well as any alter table statements).
func (*App) FetchDevices ¶
FetchDevices returns a collection of devices, ordered by the created timestamp. This function expects you to limit the size of the collection by specifying the maximum number of nodes to return in the Devices list.
func (*App) FetchDevicesExcept ¶
FetchDevicesExcept the specified device by excluding the device ID from the SQL query. Allows the creation of a device list except for the local device.
func (*App) GetDB ¶
GetDB returns the database connectio object from the app. Use with care, I didn't want to expose this outside the app ...
func (*App) GetDevice ¶
GetDevice returns the device on the app, if it is nil, it performs a database query for the device based on the Config name; if there is nothing in the database, it inserts the record from the configuration.
func (*App) GetListenAddr ¶
GetListenAddr looks up the IP address on the config or gets an external IP This is meant to be used by reflect mode to respond to echo requests.
func (*App) SetLocation ¶
SetLocation is a wrapper method that sets the location on the app struct, but also does a check about whether or not to save it to the database.
func (*App) SyncLocation ¶
SyncLocation checks the external IP address against the current IP address, if they're different then it performs another location lookup to track mobility in the generator application, but does not perform GeoIP lookups if they're not necessary (to save bandwidth and cost).
type Config ¶
type Config struct {
Debug bool `yaml:"debug"` // Print out log messages or not
Name string `yaml:"name"` // The name of hte local device
Addr string `yaml:"addr"` // The listen address of the local device
Domain string `yaml:"domain"` // The domain name of the local device
Interval int64 `yaml:"interval"` // The wait in seconds between pings to reflectors
DBPath string `yaml:"dbpath"` // The path to the SQLite3 database
MaxMind *MaxMindConfig
}
Config is read from a YAML file and defines the current configuration of the project and can be exported as such.
type Device ¶
type Device struct {
Name string // Hostname of the device
IPAddr string // IP Address of the device
Domain string // Domain name of the device
Sequence int64 // The response/reply counter for a device
ModelMeta
// contains filtered or unexported fields
}
Device is an entity that represents nodes in the network that can be pinged. Device objects are stored in the devices table.
func (*Device) Delete ¶
Delete a device from the database. Returns true if the number of rows affected is 1 or false otherwise.
func (*Device) Save ¶
Save a device struct to the database. This function checks if the device has an ID or not. If it does, it will execute a SQL UPDATE, otherwise it will execute a SQL INSERT. Returns a boolean if the device was inserted. This method handles setting the created and updated timestamps as well.
type Devices ¶
type Devices []*Device
Devices is a collection of Device objects loaded from the database.
type Location ¶
type Location struct {
IPAddr string // IP Address associated with the location
Latitude float64 // Decimal based latitude
Longitude float64 // Decimal based logitude
City string // City returned by MaxMind for the IP address
PostCode string // Postal code returned by MaxMind for the IP address
Country string // Country returned by MaxMind for the IP address
Organization string // Organization associated with the given domain (ISP)
Domain string // Domain associated with the IP address (ISP)
Note string // Any additional annotations by the user
ModelMeta
}
Location is a geographic record that is usually associated with an IP address via the geoip lookup service but could also come from GPS.
func (*Location) Delete ¶
Delete a location from the database. Returns true if the number of rows affected is 1 or false otherwise.
func (*Location) IPExists ¶
IPExists sets the ID on the location if the location's IP address is already in the database, otherwise it sets it to zero.
func (*Location) Save ¶
Save a location struct to the database. This function checks if the location has an ID or not. If it does, it will execute a SQL UPDATE, otherwise it will execute a SQL INSERT. Returns a boolean if the location was inserted. This method handles setting the meta timestamps as well.
type MaxMindClient ¶
type MaxMindClient struct {
// contains filtered or unexported fields
}
MaxMindClient is a utility for making geoip requests to MaxMind services.
func NewMaxMindClient ¶
func NewMaxMindClient(userid string, license string) *MaxMindClient
NewMaxMindClient initializes the client struct with required info and also initializes the http client to make requests with HTTP Authentication.
func (*MaxMindClient) GeoIPLookup ¶
func (mm *MaxMindClient) GeoIPLookup(ipaddr string) (map[string]interface{}, error)
GeoIPLookup fetches the raw JSON data from the MaxMind API for the given IP Address. Note that 'me' is a special input to autolookup the IP address from the request, and empty strings are converted to 'me'. This function returns the parsed JSON data as a map object.
func (*MaxMindClient) GetCurrentLocation ¶
func (mm *MaxMindClient) GetCurrentLocation() (*Location, error)
GetCurrentLocation returns the current location using the special 'me' and returns a Location struct ready to be saved to the database.
func (*MaxMindClient) NewRequest ¶
func (mm *MaxMindClient) NewRequest(ipaddr string) (*http.Request, error)
NewRequest creates an authenticated request to look up the given IP address NOTE that you can also use the string 'me' which performs auto lookup, if an empty string is passed to this function, then 'me' is used.
type MaxMindConfig ¶
type MaxMindConfig struct {
Username string // MaxMind User ID
License string // MaxMind License Key
}
MaxMindConfig specifies the MaxMind API credentials
type Model ¶
type Model interface {
Get(id int64, db *sql.DB) error // Populate model fields from the database
Save(db *sql.DB) (bool, error) // Insert or update the model
Delete(db *sql.DB) (bool, error) // Delete the model from the database
Exists(id int64, db *sql.DB) (bool, error) // Determine if the model exists by ID
}
Model specifies types that can interact with the database.
type ModelMeta ¶
type ModelMeta struct {
ID int64 // Unique ID of the model
Created time.Time // Datetime the model was added to the database
Updated time.Time // Datetime the model was updated in the database
}
ModelMeta specifies the fields that all models should have via embedding.
type Ping ¶
type Ping struct {
ID int64 // Unique ID of the record
Source *Device // Source device of the ping (always the local node)
Target *Device // Target device that the ping was sent to
Location *Location // The location of the source at the time of the ping
Request int64 // Request sequence number for the source/target pair
Response int64 // Response sequence number for the target/source pair
Sent time.Time // The time that the ping was sent
Recv time.Time // The time that the ping was received
Latency sql.NullFloat64 // The latency in milliseconds of the ping
}
Ping is a timeseries record of latency requests reflected from echo servers.
func (*Ping) Delete ¶
Delete a ping from the database. Returns true if the number of rows affected is 1 or false otherwise.
func (*Ping) Save ¶
Save a ping struct to the database. This function checks if the ping has an ID or not. If it does, it will execute a SQL UPDATE, otherwise it will execute a SQL INSERT. Returns a boolean if the device was inserted. This method handles setting the created and updated timestamps as well.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
This command implements the Mora console utility that can run one of two background processes: reflectors and generators.
|
This command implements the Mora console utility that can run one of two background processes: reflectors and generators. |
|
Package echo is a generated protocol buffer package.
|
Package echo is a generated protocol buffer package. |