moa

README

Moa

Emulation engine for Merge, including Link, Net and Wireless. Moa is a wired and wireless network emulator. It consists of two major components:

• Python-based Emulation Manager
• Emulation Engines

The Emulation Manager receives API instructions from the test infrastructure and manages those commands. Based on the commands, new emulation configurations may be created or new emulation instances may instantiated.

The Emulation Engines actually conduct the emulation under the control of the Emulation Manager. The manager can utilize arbitrary engines but currently only utilizes a Click-based emulation engine.

Moa Python Requirements:

• Currently only uses Python 2.7 (migration to 3 is planned)
• Dependencies:
  • grpcio
  • grpcio-tools
  • concurrent.futures
  • yaml
  • networkx
  • matplot-lib

Click Requirements:

• Moa will automatically install Click and DPDK packages
• Dependencies that need to be met:
  • libnuma-dev
  • libibverbs
  • rdmacore
  • lshw
  • lspci
  • zlib1g-dev
• Huge pages need to be configured. Moa will eventually do this itself. For now, a script like this must be run:

#!/bin/bash

for x in /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages /sys/devices/system/node/node?/hugepages/hugepages-2048kB/nr_hugepages
do
  echo 2048 > $x
done

[ ! -d /mnt/huge ] && mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge

Starting Moa

To start Moa, simple call:

sudo python Moa.py

Moa must be called as sudo (or setuid) to ensure that it can properly manage DPDK and Click

Controlling Moa through the API

Moa uses a gRPC API to receive instructions and configure or operate emulations. Almost all Moa commands operate on data structures which are defined below. Moa has three principle command phases:

• Emulation Node Information Reception
• Emulation Construction
• Emulation Operation

Moa must understand the environment it is operating on. Moa will attempt to learn this automatically by scraping various local resource, but the API can help Moa by providing it information directly. The commands that support this are:

• UpdateInterfaces - Add information to interfaces
  • a JSON blob which is a list of interface data structures.
• AddVXLANs - Add VXLAN information
  • a JSON blob which is a list of vxlan data structures.
• RemoveVXLANs - Remove VXLAN information
  • a JSON blob which is a list of vxlan data structures.

During emulation construction, Moa will receive emulation JSON blobs which will lead to the construction of the emulation configuration. The commands used for construction are:

• NewEmulation - Create a new emulation
  • an integer emulation ID (reference used for this emulation)
  • a String corresponding to the emulator to use ('click')
  • a JSON blob which is a list of emulation data structures
• AddToEmulation - Add more emulation objects to an emulation
  • an integer emulation ID
  • a JSON blobl which is a list of emulation data structures
• FinalizeEmulation - Tell Moa you're done constructing this emulation
  • an emulation ID.

Finally, Moa operational commands are those that start, stop, or update an existing emulation. Additionally, Moa can be shutdown.

• StartEmulation - Start an emulation
  • an integer emulation ID
• StopEmulation - Stop an emulation
  • an integer emulation ID
• UpdateEmulation (may not currently work)
• ShutdownMoa

Moa Data Structures

Below are how data structures are created for JSON. You can see some examples in test/link_configurations/simple_test.json test/mpl_configurations/link2.json test/mpl_configurations/wireless_mpl.json test/interface_configurations/iface_conf1_vxlans.json

MoaDataStructures.py has more info (generally for internal)
Moa Data Structures:

Link:  Dictionary
  'type':                   string, indicates this is a link
  'left_to_right':          dictionary, UniLink as defined below
  'right_to_left':          dictionary, UniLink as defined below
  'id':                     string, identifier for this link

UniLink:  Dictionary
  'type':                   string, indicates this is a unilink
  'inputs'                  list of strings, which will give/take packets. Can be an interface or a higher level abstraction (such as an mpl)
  'output':                 string, which will give/take packets. Can be an interface or a higher level abstraction
  'id':                     string, identifier for this link (including direction if necessary, should be unique)
  'args':                   dictionary, link properties such as bandwidth etc...  (defaults exist for everything)
       Current properties recognized are:
       'bandwidth'          string, number + units, example (10Mbps)
       'delay'              string, number + units, example (10ms)
       'loss'               string, number (0 - 1), example (0.1)
       'useCodel'           boolean
       

Vxlan:  Dictionary
  'type':                   string, indicates this is a vxlan
  'dst_ip':                 string, Destination IP of "other" host
  'dst_mac':                string, Destination MAC of "other" host
  'interface':              string, interface id attached to
  'id':                     string, VXLAN Id

Interface: Dictionary
  'type':                   string, indicates this is a interface
  'id':                     string, name of interface (as on the machine)
  'ip':                     string, interfaces IP address
  'prefix':                 string, interfaces prefix
  'mac':                    string, interfaces MAC address
  'pci':                    string, PCI address of this interface
  'vxlans':                 list of strings, vxlans attached to this interface. (optional)


MPL: Dictionary (Multipoint Link):
  'type':                   string, indicates this is a mpl
  'mpl_type':               string, MPL type ('broadcast, switched')
  'id':                     string, MPL id
  'members':                list of MPL members (string identifiers)
  'inputs':                 dictionary, keys are members in the above list, values are what should actually input to the mpl
                              example: Member might be 'vxlan121', {'vxlan121': 'link_121_in'} representing that vxlan121 will drop 
                              packets here through link_121_in
  'outputs':                dictionary, keys are members in the above list, values are where traffic should be dumped
                              example: Member might be 'vxlan121', {'vxlan121': 'link_121_out'} representing that vxlan121 will receive 
                              packets here through link_121_out
  'args':                   dictionary, MPL shaping parameters (for now there global to the MPL)

Adding New Elements

1. Add the new element type to the buildElements function in ClickBuilder.py
2. Add the new element type to the buildPipelines function in ClickBuilder.py
3. Add the new element type to the writeElements function in ClickBuilder.py
4. Add the new element type to the writePipelines function in ClickBuilder.py
5. Create a constructor for the new element similar to buildQueue in ClickBuilder.py
6. Create a primitive for the new element similar to createQueue in ClickPrimitives.py

Known Issues

Many issues can be found in the issue tracker, but the #1 known issue is if Moa dies for some reason, it looses management over its resident emulations. A storage and recovery strategy is being developed.

A second issue is to simplify the MPL structure

MAJOR ISSUE: ReachabilitySwitch is incorrectly configured by Moa, using the wrong MAC addresses. The correct MAC addresses are the ones AFTER VXLAN decap. Tofix.