gostwire

package module
v2.1.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 23, 2023 License: MIT Imports: 6 Imported by: 0

README

Siemens Industrial Edge Edgeshark

Ghostwire Virtual Communication Discovery

PkgGoDev GitHub build and test goroutines file descriptors Go Report Card Coverage

G(h)ostwire discovers the virtual (or not) network configuration inside Linux hosts – and can be deployed as a REST service or consumed as a Go module. It comes with its unique container engine discovery that auto-discovers multiple container engines in hosts, including engines inside containers. Currently supported container engines are containerd, Docker, and optionally podman. Ghostwire is Kubernetes-aware and even understands that KinD simulates Kubernetes nodes in Docker containers.

Quick Start

We provide multi-architecture Docker images for linux/amd64 and linux/arm64. First, ensure that you have the Docker compose plugin v2 installed. For Debian users it is strongly recommended to install docker-ce instead of docker.io packages, as these are updated on a regular basis.

wget -q --no-cache -O - \
  https://github.com/siemens/ghostwire/raw/main/deployments/wget/docker-compose.yaml \
  | docker compose -f - up

Finally, visit http://localhost:5000 and start looking around Linux kernel namespaces, as well as mount points with their hierarchies.

⚠ This quick start deployment will expose TCP port 5000 also to clients external to your host. Make sure to have proper network protection in place.

Eye Candy!

Virtual "Wires"

A slightly busy Industrial Edge host:

Edgeshark wiring screenshot

IP & DNS Configuration

...and then the details:

Edgeshark details screenshot

Details, details, details ... such as capabilities (with non-standard capabilities being highlighted). Open and connected ports, and forwarded ports, all with the containers and processes serving them. Other Docker containers addressable from your container using DNS names for Docker services and containers.

Real System State, Not Good Intention Configuration Data

Information is gathered as much as possible from Linux APIs in order to show the current effective state, instead of potentially invalid or stale engineering configuration. Container engines are only queried in order to understand which processes and network elements have a user-space meaning beyond the kernel-space view.

Teaming up Ghostwire with Packetflix makes Wireshark container-aware: a simple click on one of the "shark fin" buttons starts a Wireshark session, directly capturing network traffic inside a container (or even KinD pod).

Edgeshark Project Context

ghostwire/v2 is part of the "Edgeshark" project that consist of several repositories:

Documentation

For deployment, usage and development information, please see the accompanying documentation in docs/. The most convenient way to view this documentation is with the help of docsify:

make docsify

Then navigate to http://localhost:3300 to read on.

Build & Deploy

Building the Packetflix service requires the Go toolchain, make, a C compiler (used by cgeo), and finally Docker installed.

The preferred build method is as a containerized service. For development, a standalone binary as well as the webui assets can be built outside of any container.

IMPORTANT: the Ghostwire service exposes (read-only) host information on port 5000. Never expose this services without taking additional security measures.

Container Service

To build and deploy the Gostwire containerized service and exposing it at port 5000 of the host (requires the build(x) plugin to be installed, which is GA on Linux for some time now):

make deploy

Alternatively, you can build and deploy a pprof-enabled service, which enables pprof information at :5000/debug/pprof.

make pprofdeploy

For more information, please refer to the (docsified) documentation in docs/.

Standalone Service Binary
  • ensure that Go 1.20 (or later) is installed.
  • ensure that nodejs 16+, npm, and yarn (npm install --global yarn) are installed.
# build gostwire service binary.
make build
# build static webui assets that the service will serve.
(cd webui && yarn install && yarn build)
# start the gostwire service, exposed at port 5000.
./gostwire --debug

VSCode Tasks

The included gostwire.code-workspace defines the following tasks:

  • View Go module documentation task: installs pkgsite, if not done already so, then starts pkgsite and opens VSCode's integrated ("simple") browser to show the csharg documentation.
Aux Tasks
  • pksite service: auxilliary task to run pkgsite as a background service using scripts/pkgsite.sh. The script leverages browser-sync and nodemon to hot reload the Go module documentation on changes; many thanks to @mdaverde's Build your Golang package docs locally for paving the way. scripts/pkgsite.sh adds automatic installation of pkgsite, as well as the browser-sync and nodemon npm packages for the local user.
  • view pkgsite: auxilliary task to open the VSCode-integrated "simple" browser and pass it the local URL to open in order to show the module documentation rendered by pkgsite. This requires a detour via a task input with ID "pkgsite".

Make Targets

  • make: lists all targets.
  • make dist: builds snapshot packages and archives of the csharg CLI binary.
  • make pkgsite: installs x/pkgsite, as well as the browser-sync and nodemon npm packages first, if not already done so. Then runs the pkgsite and hot reloads it whenever the documentation changes.
  • make report: installs @gojp/goreportcard if not yet done so and then runs it on the code base.
  • make vuln: install (or updates) govuln and then checks the Go sources.

Contributing

Please see CONTRIBUTING.md.

(c) Siemens AG 2023

SPDX-License-Identifier: MIT

Documentation

Overview

Package gostwire provides discovery of topology and configuration of virtual networks in Linux hosts.

Discovery

The Discover function expects a context.Context as well as a so-called “containerizer” implementing the containerizer.Containerizer interface. Gostwire's turtlefinder package offers a suitable implementation, accessible as turtlefinder.New(). This implementation offers automatic detection of container engines without any need for API endpoint configuration. The containerizer needs to be allocated only once and is safe for use in concurrent discoveries. 

containerizer := turtlefinder.New()
allnetns := gostwire.Discover(req.Context(), containerizer)

For more information about Gostwire's information model, please see the network package.

Linux Capabilities

Please note that the auto-detection of container engines as well as a complete discovery require the calling process and its OS-level threads to possess sufficient capabilities. In particular:

  • CAP_SYS_ADMIN
  • CAP_SYS_CHROOT
  • CAP_SYS_PTRACE
  • CAP_DAC_READ_SEARCH
  • CAP_DAC_OVERRIDE
Example (Discovery) 
enginectx, enginecancel := context.WithCancel(context.Background())
defer enginecancel()
containerizer := turtlefinder.New(func() context.Context { return enginectx })
defer containerizer.Close()
allnetns := Discover(context.Background(), containerizer, nil)
fmt.Printf("%d network stacks found\n", len(allnetns.Netns))

Output:

Index

Examples

Constants

View Source 
const CaptureEnableHeader = "Enable-Monolith"

CaptureEnableHeader tells the Ghostwire service to serve its SPA user interface with capture button enabled.

View Source 
const SemVersion = "0.9.1-4-g368ef35"

SemVersion is the semantic version string of the ghostwire module.

Variables

This section is empty.

Functions

This section is empty.

Types

type DiscoveryResult 

type DiscoveryResult struct {
	Netns network.NetworkNamespaces // network discovery
	Lxkns *lxknsdiscover.Result     // namespaces, process, and containers discovery
}

DiscoveryResult contains the network topology and configuration discovery results, as well as the Linux-kernel namespace, process, and container discovery results.

func Discover 

func Discover(ctx context.Context, cizer containerizer.Containerizer, labels map[string]string) DiscoveryResult

Discover returns the discovered network stacks, virtual network topology, and network-related configuration. Labels optionally control certain aspects of "decorating" (that is, enriching) the discovery results for some decorator plugins supporting labels (such as the ieappicon decorator plugin).

Source Files

View all Source files

Directories

Path Synopsis
api
v1
Package v1 implements the Ghostwire v1 REST API discovery JSON results.
 Package v1 implements the Ghostwire v1 REST API discovery JSON results.
v2
Package v2 implements the Gostwire v2 API.
 Package v2 implements the Gostwire v2 API.
cmd
gostdump
Package gostdump dumps the results of a discovery as JSON in REST API v1 format.
 Package gostdump dumps the results of a discovery as JSON in REST API v1 format.
gostwire
Package gostwire implements the Gostwire service.
 Package gostwire implements the Gostwire service.
internal/wsconn
lsallnifs
Package lsallnifs pretty-prints the network discovery results to stdout.
 Package lsallnifs pretty-prints the network discovery results to stdout.
Package decorator defines the plugin interface for Gostwire decorators that post-process the discovered Gostwire information model and add usefully information.
 Package decorator defines the plugin interface for Gostwire decorators that post-process the discovered Gostwire information model and add usefully information.
all
Package all pulls in all Gostwire decorators (plugins).
 Package all pulls in all Gostwire decorators (plugins).
dockernet
Package dockernet implements a Gostwire decorator that discovers Docker-managed networks and then decorates their corresponding Linux-kernel network interfaces.
 Package dockernet implements a Gostwire decorator that discovers Docker-managed networks and then decorates their corresponding Linux-kernel network interfaces.
dockerproxy
Package dockerproxy implements port forwarding detection based on Docker's docker-proxy processes.
 Package dockerproxy implements port forwarding detection based on Docker's docker-proxy processes.
ieappicon
Package ieappicon implements a Gostwire decorator for adorning containers belonging to Industrial Edge app composer projects with their app icons.
 Package ieappicon implements a Gostwire decorator for adorning containers belonging to Industrial Edge app composer projects with their app icons.
nerdctlnet
Package nerdctlnet implements a Gostwire decorator that discovers nerdctl-managed CNI networks and then decorates their corresponding network interfaces.
 Package nerdctlnet implements a Gostwire decorator that discovers nerdctl-managed CNI networks and then decorates their corresponding network interfaces.
internal
discover
gen/version
Package metadata defines an extensible metadata mechanism that allows to augment Gostwire discoveries and discovery service results, using plugins.
 Package metadata defines an extensible metadata mechanism that allows to augment Gostwire discoveries and discovery service results, using plugins.
all
Package all pulls in all Ghostwire metadata plugins.
 Package all pulls in all Ghostwire metadata plugins.
engines
Package engines implements a metadata plugin returning information about the container engines for which containers were discovered.
 Package engines implements a metadata plugin returning information about the container engines for which containers were discovered.
host
Package host implements a metadata plugin returning information about the host system the discovery was run on.
 Package host implements a metadata plugin returning information about the host system the discovery was run on.
iecore
Package iecore implements a metadata plugin that returns information about the Industrial Edge (“core”) runtime container, if present.
 Package iecore implements a metadata plugin that returns information about the Industrial Edge (“core”) runtime container, if present.
Package mobydig implements deriving the DNS names of Docker services and containers and then resolving and pinging them, based on the discovered network topology, as well as Docker containers and Docker networks.
 Package mobydig implements deriving the DNS names of Docker services and containers and then resolving and pinging them, based on the discovered network topology, as well as Docker containers and Docker networks.
Package network defines Gostwire's virtual network configuration and topology model and implements the discovery.
 Package network defines Gostwire's virtual network configuration and topology model and implements the discovery.
test
nerdctl
Package turtlefinder provides a Containerizer that auto-detects container engines and their containers.
 Package turtlefinder provides a Containerizer that auto-detects container engines and their containers.
detector
Package detector defines the plugin interface between the TurtleFinder and its container engine detector plugins.
 Package detector defines the plugin interface between the TurtleFinder and its container engine detector plugins.
detector/all
Package all pulls in all Gostwire engine detectors.
 Package all pulls in all Gostwire engine detectors.
detector/containerd
Package containerd implements the engine detector for containerd processes.
 Package containerd implements the engine detector for containerd processes.
detector/moby
Package moby implements the engine detector for Docker “dockerd” processes.
 Package moby implements the engine detector for Docker “dockerd” processes.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.