gostwire

package module
v2.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 25, 2024 License: MIT Imports: 8 Imported by: 0

README

Siemens Industrial Edge Edgeshark

Ghostwire Virtual Communication Discovery

PkgGoDev GitHub build and test goroutines file descriptors Go Report Card Coverage

G(h)ostwire discovers the virtual (or not) network configuration inside Linux hosts – and can be deployed as a REST service or consumed as a Go module. It comes with its unique container engine discovery that auto-discovers multiple container engines in hosts, including engines inside containers. Ghostwire not only understands containers, but also Composer projects and Kubernetes pods.

Currently supported container engines are:

Ghostwire is also Kubernetes-aware and even understands that KinD simulates Kubernetes nodes in Docker containers.

Quick Start

We provide multi-architecture Docker images for linux/amd64 and linux/arm64. First, ensure that you have the Docker compose plugin v2 installed. For Debian users it is strongly recommended to install docker-ce instead of docker.io packages, as these are updated on a regular basis.

Make sure you have a Linux kernel of at least version 4.11 installed, however we highly recommend at least kernel version 5.6 or later.

wget -q --no-cache -O - \
  https://github.com/siemens/ghostwire/raw/main/deployments/wget/docker-compose.yaml \
  | docker compose -f - up

Finally, visit http://localhost:5000 and start looking around the virtual container networking, IP and DNS configuration, open and forwarded ports, and much more.

[!WARNING] This quick start deployment will expose TCP port 5000 also to clients external to your host. Make sure to have proper network protection in place.

Eye Candy!

Lots of Virtual "Wires"

A slightly busy Industrial Edge host:

Edgeshark wiring screenshot

IP & DNS Configuration

...lots of gory configuration details:

Edgeshark details screenshot

For instance: container/process capabilities, with Docker non-standard capabilities being highlighted. Open and connected ports, and forwarded ports, all with the containers and processes serving them. Other Docker containers addressable from your container using DNS names for Docker services and containers.

See the Current System State

Information is gathered as much as possible from Linux APIs in order to show the current effective state, instead of potentially invalid or stale engineering configuration. Container engines are only queried in order to understand which processes and network elements have a user-space meaning beyond the kernel-space view.

Teaming up Ghostwire with Packetflix makes Wireshark container-aware: a simple click on one of the "shark fin" buttons starts a Wireshark session, directly capturing network traffic inside a container (or even KinD pod).

Edgeshark Project Context

ghostwire/v2 is part of the "Edgeshark" project that consist of several repositories:

Documentation

For deployment, usage and development information, please see the accompanying documentation in docs/. The most convenient way to view this documentation is with the help of docsify:

make docsify

Then navigate to http://localhost:3300 to read on.

Build & Deploy

Building the Packetflix service requires the Go toolchain, make, a C compiler (used by cgeo), and finally Docker installed.

The preferred build method is as a containerized service. For development, a standalone binary as well as the webui assets can be built outside of any container.

IMPORTANT: the Ghostwire service exposes (read-only) host information on port 5000. Never expose this services without taking additional security measures.

Container Service

To build and deploy the Gostwire containerized service and exposing it at port 5000 of the host (requires the build(x) plugin to be installed, which is GA on Linux for some time now):

make deploy

Alternatively, you can build and deploy a pprof-enabled service, which enables pprof information at :5000/debug/pprof.

make pprofdeploy

For more information, please refer to the (docsified) documentation in docs/.

Standalone Service Binary
  • ensure that Go 1.20 (or later) is installed.
  • ensure that nodejs 16+, npm, and yarn (npm install --global yarn) are installed.
make yarnsetup # if not done already so
make build-embedded
./gostwire --debug

VSCode Tasks

The included gostwire.code-workspace defines the following tasks:

  • View Go module documentation task: installs pkgsite, if not done already so, then starts pkgsite and opens VSCode's integrated ("simple") browser to show the csharg documentation.
Aux Tasks
  • pksite service: auxilliary task to run pkgsite as a background service using scripts/pkgsite.sh. The script leverages browser-sync and nodemon to hot reload the Go module documentation on changes; many thanks to @mdaverde's Build your Golang package docs locally for paving the way. scripts/pkgsite.sh adds automatic installation of pkgsite, as well as the browser-sync and nodemon npm packages for the local user.
  • view pkgsite: auxilliary task to open the VSCode-integrated "simple" browser and pass it the local URL to open in order to show the module documentation rendered by pkgsite. This requires a detour via a task input with ID "pkgsite".

Make Targets

  • make: lists all targets.
  • make dist: builds snapshot packages and archives of the csharg CLI binary.
  • make pkgsite: installs x/pkgsite, as well as the browser-sync and nodemon npm packages first, if not already done so. Then runs the pkgsite and hot reloads it whenever the documentation changes.
  • make report: installs @gojp/goreportcard if not yet done so and then runs it on the code base.
  • make vuln: install (or updates) govuln and then checks the Go sources.

Contributing

Please see CONTRIBUTING.md.

(c) Siemens AG 2023

SPDX-License-Identifier: MIT

Documentation

Overview

Package gostwire provides discovery of topology and configuration of virtual networks in Linux hosts.

Discovery

The Discover function expects a context.Context as well as a so-called “containerizer” implementing the containerizer.Containerizer interface. Gostwire's turtlefinder package offers a suitable implementation, accessible as turtlefinder.New(). This implementation offers automatic detection of container engines without any need for API endpoint configuration. The containerizer needs to be allocated only once and is safe for use in concurrent discoveries. 

containerizer := turtlefinder.New()
allnetns := gostwire.Discover(req.Context(), containerizer)

For more information about Gostwire's information model, please see the network package.

Linux Capabilities

Please note that the auto-detection of container engines as well as a complete discovery require the calling process and its OS-level threads to possess sufficient capabilities. In particular:

  • CAP_SYS_ADMIN
  • CAP_SYS_CHROOT
  • CAP_SYS_PTRACE
  • CAP_DAC_READ_SEARCH
  • CAP_DAC_OVERRIDE
Example (Discovery) 
enginectx, enginecancel := context.WithCancel(context.Background())
defer enginecancel()
containerizer := turtlefinder.New(func() context.Context { return enginectx })
defer containerizer.Close()
allnetns := Discover(context.Background(), containerizer, nil)
fmt.Printf("%d network stacks found\n", len(allnetns.Netns))

Output:

Index

Examples

Constants

View Source 
const CaptureEnableHeader = "Enable-Monolith"

CaptureEnableHeader tells the Ghostwire service to serve its SPA user interface with capture button enabled.

View Source 
const SemVersion = "2.4.0-1-g7c495d8"

SemVersion is the semantic version string of the ghostwire module.

Variables

This section is empty.

Functions

This section is empty.

Types

type DiscoveryResult 

type DiscoveryResult struct {
	Netns   network.NetworkNamespaces // network discovery
	Lxkns   *lxknsdiscover.Result     // namespaces, process, and containers discovery
	Engines []*model.ContainerEngine  // discovered container engines, even if without workload
}

DiscoveryResult contains the network topology and configuration discovery results, as well as the Linux-kernel namespace, process, and container discovery results.

func Discover 

func Discover(ctx context.Context, cizer containerizer.Containerizer, labels map[string]string) DiscoveryResult

Discover returns the discovered network stacks, virtual network topology, and network-related configuration. Labels optionally control certain aspects of "decorating" (that is, enriching) the discovery results for some decorator plugins supporting labels (such as the ieappicon decorator plugin).

Source Files

View all Source files

Directories

Path Synopsis
api
v1
Package v1 implements the Ghostwire v1 REST API discovery JSON results.
 Package v1 implements the Ghostwire v1 REST API discovery JSON results.
v2
Package v2 implements the Gostwire v2 API.
 Package v2 implements the Gostwire v2 API.
cmd
gostdump
Package gostdump dumps the results of a discovery as JSON in REST API v1 format.
 Package gostdump dumps the results of a discovery as JSON in REST API v1 format.
gostwire
Package gostwire implements the Gostwire service.
 Package gostwire implements the Gostwire service.
internal/wsconn
Wraps a server-side websocket connection with its own human-readable unique ID.
 Wraps a server-side websocket connection with its own human-readable unique ID.
lsallnifs
Package lsallnifs pretty-prints the network discovery results to stdout.
 Package lsallnifs pretty-prints the network discovery results to stdout.
Package decorator defines the plugin interface for Gostwire decorators that post-process the discovered Gostwire information model and add usefully information.
 Package decorator defines the plugin interface for Gostwire decorators that post-process the discovered Gostwire information model and add usefully information.
all
Package all pulls in all Gostwire decorators (plugins).
 Package all pulls in all Gostwire decorators (plugins).
dockernet
Package dockernet implements a Gostwire decorator that discovers Docker-managed networks and then decorates their corresponding Linux-kernel network interfaces.
 Package dockernet implements a Gostwire decorator that discovers Docker-managed networks and then decorates their corresponding Linux-kernel network interfaces.
dockerproxy
Package dockerproxy implements port forwarding detection based on Docker's docker-proxy processes.
 Package dockerproxy implements port forwarding detection based on Docker's docker-proxy processes.
ieappicon
Package ieappicon implements a Gostwire decorator for adorning containers belonging to Industrial Edge app composer projects with their app icons.
 Package ieappicon implements a Gostwire decorator for adorning containers belonging to Industrial Edge app composer projects with their app icons.
nerdctlnet
Package nerdctlnet implements a Gostwire decorator that discovers nerdctl-managed CNI networks and then decorates their corresponding network interfaces.
 Package nerdctlnet implements a Gostwire decorator that discovers nerdctl-managed CNI networks and then decorates their corresponding network interfaces.
podmannet
Package podmannet implements a Gostwire decorator that discovers podman (v4+) managed networks and then decorates their corresponding Linux-kernel network interfaces.
 Package podmannet implements a Gostwire decorator that discovers podman (v4+) managed networks and then decorates their corresponding Linux-kernel network interfaces.
internal
discover
gen/version
Package metadata defines an extensible metadata mechanism that allows to augment Gostwire discoveries and discovery service results, using plugins.
 Package metadata defines an extensible metadata mechanism that allows to augment Gostwire discoveries and discovery service results, using plugins.
all
Package all pulls in all Ghostwire metadata plugins.
 Package all pulls in all Ghostwire metadata plugins.
cpus
Package cpus provides a metadata plugin returning information about the CPUs in the system that are currently “online”.
 Package cpus provides a metadata plugin returning information about the CPUs in the system that are currently “online”.
engines
Package engines implements a metadata plugin returning information about the container engines for which containers were discovered.
 Package engines implements a metadata plugin returning information about the container engines for which containers were discovered.
host
Package host implements a metadata plugin returning information about the host system the discovery was run on.
 Package host implements a metadata plugin returning information about the host system the discovery was run on.
iecore
Package iecore implements a metadata plugin that returns information about the Industrial Edge (“core”) runtime container, if present.
 Package iecore implements a metadata plugin that returns information about the Industrial Edge (“core”) runtime container, if present.
Package mobydig implements deriving the DNS names of Docker services and containers and then resolving and pinging them, based on the discovered network topology, as well as Docker containers and Docker networks.
 Package mobydig implements deriving the DNS names of Docker services and containers and then resolving and pinging them, based on the discovered network topology, as well as Docker containers and Docker networks.
Package network defines Gostwire's virtual network configuration and topology model and implements the discovery.
 Package network defines Gostwire's virtual network configuration and topology model and implements the discovery.
portfwd
Package portfwd defines a plugin interface for detecting forwarded ports from nftables.
 Package portfwd defines a plugin interface for detecting forwarded ports from nftables.
portfwd/all
Package all pulls in all Gostwire port forwarding detectors (plugins).
 Package all pulls in all Gostwire port forwarding detectors (plugins).
portfwd/docker
Package docker implements port forwarding detection for iptables (nft) rules managed by the Docker daemon.
 Package docker implements port forwarding detection for iptables (nft) rules managed by the Docker daemon.
portfwd/kubeproxy
Package kubeproxy implements port forwarding detection for iptables (nft) rules managed by Kubernete's kube-proxy.
 Package kubeproxy implements port forwarding detection for iptables (nft) rules managed by Kubernete's kube-proxy.
portfwd/nftget
Package nftget help with conveniently retrieving information from nftable expressions.
 Package nftget help with conveniently retrieving information from nftable expressions.
test
nerdctl

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.