gate

README

Gate

Run untrusted code from anonymous sources. Instead of sending messages composed of passive data, send programs which can react to their environment. Migrate or duplicate running applications across hosts and computer architectures. See Introduction to Gate.

• License: 3-clause BSD
• Author: Timo Savola timo.savola@iki.fi

Foundations

WebAssembly is the interchange format of the user programs. However, the APIs are different from the browsers' usual WebAssembly environments. See low-level C API or the higher-level Rust crate for details.

The sandboxing and containerization features of the Linux kernel provide layers of security in addition to WebAssembly. See Security for details.

Gate services are akin to syscalls, but they work differently. New services can be added easily, and available services are discovered at run time. See Service implementation for details.

Building blocks

Gate appears as Go packages and programs. The execution mechanism is implemented in C++ and assembly. It is highly Linux-dependent. x86-64 and ARM64 are supported.

Important Go packages:

• wag: The WebAssembly compiler (implemented in a separate repository).

• gate/runtime: Core functionality. Interface to the execution mechanism.

• gate/image: Low-level executable building and instance management.

• gate/build: High-level executable building and snapshot restoration.

• gate/server/web: HTTP server component which executes your code on purpose. It has a RESTful API, but some actions can be invoked also via websocket.

• gate/service: Service implementation support and built-in services.

See the complete list of Go packages.

Programs:

• gate: Command-line client for local daemon and remote servers. Uses SSH keys (Ed25519) for authentication.

• gate-daemon: D-Bus daemon for running and managing instances and wasm modules locally.

• gate-server: Standalone web server which can serve the public or require authentication.

• gate-runtime: For optionally preconfiguring the execution environment for daemon/server, e.g. as a system service.

The available services are determined by what is built into the gate-daemon and gate-server programs. The versions provided by this Go module include only the services implemented in this repository. See extension about bundling additional services.

Objectives

While code is data, most of the time data cannot be treated as code for safety reasons. Change that at the Internet level. Data encapsulated in code can describe and transform itself.

Application portability. Migrate processes between mobile devices and servers when circumstances change: user presence, resource availability or demand, continuity etc.

Overhead needs to be low enough so that the system can be practical. Low startup latency for request processing. Low memory overhead for high density of continually running programs.

Work in progress

• Linux x86-64 host support
• Android host support (#33)
• Support for WebAssembly version 1
• Planned security measures have been implemented
• HTTP server for running programs
• Client can communicate with the program it runs on the server
• Speculative execution security issue mitigations
• Pluggable authentication
• Load programs from IPFS
• Reconnect to program instance
• Snapshot
• Restore
• Mechanism for implementing external services in language agnostic way (gRPC)
• Full ARM64 host support
• Programs can discover and communicate with their peers on a server (#23)
• Milestone 1
• Clone programs locally or remotely, with or without snapshotting (#9)
• Milestone 2
• Useful resource control policies need more thought (cgroup configuration etc.)
• More services
• Stable APIs
• Additional security measures (#24, #25)
• Non-Linux host support

User program support:

• Low-level C API
• Rust support
• Go support
• Approach for splitting WebAssembly app between browser (UI) and server (state)

Requirements and build instructions

Run-time dependencies:

• Programs other than gate require Linux 5.3. gate's remote access features should work on any operating system, but are routinely tested only on Linux.

• D-Bus is used for communication between gate and gate-daemon, requiring D-Bus user service (dbus-user-session). gate doesn't require D-Bus when accessing a remote server.

• Programs other than gate may need external tools depending on their configuration and capabilities.

There are two approaches to building Gate: the normal Go way, or via the make.go build system.

Normal build using Go

Build requirements:

• The gate client program requires Go 1.13
• Other programs require Go 1.16

The Gate programs and programs importing Gate modules can be built normally using go install, go get etc. See the gate-cmd repository for a list of programs.

Gate runtime needs to execute some separately built binaries. To make the built Go programs self-contained, pre-built binaries are bundled into them by default. The pre-built binary files are under version control, and can be rebuilt using go generate. To disable bundling of pre-built binaries, specify -tags=gateexecdir for the Go build command, and use make.go to build and install them separately.

Build everything using make.go

Build targets:

• go run make.go lib builds components implemented with C++ and assembly.
• go run make.go bin builds Go programs without bundling non-Go components.
• go run make.go builds all of the above and more.
• go run make.go check runs tests.
• go run make.go -h shows all targets and options.

Build requirements:

• Linux
• C++ compiler
• Go compiler

Test requirements:

• Python 3
• uidmap (shadow-utils)

Installation

The build system builds a standalone installer which can be invoked as root:

1. go run make.go or go run make.go installer ...
2. sudo bin/install

See also

• Gain crate for Rust user programs
• C API for user programs
• ABI for user programs
• Web server API
• Service implementation
• Build-time extensions
• Security
• Container capabilities
• Go packages
• wag

Documentation

Overview

Package gate contains general documentation for its subpackages. See https://gate.computer/gate for information about the Gate project.

Errors

Error strings may contain sensitive details. Some errors returned by Gate implement this interface:

interface {
	PublicError() string
}

The public error string is intended to be exposed to the client (if the API was called via a server endpoint). If there is no PublicError method or it returns the empty string, it's an internal error with no public explanation.