guerrilla

package module
v0.0.0-...-6f895c6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 30, 2017 License: MIT Imports: 23 Imported by: 0

README

Build Status

Go-Guerrilla SMTP Daemon

A lightweight SMTP server written in Go, made for receiving large volumes of mail. To be used as a package in your Go project, or as a stand-alone daemon by running the "guerrillad" binary.

Supports MySQL and Redis out-of-the-box, with many other vendor provided processors, such as MailDir and even FastCGI! See below for a list of available processors.

Go Guerrilla

What is Go-Guerrilla?

It's an SMTP server written in Go, for the purpose of receiving large volumes of email. It started as a project for GuerrillaMail.com which processes millions of emails every day, and needed a daemon with less bloat & written in a more memory-safe language that can take advantage of modern multi-core architectures.

The purpose of this daemon is to grab the email, save it, and disconnect as quickly as possible, essentially performing the services of a Mail Transfer Agent (MTA) without the sending functionality.

The software also includes a modular backend implementation, which can extend the email processing functionality to whatever needs you may require. We refer to these modules as "Processors". Processors can be chained via the config to perform different tasks on received email, or to validate recipients.

See the list of available Processors below.

For more details about the backend system, see the: Backends, configuring and extending page.

License

The software is using MIT License (MIT) - contributors welcome.

Features
Main Features
  • Multi-server. Can spawn multiple servers, all sharing the same backend for saving email.
  • Config hot-reloading. Add/Remove/Enable/Disable servers without restarting. Reload TLS configuration, change most other settings on the fly.
  • Graceful shutdown: Minimise loss of email if you need to shutdown/restart.
  • Be a gentleman to the garbage collector: resources are pooled & recycled where possible.
  • Modular Backend system
  • Modern TLS support (STARTTLS or SMTPS).
  • Can be used as a package in your Go project. Get started in just a few lines of code!
  • Fuzz tested. Auto-tested. Battle Tested.
Backend Features
  • Arranged as workers running in parallel, using a producer/consumer type structure, taking advantage of Go's channels and go-routines.
  • Modular backend system structured using a decorator-like pattern which allows the chaining of components (a.k.a. Processors) via the config.
  • Different ways for processing / delivering email: Supports MySQL and Redis out-of-the box, many other vendor provided processors available.
Roadmap / Contributing & Bounties

Pull requests / issue reporting & discussion / code reviews always welcome. To encourage more pull requests, we are now offering bounties.

Take a look at our Bounties and Roadmap page!

Getting started

(Assuming that you have GNU make and latest Go on your system)

Dependencies

Go-Guerrilla uses Glide to manage dependencies. If you have glide installed, just run glide install as usual.

You can also run $ go get ./.. if you don't want to use glide, and then run $ make test to ensure all is good.

To build the binary run:

$ make guerrillad

This will create a executable file named guerrillad that's ready to run.

Next, copy the goguerrilla.conf.sample file to goguerrilla.conf.json. You may need to customize the pid_file setting to somewhere local, and also set tls_always_on to false if you don't have a valid certificate setup yet.

Next, run your server like this:

$ ./guerrillad serve

The configuration options are detailed on the configuration page. The main takeaway here is:

The default configuration uses 3 processors, they are set using the save_process config option. Notice that it contains the following value: "HeadersParser|Header|Debugger" - this means, once an email is received, it will first go through the HeadersParser processor where headers will be parsed. Next, it will go through the Header processor, where delivery headers will be added. Finally, it will finish at the Debugger which will log some debug messages.

Where to go next?

Use as a package

Go-Guerrilla can be imported and used as a package in your Go project.

Quickstart
1. Import the guerrilla package
import (
    "github.com/flashmob/go-guerrilla"
)

You may use $ go get ./... to get all dependencies, also Go-Guerrilla uses glide for dependency management.

2. Start a server

This will start a server with the default settings, listening on 127.0.0.1:2525


d := guerrilla.Daemon{}
err := d.Start()

if err == nil {
    fmt.Println("Server Started!")
}

d.Start() does not block after the server has been started, so make sure that you keep your program busy.

The defaults are:

  • Server listening to 127.0.0.1:2525
  • use your hostname to determine your which hosts to accept email for
  • 100 maximum clients
  • 10MB max message size
  • log to Stderror,
  • log level set to "debug"
  • timeout to 30 sec
  • Backend configured with the following processors: HeadersParser|Header|Debugger where it will log the received emails.

Next, you may want to change the interface (127.0.0.1:2525) to the one of your own choice.

API Documentation topics

Please continue to the API documentation for the following topics:

Use as a Daemon

Manual for using from the command line
Other topics

Email Processing Backend

The main job of a Go-Guerrilla backend is to validate recipients and deliver emails. The term "delivery" is often synonymous with saving email to secondary storage.

The default backend implementation manages multiple workers. These workers are composed of smaller components called "Processors" which are chained using the config to perform a series of steps. Each processor specifies a distinct feature of behaviour. For example, a processor may save the emails to a particular storage system such as MySQL, or it may add additional headers before passing the email to the next processor.

To extend or add a new feature, one would write a new Processor, then add it to the config. There are a few default processors to get you started.

Included Processors
Processor Description
Compressor Sets a zlib compressor that other processors can use later
Debugger Logs the email envelope to help with testing
Hasher Processes each envelope to produce unique hashes to be used for ids later
Header Add a delivery header to the envelope
HeadersParser Parses MIME headers and also populates the Subject field of the envelope
MySQL Saves the emails to MySQL.
Redis Saves the email data to Redis.
GuerrillaDbRedis A 'monolithic' processor used at Guerrilla Mail; included for example
Available Processors

The following processors can be imported to your project, then use the Daemon.AddProcessor function to register, then add to your config.

Processor Description
MailDir Save emails to a maildir. MailDiranasaurus is an example project
FastCGI Deliver email directly to PHP-FPM or a similar FastCGI backend.
WildcardProcessor Use wildcards for recipients host validation.

Have a processor that you would like to share? Submit a PR to add it to the list!

Releases

Current release: 1.5.1 - 4th Nov 2016

Next Planned release: 2.0.0 - TBA

See our change log for change and release history

Using Nginx as a proxy

For such purposes as load balancing, terminating TLS early, or supporting SSL versions not supported by Go (highly not recommenced if you want to use older SSL versions), it is possible to use NGINX as a proxy.

Credits

Project Lead:

Flashmob, GuerrillaMail.com, Contact: flashmob@gmail.com

Major Contributors:

Thanks to:

... and anyone else who opened an issue / sent a PR / gave suggestions!

Documentation

Index

Constants

View Source 
const (
	// The client has connected, and is awaiting our first response
	ClientGreeting = iota
	// We have responded to the client's connection and are awaiting a command
	ClientCmd
	// We have received the sender and recipient information
	ClientData
	// We have agreed with the client to secure the connection over TLS
	ClientStartTLS
	// Server will shutdown, client to shutdown on next command turn
	ClientShutdown
)
View Source 
const (
	// server has just been created
	GuerrillaStateNew = iota
	// Server has been started and is running
	GuerrillaStateStarted
	// Server has just been stopped
	GuerrillaStateStopped
)
View Source 
const (
	CommandVerbMaxLength = 16
	CommandLineMaxLength = 1024
	// Number of allowed unrecognized commands before we terminate the connection
	MaxUnrecognizedCommands = 5
	// The maximum total length of a reverse-path or forward-path is 256
	RFC2821LimitPath = 256
	// The maximum total length of a user name or other local-part is 64
	RFC2832LimitLocalPart = 64
	//The maximum total length of a domain name or number is 255
	RFC2821LimitDomain = 255
	// The minimum total number of recipients that must be buffered is 100
	RFC2821LimitRecipients = 100
)
View Source 
const (
	// server has just been created
	ServerStateNew = iota
	// Server has just been stopped
	ServerStateStopped
	// Server has been started and is running
	ServerStateRunning
	// Server could not start due to an error
	ServerStateStartError
)

Variables

View Source 
var (
	LineLimitExceeded   = errors.New("Maximum line length exceeded")
	MessageSizeExceeded = errors.New("Maximum message size exceeded")
)
View Source 
var (
	Version   string
	Commit    string
	BuildTime string

	StartTime      time.Time
	ConfigLoadTime time.Time
)
View Source 
var (
	ErrPoolShuttingDown = errors.New("server pool: shutting down")
)

Functions

func NewClient 

func NewClient(conn net.Conn, clientID uint64, logger log.Logger, envelope *mail.Pool) *client

NewClient allocates a new client.

Types

type AppConfig 

type AppConfig struct {
	// Servers can have one or more items.
	/// Defaults to 1 server listening on 127.0.0.1:2525
	Servers []ServerConfig `json:"servers"`
	// AllowedHosts lists which hosts to accept email for. Defaults to os.Hostname
	AllowedHosts []string `json:"allowed_hosts"`
	// PidFile is the path for writing out the process id. No output if empty
	PidFile string `json:"pid_file"`
	// LogFile is where the logs go. Use path to file, or "stderr", "stdout"
	// or "off". Default "stderr"
	LogFile string `json:"log_file,omitempty"`
	// LogLevel controls the lowest level we log.
	// "info", "debug", "error", "panic". Default "info"
	LogLevel string `json:"log_level,omitempty"`
	// BackendConfig configures the email envelope processing backend
	BackendConfig backends.BackendConfig `json:"backend_config"`
}

AppConfig is the holder of the configuration of the app

func (*AppConfig) EmitChangeEvents 

func (c *AppConfig) EmitChangeEvents(oldConfig *AppConfig, app Guerrilla)

Emits any configuration change events onto the event bus.

func (*AppConfig) EmitLogReopenEvents 

func (c *AppConfig) EmitLogReopenEvents(app Guerrilla)

EmitLogReopen emits log reopen events using existing config

func (*AppConfig) Load 

func (c *AppConfig) Load(jsonBytes []byte) error

Unmarshalls json data into AppConfig struct and any other initialization of the struct also does validation, returns error if validation failed or something went wrong

type ClientState 

type ClientState int

ClientState indicates which part of the SMTP transaction a given client is in.

type Daemon 

type Daemon struct {
	Config  *AppConfig
	Logger  log.Logger
	Backend backends.Backend
	// contains filtered or unexported fields
}

Daemon provides a convenient API when using go-guerrilla as a package in your Go project. Is's facade for Guerrilla, AppConfig, backends.Backend and log.Logger

func (*Daemon) AddProcessor 

func (d *Daemon) AddProcessor(name string, pc backends.ProcessorConstructor)

AddProcessor adds a processor constructor to the backend. name is the identifier to be used in the config. See backends docs for more info.

func (*Daemon) LoadConfig 

func (d *Daemon) LoadConfig(path string) (AppConfig, error)

LoadConfig reads in the config from a JSON file. Note: if d.Config is nil, the sets d.Config with the unmarshalled AppConfig which will be returned

func (*Daemon) Log 

func (d *Daemon) Log() log.Logger

log returns a logger that implements our log.Logger interface. level is set to "info" by default

func (*Daemon) Publish 

func (d *Daemon) Publish(topic Event, args ...interface{})

for publishing config change events

func (*Daemon) ReloadConfig 

func (d *Daemon) ReloadConfig(c AppConfig) error

Reload a config using the passed in AppConfig and emit config change events

func (*Daemon) ReloadConfigFile 

func (d *Daemon) ReloadConfigFile(path string) error

Reload a config from a file and emit config change events

func (*Daemon) ReopenLogs 

func (d *Daemon) ReopenLogs() error

ReopenLogs send events to re-opens all log files. Typically, one would call this after rotating logs

func (*Daemon) SetConfig 

func (d *Daemon) SetConfig(c AppConfig) error

SetConfig is same as LoadConfig, except you can pass AppConfig directly does not emit any change events, instead use ReloadConfig after daemon has started

func (*Daemon) Shutdown 

func (d *Daemon) Shutdown()

Shuts down the daemon, including servers and backend. Do not call Start on it again, use a new server.

func (*Daemon) Start 

func (d *Daemon) Start() (err error)

Starts the daemon, initializing d.Config, d.Logger and d.Backend with defaults can only be called once through the lifetime of the program

func (*Daemon) Subscribe 

func (d *Daemon) Subscribe(topic Event, fn interface{}) error

Subscribe for subscribing to config change events

func (*Daemon) Unsubscribe 

func (d *Daemon) Unsubscribe(topic Event, handler interface{}) error

for unsubscribing from config change events

type Errors 

type Errors []error

func (Errors) Error 

func (e Errors) Error() string

implement the Error interface

type Event 

type Event int
const (
	// when a new config was loaded
	EventConfigNewConfig Event = iota
	// when allowed_hosts changed
	EventConfigAllowedHosts
	// when pid_file changed
	EventConfigPidFile
	// when log_file changed
	EventConfigLogFile
	// when it's time to reload the main log file
	EventConfigLogReopen
	// when log level changed
	EventConfigLogLevel
	// when the backend's config changed
	EventConfigBackendConfig
	// when a new server was added
	EventConfigServerNew
	// when an existing server was removed
	EventConfigServerRemove
	// when a new server config was detected (general event)
	EventConfigServerConfig
	// when a server was enabled
	EventConfigServerStart
	// when a server was disabled
	EventConfigServerStop
	// when a server's log file changed
	EventConfigServerLogFile
	// when it's time to reload the server's log
	EventConfigServerLogReopen
	// when a server's timeout changed
	EventConfigServerTimeout
	// when a server's max clients changed
	EventConfigServerMaxClients
	// when a server's TLS config changed
	EventConfigServerTLSConfig
)

func (Event) String 

func (e Event) String() string

type EventHandler 

type EventHandler struct {
	evbus.Bus
}

func (*EventHandler) Publish 

func (h *EventHandler) Publish(topic Event, args ...interface{})

func (*EventHandler) Subscribe 

func (h *EventHandler) Subscribe(topic Event, fn interface{}) error

func (*EventHandler) Unsubscribe 

func (h *EventHandler) Unsubscribe(topic Event, handler interface{}) error

type Guerrilla 

type Guerrilla interface {
	Start() error
	Shutdown()
	Subscribe(topic Event, fn interface{}) error
	Publish(topic Event, args ...interface{})
	Unsubscribe(topic Event, handler interface{}) error
	SetLogger(log.Logger)
}

func New 

func New(ac *AppConfig, b backends.Backend, l log.Logger) (Guerrilla, error)

Returns a new instance of Guerrilla with the given config, not yet running. Backend started.

type Pool 

type Pool struct {
	ShutdownChan chan int
	// contains filtered or unexported fields
}

Pool holds Clients.

func NewPool 

func NewPool(poolSize int) *Pool

NewPool creates a new pool of Clients.

func (*Pool) Borrow 

func (p *Pool) Borrow(conn net.Conn, clientID uint64, logger log.Logger, ep *mail.Pool) (Poolable, error)

Borrow a Client from the pool. Will block if len(activeClients) > maxClients

func (*Pool) GetActiveClientsCount 

func (p *Pool) GetActiveClientsCount() int

Gets the number of active clients that are currently out of the pool and busy serving

func (*Pool) IsShuttingDown 

func (p *Pool) IsShuttingDown() bool

returns true if the pool is shutting down

func (*Pool) Return 

func (p *Pool) Return(c Poolable)

Return returns a Client back to the pool.

func (*Pool) SetTimeout 

func (p *Pool) SetTimeout(duration time.Duration)

set a timeout for all lent clients

func (*Pool) ShutdownState 

func (p *Pool) ShutdownState()

Lock the pool from borrowing then remove all active clients each active client's timeout is lowered to 1 sec and notified to stop accepting commands

func (*Pool) ShutdownWait 

func (p *Pool) ShutdownWait()

func (*Pool) Start 

func (p *Pool) Start()

type Poolable 

type Poolable interface {
	// contains filtered or unexported methods
}

a struct can be pooled if it has the following interface

type ServerConfig 

type ServerConfig struct {
	// IsEnabled set to true to start the server, false will ignore it
	IsEnabled bool `json:"is_enabled"`
	// Hostname will be used in the server's reply to HELO/EHLO. If TLS enabled
	// make sure that the Hostname matches the cert. Defaults to os.Hostname()
	Hostname string `json:"host_name"`
	// MaxSize is the maximum size of an email that will be accepted for delivery.
	// Defaults to 10 Mebibytes
	MaxSize int64 `json:"max_size"`
	// PrivateKeyFile path to cert private key in PEM format. Will be ignored if blank
	PrivateKeyFile string `json:"private_key_file"`
	// PublicKeyFile path to cert (public key) chain in PEM format.
	// Will be ignored if blank
	PublicKeyFile string `json:"public_key_file"`
	// Timeout specifies the connection timeout in seconds. Defaults to 30
	Timeout int `json:"timeout"`
	// Listen interface specified in <ip>:<port> - defaults to 127.0.0.1:2525
	ListenInterface string `json:"listen_interface"`
	// StartTLSOn should we offer STARTTLS command. Cert must be valid.
	// False by default
	StartTLSOn bool `json:"start_tls_on,omitempty"`
	// TLSAlwaysOn run this server as a pure TLS server, i.e. SMTPS
	TLSAlwaysOn bool `json:"tls_always_on,omitempty"`
	// MaxClients controls how many maxiumum clients we can handle at once.
	// Defaults to 100
	MaxClients int `json:"max_clients"`
	// LogFile is where the logs go. Use path to file, or "stderr", "stdout" or "off".
	// defaults to AppConfig.Log file setting
	LogFile string `json:"log_file,omitempty"`
	// XClientOn when using a proxy such as Nginx, XCLIENT command is used to pass the
	// original client's IP address & client's HELO
	XClientOn bool `json:"xclient_on,omitempty"`
	// contains filtered or unexported fields
}

ServerConfig specifies config options for a single server

func (*ServerConfig) Validate 

func (sc *ServerConfig) Validate() error

Validate validates the server's configuration.

Source Files

View all Source files

Directories

Path Synopsis
cmd
guerrillad
testcert

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.