sftpgo

README

SFTPGo

English | 简体中文

Fully featured and highly configurable SFTP server with optional HTTP/S, FTP/S and WebDAV support. Several storage backends are supported: local filesystem, encrypted local filesystem, S3 (compatible) Object Storage, Google Cloud Storage, Azure Blob Storage, SFTP.

Sponsors

If you find SFTPGo useful please consider supporting this Open Source project.

Maintaining and evolving SFTPGo is a lot of work - easily the equivalent of a full time job - for me.

I'd like to make SFTPGo into a sustainable long term project and would not like to introduce a dual licensing option and limit some features to the proprietary version only.

If you use SFTPGo, it is in your best interest to ensure that the project you rely on stays healthy and well maintained. This can only happen with your donations and sponsorships ❤

If you just take and don't return anything back, the project will die in the long run and you will be forced to pay for a similar proprietary solution.

More info.

Thank you to our sponsors

Platinum sponsors

Bronze sponsors

Support policy

SFTPGo is an Open Source project and you can of course use it for free but please don't ask for free support as well.

We will check the reported issues to see if you are experiencing a bug and if so we'll will fix it, but will only provide support to project sponsors/donors.

If you report an invalid issue or ask for step-by-step support, your issue will remain open with no answer or will be closed as invalid without further explanation. Thanks for understanding.

Features

• Support for serving local filesystem, encrypted local filesystem, S3 Compatible Object Storage, Google Cloud Storage, Azure Blob Storage or other SFTP accounts over SFTP/SCP/FTP/WebDAV.
• Virtual folders are supported: a virtual folder can use any of the supported storage backends. So you can have, for example, an S3 user that exposes a GCS bucket (or part of it) on a specified path and an encrypted local filesystem on another one. Virtual folders can be private or shared among multiple users, for shared virtual folders you can define different quota limits for each user.
• Configurable custom commands and/or HTTP hooks on upload, pre-upload, download, pre-download, delete, pre-delete, rename, mkdir, rmdir on SSH commands and on user add, update and delete.
• Virtual accounts stored within a "data provider".
• SQLite, MySQL, PostgreSQL, CockroachDB, Bolt (key/value store in pure Go) and in-memory data providers are supported.
• Chroot isolation for local accounts. Cloud-based accounts can be restricted to a certain base path.
• Per-user and per-directory virtual permissions, for each exposed path you can allow or deny: directory listing, upload, overwrite, download, delete, rename, create directories, create symlinks, change owner/group/file mode and modification time.
• REST API for users and folders management, data retention, backup, restore and real time reports of the active connections with possibility of forcibly closing a connection.
• The Event Manager allows to define custom workflows based on server events or schedules.
• Web based administration interface to easily manage users, folders and connections.
• Web client interface so that end users can change their credentials, manage and share their files in the browser.
• Public key and password authentication. Multiple public keys per-user are supported.
• SSH user certificate authentication.
• Keyboard interactive authentication. You can easily setup a customizable multi-factor authentication.
• Partial authentication. You can configure multi-step authentication requiring, for example, the user password after successful public key authentication.
• Per-user authentication methods.
• Two-factor authentication based on time-based one time passwords (RFC 6238) which works with Authy, Google Authenticator and other compatible apps.
• Simplified user administrations using groups.
• Roles allow you to create limited administrators who can only create and manage users with their role.
• Custom authentication via external programs/HTTP API.
• Web Client and Web Admin user interfaces support OpenID Connect authentication and so they can be integrated with identity providers such as Keycloak. You can find more details here.
• Data At Rest Encryption.
• Dynamic user modification before login via external programs/HTTP API.
• Quota support: accounts can have individual disk quota expressed as max total size and/or max number of files.
• Bandwidth throttling, with separate settings for upload and download and overrides based on the client's IP address.
• Data transfer bandwidth limits, with total limit or separate settings for uploads and downloads and overrides based on the client's IP address. Limits can be reset using the REST API.
• Per-protocol rate limiting is supported and can be optionally connected to the built-in defender to automatically block hosts that repeatedly exceed the configured limit.
• Per-user maximum concurrent sessions.
• Per-user and global IP filters: login can be restricted to specific ranges of IP addresses or to a specific IP address.
• Per-user and per-directory shell like patterns filters: files can be allowed, denied and optionally hidden based on shell like patterns.
• Automatically terminating idle connections.
• Automatic blocklist management using the built-in defender.
• Geo-IP filtering using a plugin.
• Atomic uploads are configurable.
• Per-user files/folders ownership mapping: you can map all the users to the system account that runs SFTPGo (all platforms are supported) or you can run SFTPGo as root user and map each user or group of users to a different system account (*NIX only).
• Support for Git repositories over SSH.
• SCP and rsync are supported.
• FTP/S is supported. You can configure the FTP service to require TLS for both control and data connections.
• WebDAV is supported.
• ACME protocol is supported. SFTPGo can obtain and automatically renew TLS certificates for HTTPS, WebDAV and FTPS from Let's Encrypt or other ACME compliant certificate authorities, using the the HTTP-01 or TLS-ALPN-01 challenge types.
• Two-Way TLS authentication, aka TLS with client certificate authentication, is supported for REST API/Web Admin, FTPS and WebDAV over HTTPS.
• Per-user protocols restrictions. You can configure the allowed protocols (SSH/HTTP/FTP/WebDAV) for each user.
• Prometheus metrics are exposed.
• Support for HAProxy PROXY protocol: you can proxy and/or load balance the SFTP/SCP/FTP service without losing the information about the client's address.
• Easy migration from Linux system user accounts.
• Portable mode: a convenient way to share a single directory on demand.
• SFTP subsystem mode: you can use SFTPGo as OpenSSH's SFTP subsystem.
• Performance analysis using built-in profiler.
• Configuration format is at your choice: JSON, TOML, YAML, HCL, envfile are supported.
• Log files are accurate and they are saved in the easily parsable JSON format (more information).
• SFTPGo supports a plugin system and therefore can be extended using external plugins.

Platforms

SFTPGo is developed and tested on Linux. After each commit, the code is automatically built and tested on Linux, macOS and Windows using GitHub Actions. The test cases are regularly manually executed and passed on FreeBSD. Other *BSD variants should work too.

Requirements

• Go as build only dependency. We support the Go version(s) used in continuous integration workflows.
• A suitable SQL server to use as data provider:
  • upstream supported versions of PostgreSQL, MySQL and MariaDB.
  • CockroachDB stable.
• The SQL server is optional: you can choose to use an embedded SQLite, bolt or in memory data provider.

Installation

Binary releases for Linux, macOS, and Windows are available. Please visit the releases page.

An official Docker image is available. Documentation is here.

Some Linux distro packages are available

• For Arch Linux via AUR:
  • sftpgo. This package follows stable releases. It requires git, gcc and go to build.
  • sftpgo-bin. This package follows stable releases downloading the prebuilt linux binary from GitHub. It does not require git, gcc and go to build.
  • sftpgo-git. This package builds and installs the latest git main branch. It requires git, gcc and go to build.
• Deb and RPM packages are built after each commit and for each release.
• For Ubuntu a PPA is available here.
• Void Linux provides an official package.

APT and YUM repositories are available.

SFTPGo is also available on some marketplaces:

• AWS Marketplace
• Azure Marketplace
• Elest.io

Purchasing from there will help keep SFTPGo a long-term sustainable project.

Windows packages

• The Windows installer to install and run SFTPGo as a Windows service.
• The portable package to start SFTPGo on demand.
• The winget package to install and run SFTPGo as a Windows service: winget install SFTPGo.
• The Chocolatey package to install and run SFTPGo as a Windows service.

On macOS you can install from the Homebrew Formula. On FreeBSD you can install from the SFTPGo port. On DragonFlyBSD you can install SFTPGo from DPorts.

You can easily test new features selecting a commit from the Actions page and downloading the matching build artifacts for Linux, macOS or Windows. GitHub stores artifacts for 90 days.

Alternately, you can build from source.

Getting Started Guide for the Impatient.

Configuration

A full explanation of all configuration methods can be found here.

Please make sure to initialize the data provider before running the daemon.

To start SFTPGo with the default settings, simply run:

sftpgo serve

Check out this documentation if you want to run SFTPGo as a service.

Data provider initialization and management

Before starting the SFTPGo server please ensure that the configured data provider is properly initialized/updated.

For PostgreSQL, MySQL and CockroachDB providers, you need to create the configured database. For SQLite, the configured database will be automatically created at startup. Memory and bolt data providers do not require an initialization but they could require an update to the existing data after upgrading SFTPGo.

SFTPGo will attempt to automatically detect if the data provider is initialized/updated and if not, will attempt to initialize/ update it on startup as needed.

Alternately, you can create/update the required data provider structures yourself using the initprovider command.

For example, you can simply execute the following command from the configuration directory:

sftpgo initprovider

Take a look at the CLI usage to learn how to specify a different configuration file:

sftpgo initprovider --help

You can disable automatic data provider checks/updates at startup by setting the update_mode configuration key to 1.

You can also reset your provider by using the resetprovider sub-command. Take a look at the CLI usage for more details:

sftpgo resetprovider --help

⚠ Please note that some data providers (e.g. MySQL and CockroachDB) do not support schema changes within a transaction, this means that you may end up with an inconsistent schema if migrations are forcibly aborted. CockroachDB doesn't support database-level locks, so make sure you don't execute migrations concurrently.

Create the first admin

To start using SFTPGo you need to create an admin user, you can do it in several ways:

• by using the web admin interface. The default URL is http://127.0.0.1:8080/web/admin
• by loading initial data
• by enabling create_default_admin in your configuration file and setting the environment variables SFTPGO_DEFAULT_ADMIN_USERNAME and SFTPGO_DEFAULT_ADMIN_PASSWORD

Upgrading

SFTPGo supports upgrading from the previous release branch to the current one. Some examples for supported upgrade paths are:

• from 1.2.x to 2.0.x
• from 2.0.x to 2.1.x and so on.

For supported upgrade paths, the data and schema are migrated automatically, alternately you can use the initprovider command.

So if, for example, you want to upgrade from a version before 1.2.x to 2.0.x, you must first install version 1.2.x, update the data provider and finally install the version 2.0.x. It is recommended to always install the latest available minor version, ie do not install 1.2.0 if 1.2.2 is available.

Loading data from a provider independent JSON dump is supported from the previous release branch to the current one too. After upgrading SFTPGo it is advisable to regenerate the JSON dump from the new version.

Downgrading

If for some reason you want to downgrade SFTPGo, you may need to downgrade your data provider schema and data as well. You can use the revertprovider command for this task.

As for upgrading, SFTPGo supports downgrading from the previous release branch to the current one.

So, if you plan to downgrade from 2.0.x to 1.2.x, before uninstalling 2.0.x version, you can prepare your data provider executing the following command from the configuration directory:

sftpgo revertprovider --to-version 4

Take a look at the CLI usage to see the supported parameter for the --to-version argument and to learn how to specify a different configuration file:

sftpgo revertprovider --help

The revertprovider command is not supported for the memory provider.

Please note that we only support the current release branch and the current main branch, if you find a bug it is better to report it rather than downgrading to an older unsupported version.

Users, groups and folders management

After starting SFTPGo you can manage users, groups, folders and other resources using:

• the web based administration interface
• the REST API

To support embedded data providers like bolt and SQLite, which do not support concurrent connections, we can't have a CLI that directly write users and other resources to the data provider, we always have to use the REST API.

Full details for users, groups, folders, admins and other resources are documented in the OpenAPI schema. If you want to render the schema without importing it manually, you can explore it on Stoplight.

⚠ SFTPGo users, groups and folders are virtual and therefore unrelated to the system ones. There is no need to create system-wide users and groups.

Tutorials

Some step-to-step tutorials can be found inside the source tree howto directory.

Authentication options

External Authentication

Custom authentication methods can easily be added. SFTPGo supports external authentication modules, and writing a new backend can be as simple as a few lines of shell script. More information can be found here.

Keyboard Interactive Authentication

Keyboard interactive authentication is, in general, a series of questions asked by the server with responses provided by the client. This authentication method is typically used for multi-factor authentication.

More information can be found here.

Dynamic user creation or modification

A user can be created or modified by an external program just before the login. More information about this can be found here.

Custom Actions

SFTPGo allows you to configure custom commands and/or HTTP hooks to receive notifications about file uploads, deletions and several other events.

More information about custom actions can be found here.

Virtual folders

Directories outside the user home directory or based on a different storage provider can be exposed as virtual folders, more information here.

Other hooks

You can get notified as soon as a new connection is established using the Post-connect hook and after each login using the Post-login hook. You can use your own hook to check passwords.

Storage backends

S3/GCP/Azure

Each user can be mapped with a S3 Compatible Object Storage /Google Cloud Storage/Azure Blob Storage bucket or a bucket virtual folder that is exposed over SFTP/SCP/FTP/WebDAV.

SFTP backend

Each user can be mapped to another SFTP server account or a subfolder of it. More information can be found here.

Encrypted backend

Data at-rest encryption is supported via the cryptfs backend.

HTTP/S backend

HTTP/S backend allows you to write your own custom storage backend by implementing a REST API. More information can be found here.

Other Storage backends

Adding new storage backends is quite easy:

• implement the Fs interface.
• update the user method GetFilesystem to return the new backend
• update the web interface and the REST API CLI
• add the flags for the new storage backed to the portable mode

Anyway, some backends require a pay per-use account (or they offer free account for a limited time period only). To be able to add support for such backends or to review pull requests, please provide a test account. The test account must be available for enough time to be able to maintain the backend and do basic tests before each new release.

Brute force protection

SFTPGo supports a built-in defender.

Alternately you can use the connection failed logs for integration in tools such as Fail2ban. Example of jails and filters working with systemd/journald are available in fail2ban directory.

Account's configuration properties

Details information about account configuration properties can be found here.

Performance

SFTPGo can easily saturate a Gigabit connection on low end hardware with no special configuration, this is generally enough for most use cases.

More in-depth analysis of performance can be found here.

Release Cadence

SFTPGo releases are feature-driven, we don't have a fixed time based schedule. As a rough estimate, you can expect 1 or 2 new releases per year.

Acknowledgements

SFTPGo makes use of the third party libraries listed inside go.mod.

We are very grateful to all the people who contributed with ideas and/or pull requests.

Thank you ysura for granting me stable access to a test AWS S3 account.

License

GNU AGPL-3.0-only

Documentation

Overview

Fully featured and highly configurable SFTP server with optional FTP/S and WebDAV support. For more details about features, installation, configuration and usage please refer to the README inside the source tree: https://github.com/aKardasz/sftpgo/blob/main/README.md