paygate

README

moov-io/paygate

Moov Paygate is a RESTful API enabling Automated Clearing House (ACH) transactions to be submitted and received without a deep understanding of a full NACHA file specification.

Docs: docs.moov.io | api docs | admin api docs

Project Status

This project is currently under development and could introduce breaking changes to reach a stable status. We are looking for community feedback so please try out our code or give us feedback!

If you are building from source for v0.8.x please use the release branch which is stable for that series.

Getting Started

Paygate can be ran or deployed in various ways. We have several guides for running paygate and offer a testing utility called apitest from the moov-io/api repository for verifying paygate (and its dependnecies) are running properly.

• Using docker-compose
• Using our Docker image
• Build from source
• How to setup open source ACH payments using Moov.io suite by Taylor Gunnoe
  • Taylor has also written paygate-cli which is a command-line interface to paygate.

Vocab

CIP

Customer Identification Program -- Verification process of customer information including KYC and social security number

customerID

Unique ID from the Moov Customers service. KYC and CIP checks utilize this to associate with paygate objects.

KYC

Know Your Customer -- Verification process of customer information including: name, date of birth and address

Originator

Entity that initiates the ACH transfer. Often usecases have one Originator to make all transfers from.

Receiver

Entity that is the target of an ACH transfer.

userID

Unique ID used to organize and isolate objects created within paygate. Objects are not accessible across userIDs

Deployment

Paygate currently requires the following services to be deployed and available:

• ACH (HTTP Server) via ACH_ENDPOINT

• FED (HTTP Server) via FED_ENDPOINT

• The X-User-Id (case insensntive) HTTP header is also required and we recommend using an auth proxy to set this. Paygate only expects this value to be unique and consistent to a user.

The following services are required by default, but can be disabled:

• Accounts (HTTP server) via ACCOUNTS_ENDPOINT and disabled with ACCOUNTS_CALLS_DISABLED=yes
• Customers (HTTP server) via CUSTOMERS_ENDPOINT and disabled with CUSTOMERS_CALLS_DISABLED=yes

Docker image

You can download our docker image moov/paygate from Docker Hub or use this repository. No configuration is required to serve on :8082 and metrics at :9092/metrics in Prometheus format.

$ docker run -p 8082:8082 moov/paygate:latest
ts=2018-12-13T19:18:11.970293Z caller=main.go:55 startup="Starting paygate server version v0.5.1"
ts=2018-12-13T19:18:11.970391Z caller=main.go:59 main="sqlite version 3.25.2"
ts=2018-12-13T19:18:11.971777Z caller=database.go:88 sqlite="starting database migrations"
ts=2018-12-13T19:18:11.971886Z caller=database.go:97 sqlite="migration #0 [create table if not exists receivers(cus...] changed 0 rows"
... (more database migration log lines)
ts=2018-12-13T19:18:11.97221Z caller=database.go:100 sqlite="finished migrations"
ts=2018-12-13T19:18:11.974316Z caller=main.go:96 ach="Pong successful to ACH service"
ts=2018-12-13T19:18:11.975093Z caller=main.go:155 transport=HTTP addr=:8082
ts=2018-12-13T19:18:11.975177Z caller=main.go:124 admin="listening on :9092"

$ curl -XPOST -H "x-user-id: test" localhost:8082/originators --data '{...}'

Local development

We support a Docker Compose environment in paygate that can be used to launch the entire Moov stack. After setup launching the stack is the following steps and we offer a testing utility apitest from the moov-io/api repository.

Using the latest released docker-compose.yml is recommended as that will use released versions of dependencies.

$ docker-compose up -d
paygate_ach_1 is up-to-date
paygate_ofac_1 is up-to-date
Recreating paygate_accounts_1 ...
paygate_fed_1 is up-to-date
Recreating paygate_accounts_1 ... done
Recreating paygate_paygate_1  ... done

# Run Moov's testing utility
$ apitest -local
2019/06/10 21:18:06.117261 main.go:61: Starting apitest v0.9.5
2019/06/10 21:18:06.117293 main.go:133: Using http://localhost as base API address
...
2019/06/10 21:18:06.276443 main.go:218: SUCCESS: Created user b1f2671bbed52ed6da88f16ce467cadecb0ee1b6 (email: festive.curran27@example.com)
...
2019/06/10 21:18:06.607817 main.go:218: SUCCESS: Created USD 204.71 transfer (id=b7ecb109574187ff726ba48275dcf88956c26841) for user

Build from source

PayGate orchestrates several services that depend on Docker and additional GoLang libraries to run. Paygate leverages Go Modules to manage dependencies. Ensure that your build environment is running Go 1.14 or greater. PayGate depends on other Docker containers that will be downloaded for testing and running the service. Ensure Docker is installed and running.

$ cd moov/paygate # wherever this project lives

$ go run .
ts=2018-12-13T19:18:11.970293Z caller=main.go:55 startup="Starting paygate server version v0.5.1"
ts=2018-12-13T19:18:11.970391Z caller=main.go:59 main="sqlite version 3.25.2"
ts=2018-12-13T19:18:11.971777Z caller=database.go:88 sqlite="starting database migrations"
ts=2018-12-13T19:18:11.971886Z caller=database.go:97 sqlite="migration #0 [create table if not exists receivers(cus...] changed 0 rows"
... (more database migration log lines)
ts=2018-12-13T19:18:11.97221Z caller=database.go:100 sqlite="finished migrations"
ts=2018-12-13T19:18:11.974316Z caller=main.go:96 ach="Pong successful to ACH service"
ts=2018-12-13T19:18:11.975093Z caller=main.go:155 transport=HTTP addr=:8082
ts=2018-12-13T19:18:11.975177Z caller=main.go:124 admin="listening on :9092"

Architecture

This is an overall diagram of PayGate and its dependencies.

Note: Arrows show connection initiation direction, not entirely dataflow direction.

Configuration

The following environmental variables can be set to configure behavior in paygate.

Environmental Variable	Description	Default
ACH_ENDPOINT	DNS record responsible for routing us to an ACH instance. If running as part of our local development setup (or in a Kubernetes cluster we setup) you won't need to set this.	http://ach.apps.svc.cluster.local:8080/
ACCOUNTS_ENDPOINT	A DNS record responsible for routing us to an Accounts instance.	http://accounts.apps.svc.cluster.local:8080
ACCOUNTS_CALLS_DISABLED=yes	Flag to completely disable all calls to an Accounts service. This is used when paygate doesn't need to integrate with a general ledger solution.	no
CUSTOMERS_ENDPOINT	A DNS record responsible for routing us to a Customers instance.	http://customers.apps.svc.cluster.local:8080
CUSTOMERS_CALLS_DISABLED=yes	Flag to completely disable all calls to a Customers service. This is used when paygate doesn't need to integrate with a KYC/CIP solution.	no
FED_ENDPOINT	HTTP address for FED interaction to lookup ABA routing numbers.	http://fed.apps.svc.cluster.local:8080
FED_CALLS_DISABLED=yes	Flag to completely disable all calls to Fed service. This is used by paygate to verify routing numbers.	no
HTTP_ADMIN_BIND_ADDRESS	Address for paygate to bind its admin HTTP server on. This overrides the command-line flag -admin.addr.	:9092
HTTP_BIND_ADDRESS	Address for paygate to bind its HTTP server on. This overrides the command-line flag -http.addr.	:8082
HTTP_CLIENT_CAFILE	Filepath for additional (CA) certificates to be added into each http.Client used within paygate.	Empty
HTTPS_CERT_FILE	Filepath containing a certificate (or intermediate chain) to be served by the HTTP server. Requires all traffic be over secure HTTP.	Empty
HTTPS_KEY_FILE	Filepath of a private key matching the leaf certificate from HTTPS_CERT_FILE.	Empty
REMOTE_ADDRESS_HEADER	HTTP header name to discover remote address. Takes first IP from comma-separated list.	X-Real-Ip
LOG_FORMAT	Format for logging lines to be written as. (Options: json, plain)	plain
DATABASE_TYPE	Which database option to use - See Storage header below for per-database configuration (Options: sqlite, mysql)	sqlite
CONFIG_FILE	File path if given will load configs from a Yaml file instead of a database.	Empty
CLOUD_PROVIDER	Provider name which determines which of the following environmental variables are used to initialize Customer's persistence.	Empty

ACH File Uploading

Environmental Variable	Description	Default
ACH_FILE_BATCH_SIZE	Number of Transfers to retrieve from the database in each batch for mergin before upload to Fed.	100
ACH_FILE_MAX_LINES	Maximum line count before an ACH file is uploaded to its remote server. NACHA guidelines have a hard limit of 10,000 lines.	10000
ACH_FILE_TRANSFERS_CAFILE	Filepath for additional (CA) certificates to be added into each FTP client used within paygate.	Empty
ACH_FILE_TRANSFER_INTERVAL	Go duration for how often to check and sync ACH files on their SFTP destinations. (Set to off to disable.)	10m
ACH_FILE_STORAGE_DIR	Filepath for temporary storage of ACH files. This is used as a scratch directory to manage outbound and incoming/returned ACH files.	./storage/
FORCED_CUTOFF_UPLOAD_DELTA	Go duration for when the current time is within the routing number's cutoff time by duration force that file to be uploaded.	5m

See our detailed documentation for FTP and SFTP configurations.

Transfers

Environmental Variable	Description	Default
TRANSFERS_ONE_DAY_USER_LIMIT	Maximum sum of transfers for each user over the current day.	5000.00
TRANSFERS_SEVEN_DAY_USER_LIMIT	Maximum sum of transfers for each user over the previous seven days.	10000.00
TRANSFERS_THIRTY_DAY_USER_LIMIT	Maximum sum of transfers for each user over the previous seven days.	25000.00

Inbound / Returned File Processing

Environmental Variable	Description	Default
UPDATE_DEPOSITORIES_FROM_CHANGE_CODE=yes	Depository objects will be updated from COR/NOC addendas from rejected files.	no

FTP Configuration

Our FTP client offers some configuration options. Paygate currently uses the jlaffaye/ftp library.

Environmental Variable	Description	Default
FTP_DIAL_TIMEOUT	Go duration for timeout when creating FTP connections.	10s
FTP_DIAL_WITH_DISABLED_ESPV	Offer EPSV to be used if the FTP server supports it.	false

SFTP Configuration

Our SFTP (SSH File Transfer Protocol) client offers some configuration options. Paygate currently uses the pkg/sftp library.

Environmental Variable	Description	Default
SFTP_DIAL_TIMEOUT	Go duration for timeout when creating SFTP connections.	10s
SFTP_MAX_CONNS_PER_FILE	Sets the maximum concurrent requests allowed for a single file.	8
SFTP_MAX_PACKET_SIZE	Sets the maximum size of the payload, measured in bytes. Try lowering this on "failed to send packet header: EOF" errors.	20480

Note: By default paygate does not verify the SFTP host public key. Write the expected public key into sftp_configs's host_public_key column to have paygate verify.

Depository Verification

In order to validate Depository objects and transfer money PayGate needs to verify the user is authorized to access the Depository. This typically involves micro-deposits (sending two debits under $1 to the account and having the user confirm these amounts) followed by a credit from the account to balance the accounting.

Micro Deposits

PayGate submits small deposits, credits the balance and has the user confirm the amounts manually. This is only required once per Depository. To setup the funding account for these deposits the following configuration options for paygate are below and are all required:

Environmental Variable	Description	Default
ODFI_ACCOUNT_NUMBER	Account Number of Financial Institution which is originating micro deposits.	123
ODFI_ACCOUNT_TYPE	Type of ODFI account used for micro-depsits (Checking or Savings)	Savings
ODFI_BANK_NAME	Legal name of Financial Institution which is originating micro deposits.	Moov, Inc
ODFI_HOLDER	Legal name of Financial Institution which is originating micro deposits.	Moov, Inc
ODFI_IDENTIFICATION	Number by which the customer is known to the Financial Institution originating micro deposits.	001
ODFI_ROUTING_NUMBER	ABA routing number of Financial Institution which is originating micro deposits.	121042882

Account Number Encryption

The following environment variables control which backend service is initialized for account number encryption. They are stored and encrypted with GoCloud CDK's Secrets. (godoc)

Local storage

Environmental Variable	Description	Default
SECRETS_LOCAL_BASE64_KEY	A base64 encoded key used to encrypt and decrypt secrets in memory. This value needs to look like base64key://value where value is a 32 byte random key.	Empty

Google Cloud Storage

Environmental Variable	Description	Default
SECRETS_GCP_KEY_RESOURCE_ID	A Google Cloud resource ID used to interact with their Key Management Service (KMS). This value has the form projects/MYPROJECT/locations/MYLOCATION/keyRings/MYKEYRING/cryptoKeys/MYKEY and their documentation has more details.	Empty

Vault storage

Environmental Variable	Description	Default
VAULT_SERVER_TOKEN	A Vault generated value used to authenticate. See the Hashicorp Vault documentation for more details.	Empty
VAULT_SERVER_URL	A URL for accessing the vault instance. In production environments this should be an HTTPS (TLS) secured connection.	Empty

Storage

Based on DATABASE_TYPE the following environment variables will be read to configure connections for a specific database.

MySQL

Environmental Variable	Description	Default
MYSQL_ADDRESS	TCP address for connecting to the mysql server. (Example tcp(hostname:3306))	Empty
MYSQL_DATABASE	Name of database to connect into.	Empty
MYSQL_PASSWORD	Password of user account for authentication.	Empty
MYSQL_USER	Username used for authentication.	Empty
MYSQL_MAX_CONNECTIONS	Max active connections to MySQL instance.	16

Refer to the mysql driver documentation for connection parameters.

SQLite

Environmental Variable	Description	Default
SQLITE_DB_PATH	Local filepath location for the paygate SQLite database.	paygate.db

Refer to the sqlite driver documentation for connection parameters.

Customers

Below is the configuration options for Customers (KYC, AML, OFAC) validation.

OFAC Refreshing

Environmental Variable	Description	Default
CUSTOMERS_OFAC_BATCH_SIZE	How many Originator and Receiver Customers to pull for refreshing.	100
CUSTOMERS_OFAC_REFRESH_EVERY	Interval for how often to refresh Customer OFAC searches	168h

Getting Help

channel	info
Project Documentation	Our project documentation available online.
Google Group moov-users	The Moov users Google group is for contributors other people contributing to the Moov project. You can join them without a google account by sending an email to moov-users+subscribe@googlegroups.com. After receiving the join-request message, you can simply reply to that to confirm the subscription.
Twitter @moov_io	You can follow Moov.IO's Twitter feed to get updates on our project(s). You can also tweet us questions or just share blogs or stories.
GitHub Issue	If you are able to reproduce a problem please open a GitHub Issue under the specific project that caused the error.
moov-io slack	Join our slack channel to have an interactive discussion about the development of the project.

Supported and Tested Platforms

• 64-bit Linux (Ubuntu, Debian), macOS, and Windows

Contributing

Yes please! Please review our Contributing guide and Code of Conduct to get started! Checkout our issues for first time contributors for something to help out with.

Paygate includes several "long" tests which spawn Docker containers, make requests over the internet, and perform more complicated tests. To skip these long tests add the -short flag to go test.

This project uses Go Modules and uses Go 1.14 or higher. See Golang's install instructions for help setting up Go. You can download the source code and we offer tagged and released versions as well. We highly recommend you use a tagged release for production.

Further Reading

• The default banking-as-a-service platform will be developer-first

Test Coverage

Improving test coverage is a good candidate for new contributors while also allowing the project to move more quickly by reducing regressions issues that might not be caught before a release is pushed out to our users. One great way to improve coverage is by adding edge cases and different inputs to functions (or contributing and running fuzzers).

Tests can run processes (like sqlite databases), but should only do so locally.

Each PR should increase the overall coverage, if possible. You can run make cover-test to save a coverage profile and make cover-web to open the HTML view in your default browser.

License

Apache License 2.0 See LICENSE for details.

Documentation

Overview

Package paygate will hold common types which are useful for working with paygate's HTTP endpoints.

Index

• Constants

Constants

const Version = "v0.8.0-rc5"

Version number