mobroute

README

Mobroute

Mobroute is a general purpose FOSS public transportation router (e.g. trip planner) Go library and CLI that works by directly ingesting timetable (GTFS) data from transit agencies themselves (sourced from the Mobility Database). After data has been fetched, routing calculations can be run offline, and locally / on-device (as opposed to making a network call). Overall, Mobroute aims to offer an opensource framework (both CLI & library) for integrating data-provider-agnostic GTFS public transit capabilities (integrated GTFS ETL, GTFS multisource support, and routing algorithm) into applications to get users from point-a to point-b via public transit without comprising privacy or user freedoms.

In addition to the Mobroute Go library & CLI, the related subproject, the Transito app offers fully integrated routing functionality on mobile devices (Android & Linux) utilizing Mobroute's Go library.

The Mobroute & Transito projects overall are currently in active development but generally currently usable on mobile & via the CLI and looking for more users & testers. Mobroute & Transito work in many well-connected metros which have publicly available GTFS data, to name a few: Lisbon, NYC, Brussels, Krakow, and Bourges. You can see sample routes at the automated test results page, check for your metro here, and see details on the Transito mobile app here.

Documentation

• Documentation Website: Documentation Website for Mobroute
• Mobroute Go Library Doc: Go Library Documentation, Go Library Examples
• Mobroute CLI Doc: CLI Userguide, CLI Documentation
• Mobroute Doc: Diagrams, GTFS Feeds, Routing Tunables, Routing Output Formats, Development Guide
• Project Management: Releases, Bugtracker, Mailing List
• CI: Test Results (master), Test Results (by commit), Builds
• Related: Transito (Mobile App), Mobsql (GTFS-to-SQLite ETL library)

Project Overview / Featureset

• Well-executed singlemodal public transportation routing via GTFS data
  • Project focuses on implementing only GTFS routing providing for algorithmic & codebase simplicity
  • Other routers bill themselves as "multimodal"; Mobroute is proudly singlemodal (doesn't integrate automobile/OSM/PBF data)
  • Overall.. does one thing well: public transportation routing
• Integration of automatic GTFS dataimport & ETL out-of-the-box via the Mobility Database
  • Frees user from having to source & manually download/import GTFS zip archives
  • System automatically downloads the data needed for a routing requests upon use
  • Supports using multiple GTFS sources independently and in combination for single routing requests
  • See Mobsql for details on ETL pipeline, uses SQLite under the hood
• Efficient algorithmic routing via the Connection Scan Algorithm (CSA)
  • Being a singlemodal router lends Mobroute's design to implementation simplicity
  • The well academically studied CSA algorithm is utilized: see paper details here
  • CSA is non-graphbased, simple, and Mobroute's CSA implementation is well tested
• Targeted to on-device and offline usage
  • Designed to be used on-device and offline once data is fetched
  • Run your routing request right on your mobile device for the privacy conscious
• Fully thought-out small, simple, & modular design supporting library, CLI, & mobile usage
  • Built in Go, the design of the project is small and modular
  • 3 primary components:
    • Mobsql (GTFS-to-SQLite ETL pipeline: CLI + Go Library)
    • Mobroute (Core routing algorithm via CSA: CLI + Go Library)
    • Transito (Graphical mobile application for Android & Linux)
  • Mobsql and Mobroute can be used as both a CLI OR as a Go library in your application
  • Transito, Mobroute's mobile application is written in Go (via Gio) and uses Mobroute's Go Library

Funding

This project is funded through NGI0 Entrust, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program. Learn more at the NLnet project page.

Documentation

Overview

Package mobroute is a library that allows you to load GTFS data, query this data for stops & schedules, and perform routing requests against a single or multiple feeds. The library can be used to add public routing functionality to existing map & navigation apps or for assistance in bootstrapping new public transit routing projects for integrating GTFS data. GTFS ingestion functionality is built-in and the library can automatically import feeds from the Mobility Database specified by feed ID or a custom GTFS archive can be provided. The core routing logic is based on the Connection-Scan-Algorithm. GTFS data is stored in a local SQLite database file and the library is geared toward on-device and offline usage.

All public functionality is exposed through the single mobroute (git.sr.ht/~mil/mobroute) package. Functionality outside of the top-level mobroute package should be considered to be implementation details, unstable, and not used by end consumers. Note that types are aliased in this package so you should look at the underlying reference types for field members etc. However if a type is unexported that means it is subject to change in future updates.

The library offers the following primary functionalities:

• Load GTFS data (either via Mobility Database feed ID or via a user-specified file:// or http:// URI) to a local SQLite DB.
• Compute GTFS-derived tables optimized for schedule & routing calculations.
• Lookup GTFS stops near a certain geocoordinate or query all stops for a given feed.
• Lookup GTFS schedules (e.g. upcoming departure times) for a given set of stops for a given set of feeds.
• Perform routing calculations via the Connection-Scan-Algorithm based on input GTFS data with various tunable properties.

Building

Ensure sqlite-dev is installed and Go >1.21

Also note: when building code that imports this package, you must pass `-tags=sqlite_math_functions` to the `go build` command as this enables the underlying sqlite library to utilize math functions which Mobroute's core logic depends on. Example for build with tags below.

Import via:

import "git.sr.ht/~mil/mobroute/v0.9.0"

Build via:

go build -tags=sqlite_math_functions foo.go

Overview of Functions

There are six public functions exposed from the library as a whole; and understanding each is equivalent to grokking the library as a whole. All functions rely on a MobrouteRuntime which can be created with RTInitialize. The 6 public functions are:

1. RTInitialize: Initialize a MobrouteRuntime runtime which pools a common SQLite database connection and is used as a param for all other library functions.
2. RTDatabase: Executes database RW manipulation allowing for: loading GTFS archives (via Mobility Database or custom URIs), creating computed tables, removing GTFS archives from DB, and more.
3. RTDatabaseq: Queries database RO to determine GTFS feed loaded status (resulting from RTDatabase) & shows metadata for unloaded and loaded feeds.
4. RTStops: Queries database for GTFS stops near a given geocoordinate (or all stops).
5. RTSchedule: Queries database for upcoming schedule (departures) for a given set of stop UIDs.
6. RTRoute: Performs point-to-point routing calculations via the Connection-Scan-Algorithm based on GTFS data extracted from database queries.

The general usage pattern for using Mobroute as an library is:

• (a) Use RTInitialize to create a MobrouteRuntime.
• (b) Load a GTFS feed into the SQLite DB via using RTDatabase with the `loadmdbgtfs` or `loadcustomgtfs` op.
• (c) Then, compute GTFS-derived tables for the feed via using RTDatabase with the `compute` op.
• (d) Finally, query database state with RTDatabaseq, query stops with RTStops, query schedules with RTSchedule, or perform routing calculations via CSA via RTRoute.

Please reference the below examples for general usage in which you'll find this same pattern.

Standalone (Runnable) Examples

Within the `examples/` folder there are a number of standalone runnable examples. You can run each of these examples by using go run; note to make sure to specify the sqlite_math_functions build tag.

go run -tags=sqlite_math_functions examples/route/main.go

Currently implemented examples are as follows:

• Load Mobility Database GTFS Feed Example: Load a Mobility Database GTFS feed to the local SQLite DB.
• Update Mobility Database GTFS Feed Example: Update a Mobility Database GTFS feed to the local SQLite DB.
• Load Custom GTFS Feed Example: Load a custom GTFS feed (either file:// or http:// URI) to the local SQLite DB.
• Route Example: Perform a routing request using the Connection-Scan-Algorithm (based on loaded GTFS data).
• Stops Example: Query stops (based on loaded GTFS data).
• Schedule Example: Query upcoming departures (based on a loaded GTFS data).

The recommended way to get started with library usage is to simply run & adapt these examples to your needs.

Additional Documentation & Resources

• Mobroute Documentation Website: Central documentation website
• System Diagrams: Diagrams which can be very helpful for understanding general usage pattern & system architecture
• Transito: A mobile app (Android & Linux) via Gio which uses Mobroute's Go library and can serve as a reference implementation.
• Mobroute CLI: Provides a barebones reference implementation for all functionality offered by the library (and can also be helpful for debugging).

Index

• type DatabaseParams
• type DatabaseResponse
  • func RTDatabase(runtime *MobrouteRuntime, params *DatabaseParams) (*DatabaseResponse, error)
• type DatabaseqParams
• type DatabaseqResponse
  • func RTDatabaseq(runtime *MobrouteRuntime, params *DatabaseqParams) (*DatabaseqResponse, error)
• type FeedStatusInfo
• type FeedsearchFilter
• type MobrouteRuntime
  • func RTInitialize(params *MobrouteRuntimeConfig) (*MobrouteRuntime, error)
• type MobrouteRuntimeConfig
• type MobsqlRuntimeConfig
• type RouteDiagnostics
• type RouteLeg
• type RouteParams
• type RouteResponse
  • func RTRoute(runtime *MobrouteRuntime, params *RouteParams) (*RouteResponse, error)
• type RouteStopDetails
• type ScheduleDeparture
• type ScheduleParams
• type ScheduleResponse
  • func RTSchedule(runtime *MobrouteRuntime, params *ScheduleParams) (*ScheduleResponse, error)
• type StopsParams
• type StopsResponse
  • func RTStops(runtime *MobrouteRuntime, params *StopsParams) (*StopsResponse, error)

Types

type DatabaseParams

type DatabaseParams = apirtdatabase.DatabaseParams

DatabaseParams holds parameters for the given RTDatabase request. The RTDatabase function and the options in these params are a thin passthrough to Mobsql (which should be referenced for in-depth doc on functionality).

FeedIDs and Op are always required.

Op should be one of

• loadmdbgtfs: Load the GTFS for the feed IDs from the Mobility Database to the SQLite DB.
• loadcustomgtfs: Load the GTFS for the provided URI to the local SQLite DB.
• compute: Create computed tables entries for the given feed IDs.
• purge_gtfs: Remove origin GTFS tables for the given feed IDs.
• purge_computed: Remove computed tables for the given feed IDs.
• purge_all: Remove both GTFS & computed tables entries for the given feed IDs.

FeedIDs are the Mobility Database feed ids (or custom feed ids). In the case of using a custom feed ID (for loadcustomgtfs), the feed ID must be user-specified *negative* integer. For more information see the Feed IDs doc.

LoadCustomGTFSURI is only used to define the URI when using loadcustomgtfs op and may refer to either an HTTP(s) (http://) URL for a remote GTFS archive or a File URI (file://) for a local GTFS archive to load.

LoadMDBGTFSUpdate is only used when using the loadmdbgtfs op and determines whether the feed should be force 'full updated'. This involves re-downloading the Mobility DB prior to GTFS load & ignoring all caching systems. This is helpful in the case of feed updates.

LoadMDBGTFSDirectURLs is only used when using the loadmdbgtfs op and determines whether the 'direct' agency URL should be used when downloading GTFS archives. By default / if false, a cloud-mirrored GTFS URL (provided by Mobility Database's CI is used). The default cloud mirrored URL is more 'reliable' but can go out-of-date.

Important Note: After running a `loadmdbgtfs` or `loadcustomgtfs` op, in most cases you will want to run a `compute` op. Computed data is based on origin GTFS data and each time updating GTFS data you should re-compute derived tables with `compute` op subsequently.

type DatabaseResponse

type DatabaseResponse = apirtdatabase.DatabaseResponse

Databaseq holds the response for the given RTDatabase function. Contains information about whether the given RTDatabase call was successful and contains metadata about effected feeds.

DatabaseResponse is an alias of git.sr.ht/~mil/mobsql.FeedOpResult. Please see underlying type for further details.

func RTDatabase

func RTDatabase(runtime *MobrouteRuntime, params *DatabaseParams) (*DatabaseResponse, error)

RTDatabase executes database RW manipulation allowing for: loading GTFS archives (via Mobility Database or custom URIs), creating computed tables, removing GTFS archives from DB, and more. Takes DatabaseParams with given database manipulation request and returns DatabaseResponse which indicates success or failure. Under-the-hood, this is a thin passthrough to the major RW functionality in Mobsql, allowing the user to manipulate the underlying database.

Note, the related RTDatabaseq allows for querying (RO) functionality on the underlying database.

type DatabaseqParams

type DatabaseqParams = apirtdatabaseq.DatabaseqParams

DatabaseqParams holds parameters for the given RTDatabaseq request. There are two fields (FeedIDs and FeedSearchFilter) which may be specified which are used in an either/xor-fashion to determine the feeds to query the database state of.

• FeedIDs: Allows you to specify particular feed IDs to query the database status of.
• FeedSearchFilter: Allows you to specify a 'search filter' to determine feeds to query the database state of. Using the feed search filter allows you to build out 'search' interfaces to match the loaded Mobility Database for feed ID matches (e.g. see underlying struct Glob field and similar).

type DatabaseqResponse

type DatabaseqResponse = apirtdatabaseq.DatabaseqResponse

DatabaseqResponse holds the response for the given RTDatabaseq function. Contains information about each feed queried.

This is an alias of the underlying FeedStatusInfo. Please see underlying type for further details.

func RTDatabaseq

func RTDatabaseq(runtime *MobrouteRuntime, params *DatabaseqParams) (*DatabaseqResponse, error)

RTDatabaseq queries database RO to determine GTFS feed loaded status (resulting from RTDatabase) & shows metadata for unloaded and loaded feeds. Takes the query params in DatabaseqParams and returns resulting set of feed data in DatabaseqResponse. This is an RO operation as opposed to RTDatabase which may manipulate the database as RW. This function is helpful for querying whether a load operation, or compute operation was successful and also determining the current database state and the state of GTFS feeds. Under-the-hood, this is a thin passthrough to the major RO functionality in Mobsql, allowing the user to query the underlying database. See DatabaseqParams for details on params.

Note, the related RTDatabase allows for manipulation (RW) functionality on the underlying database.

type FeedStatusInfo

type FeedStatusInfo = apirtdatabaseq.FeedStatusInfo

FeedStatusInfo contains underlying info about a feed returned from RTDatabaseq

This is an alias of the underlying git.sr.ht/~mil/mobsql.FeedStatusInfo. Please see underlying type for further details.

type FeedsearchFilter

type FeedsearchFilter = apirtdatabaseq.FeedsearchFilter

FeedsearchFilter contains a filter specification that can specify a search on metadatafields from the Mobility Database.

This is an alias of the underlying git.sr.ht/~mil/mobsql.FeedsearchFilter Please see underlying type for further details.

type MobrouteRuntime

type MobrouteRuntime = apirtinit.MobrouteRuntime

MobrouteRuntime contains the database connection and the git.sr.ht/~mil/mobsql.MobsqlRuntime. This may also contain other properties relevant for global Mobroute operation in the future. The database connection points to a local SQLite file and the Mobsql runtime connection generally will reference the same DB file. The reason for storing the DB separately from Mobsql's runtime (and not just using Mobsql's own DB connection) is that Mobroute itself uses the DB field for RO operations only; all RW functionality happens through Mobsql / the MobsqlRuntime.

This struct is passed to all other library functions and should be initialized by using RTInitialize.

func RTInitialize

func RTInitialize(params *MobrouteRuntimeConfig) (*MobrouteRuntime, error)

RTInitialize initializes a new MobrouteRuntime which contains a passthrough to the git.sr.ht/~mil/mobsql.MobsqlRuntime, the SQLite DB connection, and some additional metadata. On initialization this function: bootstraps the DB (e.g. creating SQL tables, views, indexes), create Mobsql connection / initialization, and fetches the Mobility Database CSV and imports to DB.

All other library functionality depend on the returned MobrouteRuntime.

type MobrouteRuntimeConfig

type MobrouteRuntimeConfig = apirtinit.MobrouteRuntimeConfig

MobrouteRuntimeConfig contains the the underlying git.sr.ht/~mil/mobsql/MobsqlRuntimeConfig and potentially in the future some other Mobroute-specific properties for the common DB connection and Mobroute params config. This is used by RTInitialize and you can use this struct to configure various properties around logging and Mobsql initialization.

The Logger field can be very helpful in redirecting log messages to see more verbose output for functions then just the returned type / error.

An empty configuration (e.g. nil values) for all properties is entirely valid and defaults will be configured in that case.

type MobsqlRuntimeConfig

type MobsqlRuntimeConfig = apirtinit.MobsqlRuntimeConfig

MobsqlRuntimeConfig alias for git.sr.ht/~mil/mobsql.RuntimeConfig. In most normal cases where this is seen in MobrouteRuntimeConfig this should just be left as nil. The case when it makes sense to configure underlying the underlying Mobsql config is if you are interested in changing logging and other underlying schema properties for Mobsql. See underlying origin git.sr.ht/~mil/mobsql.RuntimeConfig doc for more details.

type RouteDiagnostics

type RouteDiagnostics = apirtroute.RouteDiagnostics

RouteDiagnostics contains diagnostic information for a routing request measuring number of seconds for specific parts of the pipeline, memory usage, and meta route information.

• The PerfS fields indicate the amount of time taken for each stage.
• The PerfNMem fields indicate the length of the arrays for each SQL query type extracted
• The PerfOptimizedFeeds fields indicated which feed IDs were actually queries (as this is optimized earlier in the pipeline).
• The Routeinfo fields contain various calculated route metalevel diagnostics.

type RouteLeg

type RouteLeg = apirtroute.RouteLeg

RouteLeg represents a 'leg' in a route which may be one of: transfer, walk, or trip type (see the LegType field).

When the LegType is equal to "trip", all fields prefixed with Trip* will be non-nil / present. A trip leg represents one boarding of a segment of a trip (e.g. a single bus route, or a single train route etc.). Underlying this is flagged by a single trip ID sourced from the stop_times GTFS table.

When the LegType is equal to "walk", all fields prefixed with Walk* will be non-nil / present. A walk leg represents the segment of time walking to the origin station (e.g. with the first trip); or walking from the destination station (e.g. with the last trip). A walk has a correlated distance KM and the walk to/from fields will contain the relevant stop IDs. The walk KM is automatically calculated from the input constraints for walk km / hr.

When the LegType is equal to "transfer", all fields prefixed with Transfer* will be non-nil / present. A transfer leg represents the segment of time to get from one trip leg to another trip leg. Transfers can be sourced from a number of places (e.g. either implicit, feed, or generated) and always have an associated duration. The field TransferDdwell indicates the amount of inactive time in the transfer (waiting / dwelling); while TransferActive indicates active time walking getting from one stop to another.

type RouteParams

type RouteParams = apirtroute.RouteParams

RouteParams contains the parameters for a given route request for RTRoute. The FeedIDs, From, To, TransferCategories, and Output format fields are required. Other fields are optional and will be filled in with default values if unset.

• Extended information about params (tunables) are available in the Tunables Doc.
• Extended information about output formats are available in the Output Formats Doc.

type RouteResponse

type RouteResponse = apirtroute.RouteResponse

RouteResponse is the response for a successful RTRoute routing request.

Each field may optionally be present depending on if RouteParams's OutputFormats is set for the associated output format.

The Output Formats Doc has extended information on each output format, a basic overview of each format is as follows:

• RouteLegs: Contains an array of 'legs' for each segment of a calculated trip.
• GeoJSON: GeoJSON created from the RouteLegs, less information then legs but simpler to integrate with other apps that support GeoJSON.
• MapURL: A link to the GeoJSON embeded in hash on geojson.lrdu.org.
• Request: Origin RouteParams directly embedded for simpler testing and replication of request.
• RouteDiagnostics: The RouteDiagnostics contains some simple diagnostics information about the request.
• Connections: Array of raw git.sr.ht/~mil/mobroute/dbt.ConnectionVerbose that were used for the input to the CSA algorithm. This can be helpful for spot debugging.

func RTRoute

func RTRoute(runtime *MobrouteRuntime, params *RouteParams) (*RouteResponse, error)

RTRoute performs point-to-point routing calculations via the Connection Scan Algorithm based on GTFS data extracted from database queries. Effected by routing constraints defined in RouteParams and returns a RouteResponse. The entire routing flow from extracting from the DB the required data for CSA, to running the CSA algorithm, to going back to the DB to pull a few extra fields for presentation, and finally formatting the data into route legs for easy consumption is handled by this function. The underlying methodology used for for routing can be understood by reading the Connection Scan Algorithm paper which this function is an implementation of. The routing dataflow pattern diagram can be helpful in understanding overall dataflow.

Note: provided feed IDs in RouteParams should first be loaded and computed via RTDatabase.

type RouteStopDetails

type RouteStopDetails = apirtroute.RouteStopDetails

RouteStopDetails represents a stop within a trip. This allows more fine-grained detailed into each 'stop' in a common trip. Ultimately this correlates to a single connection in the table _ctconn and can be referenced by using the field ConnOID.

type ScheduleDeparture

type ScheduleDeparture = apirtschedule.ScheduleDeparture

ScheduleDeparture represents a single departure (correlating under-the-hood) to an entry in the GTFS stop_times table. This type contains some extra information about each departure as well such as the route short / long name, trip headsign, and departure time. Departure times are provided as UTC.

type ScheduleParams

type ScheduleParams = apirtschedule.ScheduleParams

ScheduleParams defines the parameters for the RTSchedule function. The fields FeedIDs, StopUIDs, and MaxSeconds are required. Note that StopUIDs may be determined using the RTStops function.

The optional Time field allows you define a custom departure time; if unset it is assumed to use the current system time (now) for departure. The MaxSeconds field is bound to the given time.

type ScheduleResponse

type ScheduleResponse = apirtschedule.ScheduleResponse

ScheduleResponse represents the response for the RTSchedule function containing upcoming departures. See the field Schedule which is an array of ScheduleDeparture indicating potential departures.

func RTSchedule

func RTSchedule(runtime *MobrouteRuntime, params *ScheduleParams) (*ScheduleResponse, error)

RTSchedule queries the database for upcoming schedule departures for a given set of stop UIDs. Takes the departure schedule constraints defined in ScheduleParams and returns departures within the timeframe in a ScheduleResponse. Note: the logic used in this function handles timezones & operates identically to routing calls in terms of schedule extraction. This function is a good way to determine upcoming departures for a given set of stops. The stop UIDs may be determined using the RTStops function.

Note: provided feed IDs in ScheduleParams should first be loaded and computed via RTDatabase.

type StopsParams

type StopsParams = apirtstops.StopsParams

StopsParams contains parameters for the RTStops function. This struct contains the constraints that will be converted to a SQL query to match stops in the DB. Note that multiple feed IDs can be queried at the same time as returned stops have stop UIDs. The only required field is FeedIDs and all other fields are optional.

If FromCoord is set, MaxDistKm should be set as well as this will limit returned stops to only be within the distance specified from the coordinate.

Glob is a wildcard that will text match against stop names and filter as such.

NLimit limits the number of returned stops to be the maximum of this value.

type StopsResponse

type StopsResponse = apirtstops.StopsResponse

StopsResponse is the response for the RTStops function that contains the matched stops. See the field Stops which contains an array of git.sr.ht/~mil/mobroute/dbt.StopWalk.

func RTStops

func RTStops(runtime *MobrouteRuntime, params *StopsParams) (*StopsResponse, error)

RTStops queries database for GTFS stops near a given geocoordinate or all stops. Takes the constraints in StopsParams and returns the matched stops as StopsResponse. This function can be very helpful both to build out a 'nearby stops' functionality in end-applications or also the return stop UIDs may be used by RTSchedule which can show upcoming departures for each stop.

Note: provided feed IDs in StopsParams should first be loaded and computed via RTDatabase.