doltgresql

README

DoltgreSQL is Dolt for Postgres!

From the creators of Dolt, the world's first version controlled SQL database, DoltgreSQL (aka Doltgres) is a Postgres-flavored version of Dolt. Doltgres offers all the Git-style log, diff, branch, and merge functionality of Dolt on your Postgres database schema and data. Instead of connecting with a MySQL client and using MySQL SQL, you connect to Doltgres with a Postgres client and use Postgres SQL. Doltgres is like Git and Postgres had a baby.

Motivation

Dolt was built MySQL-flavored. There is no MySQL code in Dolt. In 2019, when we were conceiving of Dolt, MySQL was the most popular SQL-flavor. Over the past 5 years, the tide has shifted more towards Postgres, especially among young companies, Dolt's target market. Potential customers have been clamoring for a Postgres version of Dolt.

Moreover, Dolt was conceived of and built as Git for Data. Dolt later became a version controlled database. DoltgreSQL gives us the opportunity to strip out some of the Git for Data pieces like the CLI and build directly for the version controlled database use case. With Doltgres, you start a server, connect a client, and do everything with SQL, a familiar experience for Postgres users.

Doltgres will diverge from Dolt over time to be a focused database version control solution. That said, we have a five year head start with Dolt. Dolt is a production-grade version controlled database today. Dolt is 1.0. If you are ok with using a MySQL-client, we recommend using Dolt for all use cases. Doltgres is experimental.

How You Can Help

Doltgres is experimental. We need your feedback to understand how much we should invest in it. If you are interested in using Doltgres now or in the future, please:

• Star this repo to tell us you are interested.
• Try Doltgres
• Create issues if you find bugs
• Create issues for missing functionality you want
• Contribute code for features you want (see the Contribution Guide)

Getting Started

1. Download the latest release of doltgres

2. Put doltgres on your PATH

3. Run doltgres. This will create a doltgres user and a doltgres database in ~/doltgres/databases (add the --data-dir argument or change the DOLTGRES_DATA_DIR environment variable to use a different directory).

$ doltgres
Successfully initialized dolt data repository.
Starting server with Config HP="localhost:5432"|T="28800000"|R="false"|L="info"|S="/tmp/mysql.sock"

1. Make sure you have Postgres version 15 or higher installed. I used Homebrew to install Postgres on my Mac. This requires I manually add /opt/homebrew/opt/postgresql@15/bin to my path. On Postgres version 14 or lower, \ commands (ie. \d, \l) do not yet work with Doltgres.

export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"

1. Open a new terminal. Connect with the following command: psql -h localhost -U doltgres. This will connect to the doltgres database with the doltgres user.

$ psql -h 127.0.0.1 -U doltgres
psql (15.4 (Homebrew), server 15.0)
Type "help" for help.

doltgres=>

1. Create a getting_started database. Create the getting_started example tables.

doltgres=> create database getting_started;
--
(0 rows)

doltgres=> \c getting_started;
psql (15.4 (Homebrew), server 15.0)
You are now connected to database "getting_started" as user "doltgres".
getting_started=> create table employees (
    id int8,
    last_name text,
    first_name text,
    primary key(id));
--
(0 rows)

getting_started=> create table teams (
    id int8,
    team_name text,
    primary key(id));
--
(0 rows)

getting_started=> create table employees_teams(
    team_id int8,
    employee_id int8,
    primary key(team_id, employee_id),
    foreign key (team_id) references teams(id),
    foreign key (employee_id) references employees(id));
--
(0 rows)

getting_started=> \d
              List of relations
 Schema |      Name       | Type  |  Owner   
--------+-----------------+-------+----------
 public | employees       | table | postgres
 public | employees_teams | table | postgres
 public | teams           | table | postgres
(3 rows)

1. Make a Dolt Commit.

getting_started=> select * from dolt_status;
   table_name    | staged |  status   
-----------------+--------+-----------
 employees       | 0      | new table
 employees_teams | 0      | new table
 teams           | 0      | new table
(3 rows)

getting_started=> call dolt_add('teams', 'employees', 'employees_teams');
 status 
--------
      0
(1 row)
getting_started=> select * from dolt_status; 
   table_name    | staged |  status   
-----------------+--------+-----------
 employees       | 1      | new table
 employees_teams | 1      | new table
 teams           | 1      | new table
(3 rows)

getting_started=> call dolt_commit('-m', 'Created initial schema');
               hash               
----------------------------------
 peqq98e2dl5gscvfvic71e7j6ne34533
(1 row)

1. View the Dolt log.

getting_started=> select * from dolt_log;
           commit_hash            | committer |       email        |        date         |          message           
----------------------------------+-----------+--------------------+---------------------+----------------------------
 peqq98e2dl5gscvfvic71e7j6ne34533 | doltgres  | doltgres@127.0.0.1 | 2023-11-01 22:08:04 | Created initial schema
 in7bk735qa6p6rv6i3s797jjem2pg4ru | timsehn   | tim@dolthub.com    | 2023-11-01 22:04:03 | Initialize data repository
(2 rows)

1. Continue with Dolt Getting Started to test out more Doltgres versioning functionality.

Building From Source

Please follow the Contributor's Guide to learn how to build from source.

Limitations

• No Git-style CLI for version control like in Dolt, only a SQL interface.
• Can't push to DoltHub or DoltLab, only custom remotes.
• Limited support of Postgres-specific types and functions.
• No Postgres system tables.
• No authentication or users.
• Database and schema models are merged.
• Limited support for SSL connections (non-verified connections only).
• No GSSAPI support.
• No PostgreSQL functions have been implemented, therefore only MySQL functions may be used.
• No support for replication, clustering, etc.

Performance

Dolt is 1.7X slower than MySQL as measured by a standard suite of Sysbench tests.

We use these same Sysbench tests to benchmark DoltgreSQL and compare the results to PostgreSQL.

Here are the benchmarks for DoltgreSQL version 0.3.1.

Read Tests	PostgreSQL	DoltgreSQL	Multiple
oltp_point_select	0.13	0.54	4.2
oltp_read_only	2.35	12.75	5.4
select_random_points	0.2	1.03	5.1
select_random_ranges	0.41	1.01	2.5
reads_mean_multiplier			4.3

Write Tests	PostgreSQL	DoltgreSQL	Multiple
oltp_insert	0.8	3.02	3.8
oltp_read_write	3.96	20.0	5.1
oltp_update_index	0.86	3.19	3.7
oltp_update_non_index	0.8	3.07	3.8
oltp_write_only	1.37	7.56	5.5
writes_mean_multiplier			4.4

Overall Mean Multiple	4.3

Correctness

Dolt is 99.99% compatible with MySQL based on a standard suite of correctness tests called sqllogictest.

We use these same tests to measure the correctness of DoltgreSQL.

Here are DoltgreSQL's sqllogictest results for version 0.3.1. Tests that did not run could not complete due to a timeout earlier in the run.

Results	Count
did not run	411
not ok	751738
ok	4927605
timeout	1

Total Tests	5679755

Correctness Percentage	86.757351

Architecture

Doltgres emulates a Postgres server, including parsing Postgres SQL into an Abstract Syntax Tree (AST). This AST is converted to a form that can be interpreted by the Dolt engine. Doltgres uses the same SQL engine and storage format as Dolt.

Dolt has a unique architecure that allows for version control features at OLTP database performance. Doltgres uses the same architecture.