gravity

module
v0.9.54 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 4, 2019 License: Apache-2.0

README

Gravity 简体中文


Build Status

2.0 Product

Gravity is used to replicate data between various inputs and outputs (databases, message queues).

It is designed to be a customizable data replication tool that:

  • Supports multiple sources and destinations

  • Supports Kubernetes-based replication cluster

Use Cases

  • Data Bus: Use Change Data Capture (MySQL binlog, MongoDB Oplog) and batch table scan to publish data to message queue like Kafka.
  • Unidirectional data replication: Replicates data from one MySQL cluster to another MySQL cluster.
  • Bidirectional data replication: Replicates data between two MySQL clusters bidirectionally.
  • Synchronization of shards to the merged table: Synchronizes MySQL sharded tables to the merged table. You can specify the corresponding relationship between the source table and the target table.
  • Online data mutation: Data can be changed during the replication. For example, rename the column, encrypt/decrypt data columns.

Features

Input support
Input Status
MySQL Binlog
MySQL Scan
Mongo Oplog
Mongo Scan
TiDB Binlog Doing
PostgreSQL WAL Doing
output support
Output Status
MySQL/TiDB
Kafka
Elasticsearch Beta
Data mutation support

Gravity supports the following data mutations:

  • Ignore specific data
  • Renaming columns
  • Deleting columns
Documentation

Special thanks to @dantin, @LiangShang, and @liwink for the early support to this project, and thanks @CaitinChen for the initial translation of the documentation.

Directories

Path Synopsis
cmd
dcp
dcp
mock
binlog_checker
Package mock_binlog_checker is a generated GoMock package.
Package mock_binlog_checker is a generated GoMock package.
position_store
Package mock_position_store is a generated GoMock package.
Package mock_position_store is a generated GoMock package.
sliding_window
Package mock_sliding_window is a generated GoMock package.
Package mock_sliding_window is a generated GoMock package.
pkg
app
protocol/dcp
Package dcp is a generated protocol buffer package.
Package dcp is a generated protocol buffer package.
protocol/msgpb
Package msgpb is a generated protocol buffer package.
Package msgpb is a generated protocol buffer package.
protocol/tidb
Package slave_binlog is a generated protocol buffer package.
Package slave_binlog is a generated protocol buffer package.
sarama_cluster
package kafka provides cluster extensions for Sarama, enabing users to consume topics across from multiple, balanced nodes.
package kafka provides cluster extensions for Sarama, enabing users to consume topics across from multiple, balanced nodes.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL