oracle

README

HarbourBridge: Oracle-to-Spanner Evaluation and Migration

HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation, using data from an existing Oracle database. This README provides details of the tool's Oracle capabilities. For general HarbourBridge information see this README.

We currently do not support dump file mode for Oracle. The only way to use HarbourBridge with OracleDB is connecting directly.

Note that 'oracle' can be used as an identifier with the flag -source in the command line.

Example Oracle DB Usage

HarbourBridge can be run directly on a Oracle database (via go's database/sql package).

The following examples assume a harbourbridge alias has been setup as described in the Installing HarbourBridge section of the main README.

Directly connecting to an Oracle database

In this case, HarbourBridge connects directly to the Oracle database to retrieve table schema and data. Set the -source=oracle and corresponding source profile connection parameters host, port, user, dbName and password.

For example, to perform schema conversion, run

harbourbridge schema -source=oracle -source-profile="host=<>,port=<>,user=<>,dbName=<>,password=<>"

In Oracle DB, USER is the account name, SCHEMA is the set of objects owned by that user. Oracle creates the SCHEMA object as part of the CREATE USER statement and the SCHEMA has the same name as the USER.

dbName will be the SID of the Database used. The Oracle System ID (SID) is used to uniquely identify a particular database on a system.

Schema Conversion

SQL_Server_Type	Spanner_Type
NUMBER (* , 0)	INT64
FLOAT	FLOAT64
BINARY_FLOAT	FLOAT64
BINARY_DOUBLE	FLOAT64
NUMBER (* , >0)	NUMERIC
CHAR	STRING(1)
NCHAR	STRING(N)
VARCHAR	STRING(MAX)
VARCHAR2	STRING(MAX)
NVARCHAR2	STRING(MAX)
CLOB	STRING(MAX)
NCLOB	STRING(MAX)
LONG	STRING(MAX)
ROWID	STRING(MAX)
UROWID	STRING(MAX)
DATE	DATE
TIMESTAMP	TIMESTAMP
BLOB	BYTES
BFILE	BYTES
RAW	BYTES
LONG RAW	BYTES
XMLTYPE	STRING(MAX)
INTERVAL YEAR	STRING(MAX)
INTERVAL DAY	STRING(MAX)
GEOMETRY	STRING(MAX)
JSON	JSON

Documentation

Overview

Package oracle handles schema and data migrations from oracle.

Index

• Variables
• func ProcessDataRow(conv *internal.Conv, tableId string, colIds []string, srcSchema schema.Table, ...)
• type InfoSchemaImpl
  • func (isi InfoSchemaImpl) GetColumns(conv *internal.Conv, table common.SchemaAndName, ...) (map[string]schema.Column, []string, error)
  • func (isi InfoSchemaImpl) GetConstraints(conv *internal.Conv, table common.SchemaAndName) ([]string, map[string][]string, error)
  • func (isi InfoSchemaImpl) GetForeignKeys(conv *internal.Conv, table common.SchemaAndName) (foreignKeys []schema.ForeignKey, err error)
  • func (isi InfoSchemaImpl) GetIndexes(conv *internal.Conv, table common.SchemaAndName, ...) ([]schema.Index, error)
  • func (isi InfoSchemaImpl) GetRowCount(table common.SchemaAndName) (int64, error)
  • func (isi InfoSchemaImpl) GetRowsFromTable(conv *internal.Conv, tableId string) (interface{}, error)
  • func (isi InfoSchemaImpl) GetTableName(dbName string, tableName string) string
  • func (isi InfoSchemaImpl) GetTables() ([]common.SchemaAndName, error)
  • func (isi InfoSchemaImpl) GetToDdl() common.ToDdl
  • func (isi InfoSchemaImpl) ProcessData(conv *internal.Conv, tableId string, srcSchema schema.Table, ...) error
  • func (isi InfoSchemaImpl) StartChangeDataCapture(ctx context.Context, conv *internal.Conv) (map[string]interface{}, error)
  • func (isi InfoSchemaImpl) StartStreamingMigration(ctx context.Context, client *sp.Client, conv *internal.Conv, ...) error
• type ToDdlImpl
  • func (tdi ToDdlImpl) ToSpannerType(conv *internal.Conv, spType string, srcType schema.Type) (ddl.Type, []internal.SchemaIssue)

Variables

var (
	TimestampReg = regexp.MustCompile(`TIMESTAMP`)
	IntervalReg  = regexp.MustCompile(`INTERVAL`)
)

Functions

func ProcessDataRow

func ProcessDataRow(conv *internal.Conv, tableId string, colIds []string, srcSchema schema.Table, spSchema ddl.CreateTable, vals []string)

Types

type InfoSchemaImpl

type InfoSchemaImpl struct {
	DbName        string
	Db            *sql.DB
	SourceProfile profiles.SourceProfile
	TargetProfile profiles.TargetProfile
}

func (InfoSchemaImpl) GetColumns

func (isi InfoSchemaImpl) GetColumns(conv *internal.Conv, table common.SchemaAndName, constraints map[string][]string, primaryKeys []string) (map[string]schema.Column, []string, error)

GetColumns returns a list of Column objects and names

func (InfoSchemaImpl) GetConstraints

func (isi InfoSchemaImpl) GetConstraints(conv *internal.Conv, table common.SchemaAndName) ([]string, map[string][]string, error)

GetConstraints returns a list of primary keys and by-column map of other constraints. Note: we need to preserve ordinal order of columns in primary key constraints. Note that foreign key constraints are handled in getForeignKeys.

func (InfoSchemaImpl) GetForeignKeys

func (isi InfoSchemaImpl) GetForeignKeys(conv *internal.Conv, table common.SchemaAndName) (foreignKeys []schema.ForeignKey, err error)

GetForeignKeys return list all the foreign keys constraints.

func (InfoSchemaImpl) GetIndexes

func (isi InfoSchemaImpl) GetIndexes(conv *internal.Conv, table common.SchemaAndName, colNameIdMap map[string]string) ([]schema.Index, error)

GetIndexes return a list of all indexes for the specified table. Oracle db support several types of index: 1. Normal indexes. (By default, Oracle Database creates B-tree indexes.) 2.Bitmap indexes 3.Partitioned indexes 4. Function-based indexes 5.Domain indexes, we are only considering normal index as of now.

func (InfoSchemaImpl) GetRowCount

func (isi InfoSchemaImpl) GetRowCount(table common.SchemaAndName) (int64, error)

GetRowCount with number of rows in each table.

func (InfoSchemaImpl) GetRowsFromTable

func (isi InfoSchemaImpl) GetRowsFromTable(conv *internal.Conv, tableId string) (interface{}, error)

GetRowsFromTable returns a sql Rows object for a table.

func (InfoSchemaImpl) GetTableName

func (isi InfoSchemaImpl) GetTableName(dbName string, tableName string) string

GetTableName returns table name.

func (InfoSchemaImpl) GetTables

func (isi InfoSchemaImpl) GetTables() ([]common.SchemaAndName, error)

func (InfoSchemaImpl) GetToDdl

func (isi InfoSchemaImpl) GetToDdl() common.ToDdl

GetToDdl function below implement the common.InfoSchema interface.

func (InfoSchemaImpl) ProcessData

func (isi InfoSchemaImpl) ProcessData(conv *internal.Conv, tableId string, srcSchema schema.Table, commonColIds []string, spSchema ddl.CreateTable, additionalAttributes internal.AdditionalDataAttributes) error

ProcessData performs data conversion for source database.

func (InfoSchemaImpl) StartChangeDataCapture

func (isi InfoSchemaImpl) StartChangeDataCapture(ctx context.Context, conv *internal.Conv) (map[string]interface{}, error)

StartChangeDataCapture is used for automatic triggering of Datastream job when performing a streaming migration.

func (InfoSchemaImpl) StartStreamingMigration

func (isi InfoSchemaImpl) StartStreamingMigration(ctx context.Context, client *sp.Client, conv *internal.Conv, streamingInfo map[string]interface{}) error

StartStreamingMigration is used for automatic triggering of Dataflow job when performing a streaming migration.

type ToDdlImpl

type ToDdlImpl struct {
}

ToDdlImpl oracle specific implementation for ToDdl.

func (ToDdlImpl) ToSpannerType

func (tdi ToDdlImpl) ToSpannerType(conv *internal.Conv, spType string, srcType schema.Type) (ddl.Type, []internal.SchemaIssue)

ToSpannerType maps a scalar source schema type (defined by id and mods) into a Spanner type. This is the core source-to-Spanner type mapping. toSpannerType returns the Spanner type and a list of type conversion issues encountered.