Documentation ¶
Overview ¶
Copyright © 2020 Marvin
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2020 Marvin ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2020 Marvin ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2020 Marvin ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2020 Marvin ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2020 Marvin ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Index ¶
- Constants
- Variables
- func CharsetConvert(data []byte, fromCharset, toCharset string) ([]byte, error)
- func DiffStructArray(structA, structB interface{}) ([]interface{}, []interface{}, bool)
- func FilterDifferenceStringItems(originItems, excludeItems []string) []string
- func FilterIntersectionStringItems(originItems, newItems []string) []string
- func GetStringSliceElementIndex(items []string, item string) int
- func IsContainString(items []string, item string) bool
- func IsEmptyString(str string) bool
- func IsNum(s string) bool
- func IsSubsetString(originItems, checkItems []string) (bool, []string)
- func PathExist(path string) error
- func ReSplit(text string, cut string) []string
- func ReplaceQuotesString(s string) string
- func ReplaceSpecifiedString(s string, oldStr, newStr string) string
- func SpecialLettersUsingMySQL(bs []byte) string
- func SpecialLettersUsingOracle(bs []byte) string
- func SplitMultipleSlice(arr []interface{}, num int64) [][]interface{}
- func SplitMultipleStringSlice(arr [][]string, num int64) [][][]string
- func StrconvFloatBitSize(s string, bitSize int) (float64, error)
- func StrconvIntBitSize(s string, bitSize int) (int64, error)
- func StrconvRune(s string) (int32, error)
- func StrconvUintBitSize(s string, bitSize int) (uint64, error)
- func StringArrayToCapitalChar(strs []string) string
- func StringJOIN(strs []string, strPrefix, strSuffix, joinS string) string
- func StringUPPER(str string) string
- func StringsBuilder(str ...string) string
- func TrimLastChar(s string) string
- func VersionOrdinal(version string) string
Constants ¶
const ( AssessYesCompatible = "Y" AssessNoCompatible = "N" AssessYesConvertible = "Y" AssessNoConvertible = "N" AssessYesEquivalent = "Y" AssessNoEquivalent = "N" )
Assess Status
const ( AssessTypeDatabaseOverview = "DATABASE_OVERVIEW" AssessTypeObjectTypeCompatible = "OBJECT_TYPE_COMPATIBLE" AssessTypeObjectTypeCheck = "OBJECT_TYPE_CHECK" AssessTypeObjectTypeRelated = "OBJECT_TYPE_RELATED" )
Assess Type
const ( AssessNameDBOverview = "DB_OVERVIEW" AssessNameTableTypeCompatible = "TABLE_TYPE_COMPATIBLE" AssessNameColumnTypeCompatible = "COLUMN_TYPE_COMPATIBLE" AssessNameConstraintTypeCompatible = "CONSTRAINT_TYPE_COMPATIBLE" AssessNameIndexTypeCompatible = "INDEX_TYPE_COMPATIBLE" AssessNameDefaultValueCompatible = "DEFAULT_VALUE_COMPATIBLE" AssessNameViewTypeCompatible = "VIEW_TYPE_COMPATIBLE" AssessNameObjectTypeCompatible = "OBJECT_TYPE_COMPATIBLE" AssessNamePartitionTypeCompatible = "PARTITION_TYPE_COMPATIBLE" AssessNameSubPartitionTypeCompatible = "SUBPARTITION_TYPE_COMPATIBLE" AssessNameTemporaryTableTypeCompatible = "TEMPORARY_TABLE_TYPE_COMPATIBLE" AssessNamePartitionTableCountsCheck = "PARTITION_TABLE_COUNTS_CHECK" AssessNameTableColumnCountsCheck = "TABLE_COLUMN_COUNTS_CHECK" AssessNameTableIndexCountsCheck = "TABLE_INDEX_COUNTS_CHECK" AssessNameTableRowLengthCheck = "TABLE_ROW_LENGTH_CHECK" AssessNameIndexRowLengthCheck = "INDEX_ROW_LENGTH_CHECK" AssessNameUsernameLengthCheck = "USERNAME_LENGTH_CHECK" AssessNameTableNameLengthCheck = "TABLE_NAME_LENGTH_CHECK" AssessNameColumnNameLengthCheck = "COLUMN_NAME_LENGTH_CHECK" AssessNameIndexNameLengthCheck = "INDEX_NAME_LENGTH_CHECK" AssessNameViewNameLengthCheck = "VIEW_NAME_LENGTH_CHECK" AssessNameSequenceNameLengthCheck = "SEQUENCE_NAME_LENGTH_CHECK" AssessNameSchemaDataSizeRelated = "SCHEMA_DATA_SIZE_RELATED" AssessNameSchemaActiveSessionRelated = "SCHEMA_ACTIVE_SESSION_RELATED" AssessNameSchemaTableRowsTopRelated = "SCHEMA_TABLE_ROWS_TOP_RELATED" AssessNameSchemaCodeObjectRelated = "SCHEMA_CODE_OBJECT_RELATED" AssessNameSchemaSynonymObjectRelated = "SCHEMA_SYNONYM_OBJECT_RELATED" AssessNameSchemaMaterializedViewRelated = "SCHEMA_MATERIALIZED_VIEW_OBJECT_RELATED" AssessNameSchemaTableAvgRowLengthTopRelated = "SCHEMA_TABLE_AVG_ROW_LENGTH_TOP_RELATED" AssessNameSchemaTableNumberTypeEqual0 = "SCHEMA_TABLE_NUMBER_TYPE_EQUAL0" )
Assess Name
const ( BuildInOracleColumnDefaultValueSysdate = "SYSDATE" BuildInOracleColumnDefaultValueSYSGUID = "SYS_GUID()" BuildInOracleColumnDefaultValueNULL = "" )
Database Default Value Map
ORACLE 默认值规则映射规则 O2M
const ( BuildInMySQLColumnDefaultValueCurrentTimestamp = "CURRENT_TIMESTAMP" BuildInMySQLColumnDefaultValueNULL = "NULL" )
MySQL 默认值规则映射规则 M2O
const ( BuildInOracleDatatypeNumber = "NUMBER" BuildInOracleDatatypeBfile = "BFILE" BuildInOracleDatatypeChar = "CHAR" BuildInOracleDatatypeCharacter = "CHARACTER" BuildInOracleDatatypeClob = "CLOB" BuildInOracleDatatypeBlob = "BLOB" BuildInOracleDatatypeDate = "DATE" BuildInOracleDatatypeDecimal = "DECIMAL" BuildInOracleDatatypeDec = "DEC" BuildInOracleDatatypeDoublePrecision = "DOUBLE PRECISION" BuildInOracleDatatypeFloat = "FLOAT" BuildInOracleDatatypeInteger = "INTEGER" BuildInOracleDatatypeInt = "INT" BuildInOracleDatatypeLong = "LONG" BuildInOracleDatatypeLongRAW = "LONG RAW" BuildInOracleDatatypeBinaryFloat = "BINARY_FLOAT" BuildInOracleDatatypeBinaryDouble = "BINARY_DOUBLE" BuildInOracleDatatypeNchar = "NCHAR" BuildInOracleDatatypeNcharVarying = "NCHAR VARYING" BuildInOracleDatatypeNclob = "NCLOB" BuildInOracleDatatypeNumeric = "NUMERIC" BuildInOracleDatatypeNvarchar2 = "NVARCHAR2" BuildInOracleDatatypeRaw = "RAW" BuildInOracleDatatypeReal = "REAL" BuildInOracleDatatypeRowid = "ROWID" BuildInOracleDatatypeSmallint = "SMALLINT" BuildInOracleDatatypeUrowid = "UROWID" BuildInOracleDatatypeVarchar2 = "VARCHAR2" BuildInOracleDatatypeVarchar = "VARCHAR" BuildInOracleDatatypeXmltype = "XMLTYPE" BuildInOracleDatatypeIntervalYearMonth0 = "INTERVAL YEAR(0) TO MONTH" BuildInOracleDatatypeIntervalYearMonth1 = "INTERVAL YEAR(1) TO MONTH" BuildInOracleDatatypeIntervalYearMonth2 = "INTERVAL YEAR(2) TO MONTH" BuildInOracleDatatypeIntervalYearMonth3 = "INTERVAL YEAR(3) TO MONTH" BuildInOracleDatatypeIntervalYearMonth4 = "INTERVAL YEAR(4) TO MONTH" BuildInOracleDatatypeIntervalYearMonth5 = "INTERVAL YEAR(5) TO MONTH" BuildInOracleDatatypeIntervalYearMonth6 = "INTERVAL YEAR(6) TO MONTH" BuildInOracleDatatypeIntervalYearMonth7 = "INTERVAL YEAR(7) TO MONTH" BuildInOracleDatatypeIntervalYearMonth8 = "INTERVAL YEAR(8) TO MONTH" BuildInOracleDatatypeIntervalYearMonth9 = "INTERVAL YEAR(9) TO MONTH" BuildInOracleDatatypeTimestamp = "TIMESTAMP" BuildInOracleDatatypeTimestamp0 = "TIMESTAMP(0)" BuildInOracleDatatypeTimestamp1 = "TIMESTAMP(1)" BuildInOracleDatatypeTimestamp2 = "TIMESTAMP(2)" BuildInOracleDatatypeTimestamp3 = "TIMESTAMP(3)" BuildInOracleDatatypeTimestamp4 = "TIMESTAMP(4)" BuildInOracleDatatypeTimestamp5 = "TIMESTAMP(5)" // datatype timestamp with pricision,view query show default timestamp(6) BuildInOracleDatatypeTimestamp6 = "TIMESTAMP(6)" BuildInOracleDatatypeTimestamp7 = "TIMESTAMP(7)" BuildInOracleDatatypeTimestamp8 = "TIMESTAMP(8)" BuildInOracleDatatypeTimestamp9 = "TIMESTAMP(9)" BuildInOracleDatatypeTimestampWithTimeZone0 = "TIMESTAMP(0) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone1 = "TIMESTAMP(1) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone2 = "TIMESTAMP(2) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone3 = "TIMESTAMP(3) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone4 = "TIMESTAMP(4) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone5 = "TIMESTAMP(5) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone6 = "TIMESTAMP(6) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone7 = "TIMESTAMP(7) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone8 = "TIMESTAMP(8) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithTimeZone9 = "TIMESTAMP(9) WITH TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone0 = "TIMESTAMP(0) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone1 = "TIMESTAMP(1) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone2 = "TIMESTAMP(2) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone3 = "TIMESTAMP(3) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone4 = "TIMESTAMP(4) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone5 = "TIMESTAMP(5) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone6 = "TIMESTAMP(6) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone7 = "TIMESTAMP(7) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone8 = "TIMESTAMP(8) WITH LOCAL TIME ZONE" BuildInOracleDatatypeTimestampWithLocalTimeZone9 = "TIMESTAMP(9) WITH LOCAL TIME ZONE" // INTERVAL DAY(p) TO SECOND(s) 组合过多,代码内匹配处理 // p and s value max 9 -> INTERVAL DAY(9) TO SECOND(9) BuildInOracleDatatypeIntervalDay = "INTERVAL DAY" )
Database Datatype Name
Oracle 数据类型名
const ( BuildInMySQLDatatypeBigint = "BIGINT" BuildInMySQLDatatypeDecimal = "DECIMAL" BuildInMySQLDatatypeDouble = "DOUBLE" BuildInMySQLDatatypeDoublePrecision = "DOUBLE PRECISION" BuildInMySQLDatatypeFloat = "FLOAT" BuildInMySQLDatatypeInt = "INT" BuildInMySQLDatatypeInteger = "INTEGER" BuildInMySQLDatatypeMediumint = "MEDIUMINT" BuildInMySQLDatatypeNumeric = "NUMERIC" BuildInMySQLDatatypeReal = "REAL" BuildInMySQLDatatypeSmallint = "SMALLINT" BuildInMySQLDatatypeTinyint = "TINYINT" BuildInMySQLDatatypeBit = "BIT" BuildInMySQLDatatypeDate = "DATE" BuildInMySQLDatatypeDatetime = "DATETIME" BuildInMySQLDatatypeTimestamp = "TIMESTAMP" BuildInMySQLDatatypeTime = "TIME" BuildInMySQLDatatypeYear = "YEAR" BuildInMySQLDatatypeBlob = "BLOB" BuildInMySQLDatatypeChar = "CHAR" BuildInMySQLDatatypeLongBlob = "LONGBLOB" BuildInMySQLDatatypeLongText = "LONGTEXT" BuildInMySQLDatatypeMediumBlob = "MEDIUMBLOB" BuildInMySQLDatatypeMediumText = "MEDIUMTEXT" BuildInMySQLDatatypeText = "TEXT" BuildInMySQLDatatypeTinyBlob = "TINYBLOB" BuildInMySQLDatatypeTinyText = "TINYTEXT" BuildInMySQLDatatypeVarchar = "VARCHAR" BuildInMySQLDatatypeBinary = "BINARY" BuildInMySQLDatatypeVarbinary = "VARBINARY" // ORACLE ISN'T SUPPORT BuildInMySQLDatatypeSet = "SET" BuildInMySQLDatatypeEnum = "ENUM" )
MySQL 数据类型名
const ( BuildInOracleCharacterSetAL32UTF8 = "AL32UTF8" BuildInOracleCharacterSetZHS16GBK = "ZHS16GBK" )
Database Object Name
Oracle 字符集
const ( BuildInOracleTableTypeHeap = "HEAP" BuildInOracleTableTypeClustered = "CLUSTERED" BuildInOracleTableTypeTemporary = "TEMPORARY" BuildInOracleTableTypePartition = "PARTITIONED" BuildInOracleConstraintTypePrimary = "P" BuildInOracleConstraintTypeCheck = "C" BuildInOracleConstraintTypeUnique = "U" BuildInOracleConstraintTypeForeign = "F" BuildInOracleIndexTypeNormal = "NORMAL" BuildInOracleIndexTypeFunctionBasedNormal = "FUNCTION-BASED NORMAL" BuildInOracleIndexTypeBitmap = "BITMAP" BuildInOracleIndexTypeFunctionBasedBitmap = "FUNCTION-BASED BITMAP" BuildInOracleIndexTypeDomain = "DOMAIN" BuildInOracleViewTypeView = "VIEW" BuildInOracleCodeTypeMaterializedView = "MATERIALIZED VIEW" BuildInOracleCodeTypeCluster = "CLUSTER" BuildInOracleCodeTypeConsumerGroup = "CONSUMER GROUP" BuildInOracleCodeTypeContext = "CONTEXT" BuildInOracleCodeTypeDestination = "DESTINATION" BuildInOracleCodeTypeDirectory = "DIRECTORY" BuildInOracleCodeTypeEdition = "EDITION" BuildInOracleCodeTypeEvaluationContext = "EVALUATION CONTEXT" BuildInOracleCodeTypeFunction = "FUNCTION" BuildInOracleCodeTypeIndexPartition = "INDEX PARTITION" BuildInOracleCodeTypeIndexType = "INDEXTYPE" BuildInOracleCodeTypeJavaClass = "JAVA CLASS" BuildInOracleCodeTypeJavaData = "JAVA DATA" BuildInOracleCodeTypeJavaResource = "JAVA RESOURCE" BuildInOracleCodeTypeJavaSource = "JAVA SOURCE" BuildInOracleCodeTypeJob = "JOB" BuildInOracleCodeTypeJobClass = "JOB CLASS" BuildInOracleCodeTypeLibrary = "LIBRARY" BuildInOracleCodeTypeLob = "LOB" BuildInOracleCodeTypeLobPartition = "LOB PARTITION" BuildInOracleCodeTypeLockdownProfile = "LOCKDOWN PROFILE" BuildInOracleCodeTypeOperator = "OPERATOR" BuildInOracleCodeTypePackage = "PACKAGE" BuildInOracleCodeTypePackageBody = "PACKAGE BODY" BuildInOracleCodeTypeProcedure = "PROCEDURE" BuildInOracleCodeTypeProgram = "PROGRAM" BuildInOracleCodeTypeQueue = "QUEUE" BuildInOracleCodeTypeResourcePlan = "RESOURCE PLAN" BuildInOracleCodeTypeRule = "RULE" BuildInOracleCodeTypeRuleSet = "RULE SET" BuildInOracleCodeTypeSchedule = "SCHEDULE" BuildInOracleCodeTypeSchedulerGroup = "SCHEDULER GROUP" BuildInOracleCodeTypeSequence = "SEQUENCE" BuildInOracleCodeTypeTrigger = "TRIGGER" BuildInOracleCodeTypeType = "TYPE" BuildInOracleCodeTypeTypeBody = "TYPE BODY" BuildInOracleCodeTypeUndefined = "UNDEFINED" BuildInOracleCodeTypeUnifiedAuditPolicy = "UNIFIED AUDIT POLICY" BuildInOracleCodeTypeWindow = "WINDOW" BuildInOracleCodeTypeXMLSchema = "XML SCHEMA" BuildInOracleCodeTypeDatabaseLink = "DATABASE LINK" BuildInOracleCodeTypeSynonym = "SYNONYM" BuildInOraclePartitionTypeRange = "RANGE" BuildInOraclePartitionTypeHash = "HASH" BuildInOraclePartitionTypeList = "LIST" BuildInOraclePartitionTypeInterval = "INTERVAL" BuildInOraclePartitionTypeReference = "REFERENCE" BuildInOraclePartitionTypeComposite = "COMPOSITE" BuildInOraclePartitionTypeSystem = "SYSTEM" BuildInOraclePartitionTypeRangeHash = "RANGE-HASH" BuildInOraclePartitionTypeRangeList = "RANGE-LIST" BuildInOraclePartitionTypeRangeRange = "RANGE-RANGE" BuildInOraclePartitionTypeListList = "LIST-LIST" BuildInOraclePartitionTypeListHash = "LIST-HASH" BuildInOraclePartitionTypeListRange = "LIST-RANGE" BuildInOracleTemporaryTypeSession = "SYS$SESSION" BuildInOracleTemporaryTypeTransaction = "SYS$TRANSACTION" )
Oracle 数据类型名字
const ( MigrateOperationUpdate = "UPDATE" MigrateOperationInsert = "INSERT" MigrateOperationDelete = "DELETE" MigrateOperationTruncate = "TRUNCATE" MigrateOperationDrop = "DROP" MigrateOperationDDL = "DDL" MigrateOperationTruncateTable = "TRUNCATE TABLE" MigrateOperationDropTable = "DROP TABLE" )
Oracle Redo 同步操作类型
const ( // MySQL 支持 check 约束版本 > 8.0.15 MySQLCheckConsVersion = "8.0.15" // MySQL 表达式索引版本 > 8.0.0 MySQLExpressionIndexVersion = "8.0.0" // MySQL 版本分隔符号 MySQLVersionDelimiter = "-" // 允许 Oracle 表、字段 Collation // 需要 oracle 12.2g 及以上 OracleTableColumnCollationDBVersion = "12.2" // Oracle 用户、表、字段默认使用 DB 排序规则 OracleUserTableColumnDefaultCollation = "USING_NLS_COMP" // Struct JSON 格式化 -> Check 阶段 JSONColumns = "COLUMN" JSONIndex = "INDEX" JSONPUConstraint = "PUK" JSONFKConstraint = "FK" JSONCKConstraint = "CK" JSONPartition = "PARTITION" )
const ( // TiDB 数据库 TiDBClusteredIndexIntOnlyValue = "INT_ONLY" TiDBClusteredIndexONValue = "ON" TiDBClusteredIndexOFFValue = "OFF" )
const ( CharsetUTF8MB4 = "UTF8MB4" CharsetGB18030 = "GB18030" CharsetBIG5 = "BIG5" CharsetGBK = "GBK" )
数据迁移、数据校验、表结构默认值、注释 字符类型数据映射规则 1、用于程序连接源端数据库读取数据字符类型数据,以对应字符集写入下游数据库 2、用于数据表默认值或者注释
const ( MigrateTableStructFieldNameOriginCase = "0" MigrateTableStructFieldNameLowerCase = "1" MigrateTableStructFieldNameUpperCase = "2" )
表结构迁移大小写
const ( OracleNULLSTRINGTableAttrWithoutNULL = "NULLSTRING" OracleNULLSTRINGTableAttrWithCustom = "" OracleNULLSTRINGTableAttrWithNULL = "NULL" )
Table Attr Null 以及空字符串特殊处理
const ( MySQLMaxIdleConn = 512 MySQLMaxConn = 1024 MySQLConnMaxLifeTime = 300 * time.Second MySQLConnMaxIdleTime = 200 * time.Second )
MySQL 连接配置
const ( TaskModePrepare = "PREPARE" TaskModeAssess = "ASSESS" TaskModeReverse = "REVERSE" TaskModeCheck = "CHECK" TaskModeCompare = "COMPARE" TaskModeCSV = "CSV" TaskModeFull = "FULL" TaskModeAll = "ALL" )
任务模式
const ( TaskStatusWaiting = "WAITING" TaskStatusRunning = "RUNNING" TaskStatusSuccess = "SUCCESS" TaskStatusFailed = "FAILED" )
任务状态
const ( // 值 0 代表源端表未进行初始化 -> 适用于 full/csv/all 模式 TaskTableDefaultSourceGlobalSCN = 0 // 值 -1 代表源端表未进行 chunk 切分 -> 适用于 full/csv/all 模式 TaskTableDefaultSplitChunkNums = -1 )
任务初始值
const ( DatabaseTypeOracle = "ORACLE" DatabaseTypeTiDB = "TIDB" DatabaseTypeMySQL = "MYSQL" )
任务 DB 类型
const ( TaskTypeOracle2MySQL = "ORACLE2MYSQL" TaskTypeOracle2TiDB = "ORACLE2TIDB" TaskTypeMySQL2Oracle = "MYSQL2ORACLE" TaskTypeTiDB2Oracle = "TIDB2ORACLE" )
任务类型
const ChannelBufferSize = 1024
任务并发通道 Channle Size
const RequireOracleDBVersion = "11"
数据全量/实时同步 Oracle 版本要求 要求 oracle 11g 及以上
Variables ¶
var ( MYSQLCharsetUTF8MB4 = "UTF8MB4" MYSQLCharsetUTF8 = "UTF8" MYSQLCharsetBIG5 = "BIG5" MYSQLCharsetGBK = "GBK" MYSQLCharsetGB18030 = "GB18030" ORACLECharsetAL32UTF8 = "AL32UTF8" ORACLECharsetZHT16BIG5 = "ZHT16BIG5" ORACLECharsetZHS16GBK = "ZHS16GBK" ORACLECharsetZHS32GB18030 = "ZHS32GB18030" )
transferdb 字符类型数据字符集转换支持列表
var BuildInMySQLM2OColumnDefaultValueMap = map[string]string{ BuildInMySQLColumnDefaultValueCurrentTimestamp: "SYSDATE", BuildInMySQLColumnDefaultValueNULL: "NULL", }
var BuildInMySQLM2ODatatypeNameMap = map[string]string{ BuildInMySQLDatatypeSmallint: "NUMBER", BuildInMySQLDatatypeTinyint: "NUMBER", BuildInMySQLDatatypeBigint: "NUMBER", BuildInMySQLDatatypeDecimal: "DECIMAL", BuildInMySQLDatatypeDouble: "BINARY_DOUBLE", BuildInMySQLDatatypeDoublePrecision: "BINARY_DOUBLE", BuildInMySQLDatatypeFloat: "BINARY_FLOAT", BuildInMySQLDatatypeInt: "NUMBER", BuildInMySQLDatatypeInteger: "NUMBER", BuildInMySQLDatatypeMediumint: "NUMBER", BuildInMySQLDatatypeNumeric: "NUMBER", BuildInMySQLDatatypeReal: "BINARY_FLOAT", BuildInMySQLDatatypeBit: "RAW", BuildInMySQLDatatypeDate: "DATE", BuildInMySQLDatatypeDatetime: "DATE", BuildInMySQLDatatypeTimestamp: "TIMESTAMP", BuildInMySQLDatatypeTime: "DATE", BuildInMySQLDatatypeYear: "NUMBER", BuildInMySQLDatatypeBlob: "BLOB", BuildInMySQLDatatypeChar: "CHAR", BuildInMySQLDatatypeLongBlob: "BLOB", BuildInMySQLDatatypeLongText: "CLOB", BuildInMySQLDatatypeMediumBlob: "BLOB", BuildInMySQLDatatypeMediumText: "CLOB", BuildInMySQLDatatypeText: "CLOB", BuildInMySQLDatatypeTinyBlob: "BLOB", BuildInMySQLDatatypeTinyText: "VARCHAR2", BuildInMySQLDatatypeVarchar: "VARCHAR2", BuildInMySQLDatatypeBinary: "RAW", BuildInMySQLDatatypeVarbinary: "RAW", }
MySQL 数据类型名映射规则 M2O
var BuildInOracleO2MColumnDefaultValueMap = map[string]string{ BuildInOracleColumnDefaultValueSysdate: "NOW()", BuildInOracleColumnDefaultValueSYSGUID: "UUID()", BuildInOracleColumnDefaultValueNULL: "NULL", }
var BuildInOracleO2MDatatypeNameMap = map[string]string{ BuildInOracleDatatypeNumber: "TINYINT/SMALLINT/INT/BIGINT/DECIMAL", BuildInOracleDatatypeBfile: "VARCHAR", BuildInOracleDatatypeChar: "VARCHAR", BuildInOracleDatatypeCharacter: "VARCHAR", BuildInOracleDatatypeClob: "LONGTEXT", BuildInOracleDatatypeBlob: "BLOB", BuildInOracleDatatypeDate: "DATETIME", BuildInOracleDatatypeDecimal: "DECIMAL", BuildInOracleDatatypeDec: "DECIMAL", BuildInOracleDatatypeDoublePrecision: "DOUBLE PRECISION", BuildInOracleDatatypeFloat: "DOUBLE", BuildInOracleDatatypeInteger: "INT", BuildInOracleDatatypeInt: "INT", BuildInOracleDatatypeLong: "LONGTEXT", BuildInOracleDatatypeLongRAW: "LONGBLOB", BuildInOracleDatatypeBinaryFloat: "DOUBLE", BuildInOracleDatatypeBinaryDouble: "DOUBLE", BuildInOracleDatatypeNchar: "VARCHAR", BuildInOracleDatatypeNcharVarying: "NCHAR VARYING", BuildInOracleDatatypeNclob: "TEXT", BuildInOracleDatatypeNumeric: "NUMERIC", BuildInOracleDatatypeNvarchar2: "VARCHAR", BuildInOracleDatatypeRaw: "VARBINARY", BuildInOracleDatatypeReal: "DOUBLE", BuildInOracleDatatypeRowid: "VARCHAR", BuildInOracleDatatypeSmallint: "SMALLINT", BuildInOracleDatatypeUrowid: "VARCHAR", BuildInOracleDatatypeVarchar2: "VARCHAR", BuildInOracleDatatypeVarchar: "VARCHAR", BuildInOracleDatatypeXmltype: "LONGTEXT", BuildInOracleDatatypeIntervalYearMonth0: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth1: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth2: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth3: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth4: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth5: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth6: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth7: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth8: "VARCHAR", BuildInOracleDatatypeIntervalYearMonth9: "VARCHAR", BuildInOracleDatatypeTimestamp: "TIMESTAMP", BuildInOracleDatatypeTimestamp0: "TIMESTAMP", BuildInOracleDatatypeTimestamp1: "TIMESTAMP", BuildInOracleDatatypeTimestamp2: "TIMESTAMP", BuildInOracleDatatypeTimestamp3: "TIMESTAMP", BuildInOracleDatatypeTimestamp4: "TIMESTAMP", BuildInOracleDatatypeTimestamp5: "TIMESTAMP", BuildInOracleDatatypeTimestamp6: "TIMESTAMP", BuildInOracleDatatypeTimestamp7: "TIMESTAMP", BuildInOracleDatatypeTimestamp8: "TIMESTAMP", BuildInOracleDatatypeTimestamp9: "TIMESTAMP", BuildInOracleDatatypeTimestampWithTimeZone0: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone1: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone2: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone3: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone4: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone5: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone6: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone7: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone8: "DATETIME", BuildInOracleDatatypeTimestampWithTimeZone9: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone0: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone1: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone2: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone3: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone4: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone5: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone6: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone7: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone8: "DATETIME", BuildInOracleDatatypeTimestampWithLocalTimeZone9: "DATETIME", BuildInOracleDatatypeIntervalDay: "VARCHAR", }
Oracle 数据类型名映射规则 O2M
var MigrateCurrentResetFlag = 0
用于控制当程序消费追平到当前 CURRENT 重做日志, 当值 == 0 启用 filterOracleIncrRecord 大于或者等于逻辑 当值 == 1 启用 filterOracleIncrRecord 大于逻辑,避免已被消费得日志一直被重复消费
var MigrateDataSupportCharset = []string{CharsetUTF8MB4, CharsetGBK, CharsetBIG5, CharsetGB18030}
var MigrateMYSQLCompatibleCharsetStringConvertMapping = map[string]string{ MYSQLCharsetUTF8MB4: CharsetUTF8MB4, MYSQLCharsetUTF8: CharsetUTF8MB4, MYSQLCharsetBIG5: CharsetBIG5, MYSQLCharsetGBK: CharsetGBK, MYSQLCharsetGB18030: CharsetGB18030, }
var MigrateOracleCharsetStringConvertMapping = map[string]string{ ORACLECharsetAL32UTF8: CharsetUTF8MB4, ORACLECharsetZHT16BIG5: CharsetBIG5, ORACLECharsetZHS16GBK: CharsetGBK, ORACLECharsetZHS32GB18030: CharsetGB18030, }
var MigrateTableStructureDatabaseCharsetMap = map[string]map[string]string{ TaskTypeOracle2MySQL: { ORACLECharsetAL32UTF8: MYSQLCharsetUTF8MB4, ORACLECharsetZHT16BIG5: MYSQLCharsetBIG5, ORACLECharsetZHS16GBK: MYSQLCharsetGBK, ORACLECharsetZHS32GB18030: MYSQLCharsetGB18030, }, TaskTypeOracle2TiDB: { ORACLECharsetAL32UTF8: MYSQLCharsetUTF8MB4, ORACLECharsetZHT16BIG5: MYSQLCharsetUTF8MB4, ORACLECharsetZHS16GBK: MYSQLCharsetUTF8MB4, ORACLECharsetZHS32GB18030: MYSQLCharsetUTF8MB4, }, TaskTypeMySQL2Oracle: { MYSQLCharsetUTF8MB4: ORACLECharsetAL32UTF8, MYSQLCharsetUTF8: ORACLECharsetAL32UTF8, MYSQLCharsetBIG5: ORACLECharsetZHT16BIG5, MYSQLCharsetGBK: ORACLECharsetZHS16GBK, MYSQLCharsetGB18030: ORACLECharsetZHS32GB18030, }, TaskTypeTiDB2Oracle: { MYSQLCharsetUTF8MB4: ORACLECharsetAL32UTF8, MYSQLCharsetUTF8: ORACLECharsetAL32UTF8, MYSQLCharsetGBK: ORACLECharsetZHS16GBK, }, }
表结构迁移以及表结构校验字符集、排序规则 用于表结构以及字段属性字符集映射规则
var MigrateTableStructureDatabaseCollationMap = map[string]map[string]map[string]string{ TaskTypeOracle2MySQL: { "BINARY_CI": { MYSQLCharsetUTF8MB4: "UTF8MB4_0900_AS_CI", MYSQLCharsetUTF8: "UTF8_0900_AS_CI", MYSQLCharsetBIG5: "BIG5_CHINESE_CI", MYSQLCharsetGBK: "GBK_CHINESE_CI", MYSQLCharsetGB18030: "GB18030_CHINESE_CI", }, "BINARY_AI": { MYSQLCharsetUTF8MB4: "UTF8MB4_GENERAL_CI", MYSQLCharsetUTF8: "UTF8_GENERAL_CI", MYSQLCharsetBIG5: "BIG5_CHINESE_CI", MYSQLCharsetGBK: "GBK_CHINESE_CI", MYSQLCharsetGB18030: "GB18030_CHINESE_CI", }, "BINARY_CS": { MYSQLCharsetUTF8MB4: "UTF8MB4_BIN", MYSQLCharsetUTF8: "UTF8_BIN", MYSQLCharsetBIG5: "BIG5_BIN", MYSQLCharsetGBK: "GBK_BIN", MYSQLCharsetGB18030: "GB18030_BIN", }, "BINARY": { MYSQLCharsetUTF8MB4: "UTF8MB4_BIN", MYSQLCharsetUTF8: "UTF8_BIN", MYSQLCharsetBIG5: "BIG5_BIN", MYSQLCharsetGBK: "GBK_BIN", MYSQLCharsetGB18030: "GB18030_BIN", }, }, TaskTypeOracle2TiDB: { "BINARY_CI": { MYSQLCharsetUTF8MB4: "UTF8MB4_0900_AS_CI", MYSQLCharsetUTF8: "UTF8_0900_AS_CI", MYSQLCharsetBIG5: "BIG5_CHINESE_CI", MYSQLCharsetGBK: "GBK_CHINESE_CI", MYSQLCharsetGB18030: "GB18030_CHINESE_CI", }, "BINARY_AI": { MYSQLCharsetUTF8MB4: "UTF8MB4_GENERAL_CI", MYSQLCharsetUTF8: "UTF8_GENERAL_CI", MYSQLCharsetBIG5: "BIG5_CHINESE_CI", MYSQLCharsetGBK: "GBK_CHINESE_CI", MYSQLCharsetGB18030: "GB18030_CHINESE_CI", }, "BINARY_CS": { MYSQLCharsetUTF8MB4: "UTF8MB4_BIN", MYSQLCharsetUTF8: "UTF8_BIN", MYSQLCharsetBIG5: "BIG5_BIN", MYSQLCharsetGBK: "GBK_BIN", MYSQLCharsetGB18030: "GB18030_BIN", }, "BINARY": { MYSQLCharsetUTF8MB4: "UTF8MB4_BIN", MYSQLCharsetUTF8: "UTF8_BIN", MYSQLCharsetBIG5: "BIG5_BIN", MYSQLCharsetGBK: "GBK_BIN", MYSQLCharsetGB18030: "GB18030_BIN", }, }, TaskTypeMySQL2Oracle: { "UTF8MB4_0900_AS_CI": { ORACLECharsetAL32UTF8: "BINARY_CI", ORACLECharsetZHT16BIG5: "BINARY_CI", ORACLECharsetZHS16GBK: "BINARY_CI", ORACLECharsetZHS32GB18030: "BINARY_CI", }, "UTF8MB4_GENERAL_CI": { ORACLECharsetAL32UTF8: "BINARY_AI", ORACLECharsetZHT16BIG5: "BINARY_AI", ORACLECharsetZHS16GBK: "BINARY_AI", ORACLECharsetZHS32GB18030: "BINARY_AI", }, "UTF8_GENERAL_CI": { ORACLECharsetAL32UTF8: "BINARY_AI", ORACLECharsetZHT16BIG5: "BINARY_AI", ORACLECharsetZHS16GBK: "BINARY_AI", ORACLECharsetZHS32GB18030: "BINARY_AI", }, "BIG5_CHINESE_CI": { ORACLECharsetAL32UTF8: "BINARY_AI/BINARY_CI", ORACLECharsetZHT16BIG5: "BINARY_AI/BINARY_CI", ORACLECharsetZHS16GBK: "BINARY_AI/BINARY_CI", ORACLECharsetZHS32GB18030: "BINARY_AI/BINARY_CI", }, "GBK_CHINESE_CI": { ORACLECharsetAL32UTF8: "BINARY_AI/BINARY_CI", ORACLECharsetZHT16BIG5: "BINARY_AI/BINARY_CI", ORACLECharsetZHS16GBK: "BINARY_AI/BINARY_CI", ORACLECharsetZHS32GB18030: "BINARY_AI/BINARY_CI", }, "GB18030_CHINESE_CI": { ORACLECharsetAL32UTF8: "BINARY_AI/BINARY_CI", ORACLECharsetZHT16BIG5: "BINARY_AI/BINARY_CI", ORACLECharsetZHS16GBK: "BINARY_AI/BINARY_CI", ORACLECharsetZHS32GB18030: "BINARY_AI/BINARY_CI", }, "UTF8MB4_BIN": { ORACLECharsetAL32UTF8: "BINARY/BINARY_CS", ORACLECharsetZHT16BIG5: "BINARY/BINARY_CS", ORACLECharsetZHS16GBK: "BINARY/BINARY_CS", ORACLECharsetZHS32GB18030: "BINARY/BINARY_CS", }, "UTF8_BIN": { ORACLECharsetAL32UTF8: "BINARY/BINARY_CS", ORACLECharsetZHT16BIG5: "BINARY/BINARY_CS", ORACLECharsetZHS16GBK: "BINARY/BINARY_CS", ORACLECharsetZHS32GB18030: "BINARY/BINARY_CS", }, "BIG5_BIN": { ORACLECharsetAL32UTF8: "BINARY/BINARY_CS", ORACLECharsetZHT16BIG5: "BINARY/BINARY_CS", ORACLECharsetZHS16GBK: "BINARY/BINARY_CS", ORACLECharsetZHS32GB18030: "BINARY/BINARY_CS", }, "GBK_BIN": { ORACLECharsetAL32UTF8: "BINARY/BINARY_CS", ORACLECharsetZHT16BIG5: "BINARY/BINARY_CS", ORACLECharsetZHS16GBK: "BINARY/BINARY_CS", ORACLECharsetZHS32GB18030: "BINARY/BINARY_CS", }, "GB18030_BIN": { ORACLECharsetAL32UTF8: "BINARY/BINARY_CS", ORACLECharsetZHT16BIG5: "BINARY/BINARY_CS", ORACLECharsetZHS16GBK: "BINARY/BINARY_CS", ORACLECharsetZHS32GB18030: "BINARY/BINARY_CS", }, }, TaskTypeTiDB2Oracle: { "UTF8MB4_0900_AS_CI": { ORACLECharsetAL32UTF8: "BINARY_CI", ORACLECharsetZHT16BIG5: "BINARY_CI", ORACLECharsetZHS16GBK: "BINARY_CI", ORACLECharsetZHS32GB18030: "BINARY_CI", }, "UTF8MB4_GENERAL_CI": { ORACLECharsetAL32UTF8: "BINARY_AI", ORACLECharsetZHT16BIG5: "BINARY_AI", ORACLECharsetZHS16GBK: "BINARY_AI", ORACLECharsetZHS32GB18030: "BINARY_AI", }, "UTF8_GENERAL_CI": { ORACLECharsetAL32UTF8: "BINARY_AI", ORACLECharsetZHT16BIG5: "BINARY_AI", ORACLECharsetZHS16GBK: "BINARY_AI", ORACLECharsetZHS32GB18030: "BINARY_AI", }, "GBK_CHINESE_CI": { ORACLECharsetAL32UTF8: "BINARY_AI/BINARY_CI", ORACLECharsetZHT16BIG5: "BINARY_AI/BINARY_CI", ORACLECharsetZHS16GBK: "BINARY_AI/BINARY_CI", ORACLECharsetZHS32GB18030: "BINARY_AI/BINARY_CI", }, "UTF8MB4_BIN": { ORACLECharsetAL32UTF8: "BINARY/BINARY_CS", ORACLECharsetZHT16BIG5: "BINARY/BINARY_CS", ORACLECharsetZHS16GBK: "BINARY/BINARY_CS", ORACLECharsetZHS32GB18030: "BINARY/BINARY_CS", }, "UTF8_BIN": { ORACLECharsetAL32UTF8: "BINARY/BINARY_CS", ORACLECharsetZHT16BIG5: "BINARY/BINARY_CS", ORACLECharsetZHS16GBK: "BINARY/BINARY_CS", ORACLECharsetZHS32GB18030: "BINARY/BINARY_CS", }, "GBK_BIN": { ORACLECharsetAL32UTF8: "BINARY/BINARY_CS", ORACLECharsetZHT16BIG5: "BINARY/BINARY_CS", ORACLECharsetZHS16GBK: "BINARY/BINARY_CS", ORACLECharsetZHS32GB18030: "BINARY/BINARY_CS", }, }, }
MySQL 8.0 utf8mb4_0900_as_cs 区分重音、区分大小写的排序规则 utf8mb4_0900_ai_ci 不区分重音和不区分大小写的排序规则 utf8mb4_0900_as_ci 区分重音、不区分大小写的排序规则 Oracle 字段 Collation 映射
var OracleIsNotSupportDataType = []string{"ENUM", "SET"}
Oracle 不支持数据类型 -> M2O
var SpecialMySQLColumnCollationWithDataTYPE = []string{"TINYTEXT",
"TEXT",
"MEDIUMTEXT",
"LONGTEXT"}
MySQL Data Type reverse Oracle CLOB or NCLOB configure collation error, need configure columnCollation = "" ORA-43912: invalid collation specified for a CLOB or NCLOB value columnCollation = ""
var SpecialMySQLDataDefaultsWithDataTYPE = []string{"TIME",
"DATE",
"DATETIME",
"TIMESTAMP",
"CHAR",
"VARCHAR",
"TINYTEXT",
"TEXT", "MEDIUMTEX", "LONGTEXT", "BIT", "BINARY", "VARBINARY", "TINYBLOB", "BLOB", "MEDIUMBLOB", "LONGBLOB"}
MySQL Reverse M2O mysql 默认值未区分,字符数据、数值数据,用于匹配 mysql 字符串默认值,判断是否需单引号 默认值 uuid() 匹配到 xxx() 括号结尾,不需要单引号 默认值 CURRENT_TIMESTAMP 不需要括号,内置转换成 ORACLE SYSDATE 默认值 skp 或者 1 需要单引号
var TiDBIntegerPrimaryKeyList = []string{"TINYINT", "SMALLINT", "INT", "BIGINT", "DECIMAL"}
alter-primary-key = fase 主键整型数据类型列表
Functions ¶
func CharsetConvert ¶ added in v1.0.14
func DiffStructArray ¶
func DiffStructArray(structA, structB interface{}) ([]interface{}, []interface{}, bool)
用于对比 struct 是否相等
func FilterDifferenceStringItems ¶
过滤排除元素,返回新数组
func FilterIntersectionStringItems ¶
过滤两个数组相同元素(交集),返回新数组
func GetStringSliceElementIndex ¶
获取数组元素下标
func IsSubsetString ¶
两个数组对比,判断 checkItems 是否是 originItems 子集
func ReplaceSpecifiedString ¶
替换指定字符
func SpecialLettersUsingMySQL ¶
如果存在特殊字符,直接在特殊字符前添加\ * 判断是否为字母: unicode.IsLetter(v) 判断是否为十进制数字: unicode.IsDigit(v) 判断是否为数字: unicode.IsNumber(v) 判断是否为空白符号: unicode.IsSpace(v) 判断师傅是特殊符号:unicode.IsSymbol(v) 判断是否为Unicode标点字符 :unicode.IsPunct(v) 判断是否为中文:unicode.Han(v)
func SplitMultipleSlice ¶
func SplitMultipleSlice(arr []interface{}, num int64) [][]interface{}
多重数组拆分
func SplitMultipleStringSlice ¶
数组拆分
func StrconvRune ¶
func StringJOIN ¶
字符串 JOIN
Types ¶
This section is empty.