說明：由於目前的datax二進制包里不含oceanbasev10reader和oceanbasev10writer插件，需要源碼編譯后方可使用。
1、源碼安裝
參考：https://github.com/alibaba/DataX/blob/master/userGuid.md
1.1. 若沒有git環境，先安裝一下。
[root@test datax_code2]# yum install git -y [root@test datax_code2]#
1.2 git clone git@github.com:alibaba/DataX.git
[root@test datax_code2]# git clone git@github.com:alibaba/DataX.git 正克隆到 'DataX'... The authenticity of host 'github.com (52.74.223.119)' can't be established. RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8. RSA key fingerprint is MD5:16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'github.com,52.74.223.119' (RSA) to the list of known hosts. Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. [root@test datax_code2]# 登錄個人的github賬號，點擊頭像，選擇Settings,選擇SSH and GPS keys，添加一個ssh key, 將執行git命令的機器上對應用戶的公鑰(/root/.ssh/id_rsa.pub)內容拷貝到github網頁。 再次執行成功 [root@test datax_code2]# git clone git@github.com:alibaba/DataX.git 正克隆到 'DataX'... Warning: Permanently added the RSA host key for IP address '20.205.243.166' to the list of known hosts. remote: Enumerating objects: 2977, done. remote: Counting objects: 100% (190/190), done. remote: Compressing objects: 100% (115/115), done. remote: Total 2977 (delta 41), reused 135 (delta 23), pack-reused 2787 接收對象中: 100% (2977/2977), 18.93 MiB | 5.23 MiB/s, done. 處理 delta 中: 100% (770/770), done. [root@test datax_code2]#
1.3 通過maven打包
wget https://dlcdn.apache.org/maven/maven-3/3.8.2/binaries/apache-maven-3.8.2-bin.tar.gz 如果版本不可用，可以登錄http://maven.apache.org/download.cgi 選擇可用的版本。 [root@test opt]# tar zxf apache-maven-3.8.2-bin.tar.gz [root@test opt]# cd apache-maven-3.8.2 [root@test apache-maven-3.8.2]# ll 總用量 36 drwxr-xr-x. 2 root root 97 8月 18 17:58 bin drwxr-xr-x. 2 root root 76 8月 18 17:58 boot drwxr-xr-x. 3 root root 63 8月 5 02:57 conf drwxr-xr-x. 3 root root 4096 8月 18 17:58 lib -rw-r--r--. 1 root root 17511 8月 5 02:57 LICENSE -rw-r--r--. 1 root root 5141 8月 5 02:57 NOTICE -rw-r--r--. 1 root root 2612 8月 5 02:57 README.txt [root@test bin]# ln -s /opt/apache-maven-3.8.2/bin/mvn /usr/bin/mvn [root@test DataX]# mvn -U clean package assembly:assembly -Dmaven.test.skip=true which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin) The JAVA_HOME environment variable is not defined correctly, this environment variable is needed to run this program. [root@test DataX]# 下載JDK https://www.oracle.com/cn/java/technologies/javase/javase-jdk8-downloads.html [root@test ~]# java -version java version "1.8.0_77" Java(TM) SE Runtime Environment (build 1.8.0_77-b03) Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode) [root@test ~]# [root@test DataX]# pwd /opt/datax_code2/DataX [root@test DataX]# mvn -U clean package assembly:assembly -Dmaven.test.skip=true
說明：如果想一次性打包成功，經過測試可以直接把不需要的插件刪除掉
[root@test DataX]# pwd /opt/datax_code2/DataX/pom.xml 刪除<module>hdfsreader</module> 刪除<module>hdfswriter</module> 刪除<module>oscarwriter</module>
#第一次執行報錯 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:06 min [INFO] Finished at: 2021-08-18T18:14:03+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project hdfsreader: Could not resolve dependencies for project com.alibaba.datax:hdfsreader:jar:0.0.1-SNAPSHOT: Failed to collect dependencies at org.apache.hive:hive-exec:jar:1.1.1 -> org.apache.calcite:calcite-core:jar:1.0.0-incubating -> org.apache.calcite:calcite-avatica:jar:1.0.0-incubating -> eigenbase:eigenbase-properties:jar:1.1.4: Failed to read artifact descriptor for eigenbase:eigenbase-properties:jar:1.1.4: Could not transfer artifact eigenbase:eigenbase-properties:pom:1.1.4 from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: [datanucleus (http://www.datanucleus.org/downloads/maven2, default, releases), glassfish-repository (http://maven.glassfish.org/content/groups/glassfish, default, disabled), glassfish-repo-archive (http://maven.glassfish.org/content/groups/glassfish, default, disabled), apache.snapshots (http://repository.apache.org/snapshots, default, snapshots), conjars (http://conjars.org/repo, default, releases+snapshots)] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <args> -rf :hdfsreader [root@test DataX]# #第二次執行報錯 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 02:24 min [INFO] Finished at: 2021-08-18T18:17:31+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project hdfswriter: Could not resolve dependencies for project com.alibaba.datax:hdfswriter:jar:0.0.1-SNAPSHOT: Failed to collect dependencies at org.apache.hive:hive-exec:jar:1.1.1 -> org.apache.calcite:calcite-core:jar:1.0.0-incubating -> org.apache.calcite:calcite-avatica:jar:1.0.0-incubating -> eigenbase:eigenbase-properties:jar:1.1.4: Failed to read artifact descriptor for eigenbase:eigenbase-properties:jar:1.1.4: Could not transfer artifact eigenbase:eigenbase-properties:pom:1.1.4 from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: [datanucleus (http://www.datanucleus.org/downloads/maven2, default, releases), glassfish-repository (http://maven.glassfish.org/content/groups/glassfish, default, disabled), glassfish-repo-archive (http://maven.glassfish.org/content/groups/glassfish, default, disabled), apache.snapshots (http://repository.apache.org/snapshots, default, snapshots), conjars (http://conjars.org/repo, default, releases+snapshots)] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <args> -rf :hdfswriter [root@test DataX]# #第三次執行報錯 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 02:46 min [INFO] Finished at: 2021-08-18T18:21:23+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project oscarwriter: Could not resolve dependencies for project com.alibaba.datax:oscarwriter:jar:0.0.1-SNAPSHOT: Could not find artifact com.oscar:oscar:jar:7.0.8 at specified path /opt/datax_code2/DataX/oscarwriter/src/main/lib/oscarJDBC.jar -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <args> -rf :oscarwriter [root@test DataX]#
第四次執行成功
[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] datax-all 0.0.1-SNAPSHOT ........................... SUCCESS [01:57 min] [INFO] datax-common 0.0.1-SNAPSHOT ........................ SUCCESS [ 1.981 s] [INFO] datax-transformer 0.0.1-SNAPSHOT ................... SUCCESS [ 1.095 s] [INFO] datax-core 0.0.1-SNAPSHOT .......................... SUCCESS [ 2.626 s] [INFO] plugin-rdbms-util 0.0.1-SNAPSHOT ................... SUCCESS [ 1.055 s] [INFO] mysqlreader 0.0.1-SNAPSHOT ......................... SUCCESS [ 0.872 s] [INFO] drdsreader 0.0.1-SNAPSHOT .......................... SUCCESS [ 0.996 s] [INFO] sqlserverreader 0.0.1-SNAPSHOT ..................... SUCCESS [ 1.237 s] [INFO] postgresqlreader 0.0.1-SNAPSHOT .................... SUCCESS [ 0.876 s] [INFO] kingbaseesreader 0.0.1-SNAPSHOT .................... SUCCESS [ 0.878 s] [INFO] oraclereader 0.0.1-SNAPSHOT ........................ SUCCESS [ 0.867 s] [INFO] odpsreader 0.0.1-SNAPSHOT .......................... SUCCESS [ 1.542 s] [INFO] otsreader 0.0.1-SNAPSHOT ........................... SUCCESS [ 1.992 s] [INFO] otsstreamreader 0.0.1 .............................. SUCCESS [ 2.048 s] [INFO] plugin-unstructured-storage-util 0.0.1-SNAPSHOT .... SUCCESS [ 0.904 s] [INFO] txtfilereader 0.0.1-SNAPSHOT ....................... SUCCESS [ 3.129 s] [INFO] streamreader 0.0.1-SNAPSHOT ........................ SUCCESS [ 0.872 s] [INFO] ossreader 0.0.1-SNAPSHOT ........................... SUCCESS [ 3.184 s] [INFO] ftpreader 0.0.1-SNAPSHOT ........................... SUCCESS [ 2.921 s] [INFO] mongodbreader 0.0.1-SNAPSHOT ....................... SUCCESS [ 2.812 s] [INFO] rdbmsreader 0.0.1-SNAPSHOT ......................... SUCCESS [ 0.958 s] [INFO] hbase11xreader 0.0.1-SNAPSHOT ...................... SUCCESS [ 4.730 s] [INFO] hbase094xreader 0.0.1-SNAPSHOT ..................... SUCCESS [ 2.840 s] [INFO] tsdbreader 0.0.1-SNAPSHOT .......................... SUCCESS [ 1.235 s] [INFO] opentsdbreader 0.0.1-SNAPSHOT ...................... SUCCESS [ 1.967 s] [INFO] cassandrareader 0.0.1-SNAPSHOT ..................... SUCCESS [ 1.607 s] [INFO] gdbreader 0.0.1-SNAPSHOT ........................... SUCCESS [ 3.300 s] [INFO] oceanbasev10reader 0.0.1-SNAPSHOT .................. SUCCESS [ 1.205 s] [INFO] mysqlwriter 0.0.1-SNAPSHOT ......................... SUCCESS [ 0.788 s] [INFO] drdswriter 0.0.1-SNAPSHOT .......................... SUCCESS [ 0.831 s] [INFO] odpswriter 0.0.1-SNAPSHOT .......................... SUCCESS [ 1.582 s] [INFO] txtfilewriter 0.0.1-SNAPSHOT ....................... SUCCESS [ 2.693 s] [INFO] ftpwriter 0.0.1-SNAPSHOT ........................... SUCCESS [ 2.816 s] [INFO] streamwriter 0.0.1-SNAPSHOT ........................ SUCCESS [ 0.829 s] [INFO] otswriter 0.0.1-SNAPSHOT ........................... SUCCESS [ 1.704 s] [INFO] oraclewriter 0.0.1-SNAPSHOT ........................ SUCCESS [ 0.820 s] [INFO] sqlserverwriter 0.0.1-SNAPSHOT ..................... SUCCESS [ 0.814 s] [INFO] postgresqlwriter 0.0.1-SNAPSHOT .................... SUCCESS [ 0.838 s] [INFO] kingbaseeswriter 0.0.1-SNAPSHOT .................... SUCCESS [ 0.873 s] [INFO] osswriter 0.0.1-SNAPSHOT ........................... SUCCESS [ 2.925 s] [INFO] mongodbwriter 0.0.1-SNAPSHOT ....................... SUCCESS [ 2.936 s] [INFO] adswriter 0.0.1-SNAPSHOT ........................... SUCCESS [ 2.580 s] [INFO] ocswriter 0.0.1-SNAPSHOT ........................... SUCCESS [ 1.692 s] [INFO] rdbmswriter 0.0.1-SNAPSHOT ......................... SUCCESS [ 0.995 s] [INFO] hbase11xwriter 0.0.1-SNAPSHOT ...................... SUCCESS [ 4.564 s] [INFO] hbase094xwriter 0.0.1-SNAPSHOT ..................... SUCCESS [ 2.812 s] [INFO] hbase11xsqlwriter 0.0.1-SNAPSHOT ................... SUCCESS [ 7.919 s] [INFO] hbase11xsqlreader 0.0.1-SNAPSHOT ................... SUCCESS [ 8.665 s] [INFO] elasticsearchwriter 0.0.1-SNAPSHOT ................. SUCCESS [ 1.460 s] [INFO] tsdbwriter 0.0.1-SNAPSHOT .......................... SUCCESS [ 1.038 s] [INFO] adbpgwriter 0.0.1-SNAPSHOT ......................... SUCCESS [ 1.734 s] [INFO] gdbwriter 0.0.1-SNAPSHOT ........................... SUCCESS [ 3.027 s] [INFO] cassandrawriter 0.0.1-SNAPSHOT ..................... SUCCESS [ 1.564 s] [INFO] clickhousewriter 0.0.1-SNAPSHOT .................... SUCCESS [ 1.753 s] [INFO] oceanbasev10writer 0.0.1-SNAPSHOT .................. SUCCESS [ 4.283 s] [INFO] hbase20xsqlreader 0.0.1-SNAPSHOT ................... SUCCESS [ 2.773 s] [INFO] hbase20xsqlwriter 0.0.1-SNAPSHOT ................... SUCCESS [ 0.984 s] [INFO] kuduwriter 0.0.1-SNAPSHOT .......................... SUCCESS [ 2.710 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:59 min [INFO] Finished at: 2021-08-18T18:26:06+08:00 [INFO] ------------------------------------------------------------------------ [root@test DataX]#
注意:
目前源碼編譯完成后以下2個模板文件不存在，需要手動創建 /opt/datax_code2/DataX/target/datax/datax/plugin/reader/oceanbasev10reader/plugin_job_template.json /opt/datax_code2/DataX/target/datax/datax/plugin/writer/oceanbasev10writer/plugin_job_template.json
2、添加OB對應的插件模板文件
2.1 oceanbasev10reader 模板
雖然本次測試未使用到，便於后續的測試也一起添加。
[root@test oceanbasev10reader]# pwd
/opt/datax_code2/DataX/target/datax/datax/plugin/reader/oceanbasev10reader
[root@test oceanbasev10reader]# cat plugin_job_template.json
{
"name": "oceanbasev10reader",
"parameter": {
"username": "",
"password": "",
"readBatchSize": 100000,
"weakRead": false,
"column": [
"*"
],
"connection": [
{
"jdbcUrl": [
"||_dsc_ob10_dsc_||clusterName:tenantName||_dsc_ob10_dsc_||jdbc:oceanbase://obproxyIp:port/dbname"
],
"table": [
"tabName"
]
}
]
}
}
2.2 oceanbasev10writer 模板
說明：obWriteMode 群里老師提供的ob2ob.json里對應的是update，不是太理解，這里測試先調整成insert。
[root@test oceanbasev10writer]# pwd
/opt/datax_code/DataX/target/datax/datax/plugin/writer/oceanbasev10writer
[root@test oceanbasev10writer]# cat plugin_job_template.json
{
"name": "oceanbasev10writer",
"parameter": {
"obWriteMode": "insert",
"username": "",
"password": "",
"preSql": [
""
],
"column": [
"*"
],
"connection": [
{
"jdbcUrl": "||_dsc_ob10_dsc_||clusterName:tenantName||_dsc_ob10_dsc_||jdbc:oceanbase://obproxyIp:port/dbname?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true",
"table": [
"tabName"
]
}
]
}
}
3、同步數據
3.1 編輯JOB配置文件
[root@test bin]# cat mysql2ob.json
{
"job": {
"content": [
{
"reader": {
"name": "mysqlreader",
"parameter": {
"username": "root",
"password": "root654321",
"column": ["name"],
"splitPk": "id",
"connection": [
{
"table": [
"t1","t2"
],
"jdbcUrl": [
"jdbc:mysql://192.168.1.100:6008/db1"
]
},
{
"table": [
"t3","t4"
],
"jdbcUrl": [
"jdbc:mysql://192.168.1.100:6008/db2"
]
},
{
"table": [
"t5","t6"
],
"jdbcUrl": [
"jdbc:mysql://192.168.1.100:6008/db3"
]
},
{
"table": [
"t7","t8"
],
"jdbcUrl": [
"jdbc:mysql://192.168.1.100:6008/db4"
]
}
]
}
},
"writer": {
"name": "oceanbasev10writer",
"parameter": {
"column": [
"name"
],
"connection": [
{
"jdbcUrl": "||_dsc_ob10_dsc_||myob_test:test_tenant_1||_dsc_ob10_dsc_||jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true",
"table": [
"t_sum"
]
}
],
"obWriteMode": "insert",
"password": "root",
"preSql": [
"truncate table t_sum;"
],
"username": "root"
}
}
}
],
"setting": {
"speed": {
"channel": "1"
}
}
}
}
[root@test bin]#
3.2 啟動Datax同步數據
[root@test bin]# python datax.py mysql2ob.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2021-08-18 18:31:47.498 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2021-08-18 18:31:47.506 [main] INFO Engine - the machine info =>
osInfo: Oracle Corporation 1.8 25.77-b03
jvmInfo: Linux amd64 3.10.0-862.el7.x86_64
cpu num: 32
totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1
GC Names [PS MarkSweep, PS Scavenge]
MEMORY_NAME | allocation_size | init_size
PS Eden Space | 256.00MB | 256.00MB
Code Cache | 240.00MB | 2.44MB
Compressed Class Space | 1,024.00MB | 0.00MB
PS Survivor Space | 42.50MB | 42.50MB
PS Old Gen | 683.00MB | 683.00MB
Metaspace | -0.00MB | 0.00MB
2021-08-18 18:31:47.527 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"mysqlreader",
"parameter":{
"column":[
"name"
],
"connection":[
{
"jdbcUrl":[
"jdbc:mysql://192.168.1.100:6008/db1"
],
"table":[
"t1",
"t2"
]
},
{
"jdbcUrl":[
"jdbc:mysql://192.168.1.100:6008/db2"
],
"table":[
"t3",
"t4"
]
},
{
"jdbcUrl":[
"jdbc:mysql://192.168.1.100:6008/db3"
],
"table":[
"t5",
"t6"
]
},
{
"jdbcUrl":[
"jdbc:mysql://192.168.1.100:6008/db4"
],
"table":[
"t7",
"t8"
]
}
],
"password":"**********",
"splitPk":"id",
"username":"root"
}
},
"writer":{
"name":"oceanbasev10writer",
"parameter":{
"column":[
"name"
],
"connection":[
{
"jdbcUrl":"||_dsc_ob10_dsc_||myob_test:test_tenant_1||_dsc_ob10_dsc_||jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true",
"table":[
"t_sum"
]
}
],
"obWriteMode":"insert",
"password":"****",
"preSql":[
"truncate table t_sum;"
],
"username":"root"
}
}
}
],
"setting":{
"speed":{
"channel":"1"
}
}
}
2021-08-18 18:31:47.546 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null
2021-08-18 18:31:47.548 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2021-08-18 18:31:47.549 [main] INFO JobContainer - DataX jobContainer starts job.
2021-08-18 18:31:47.550 [main] INFO JobContainer - Set jobId = 0
Wed Aug 18 18:31:47 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:48.205 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.1.100:6008/db1?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
Wed Aug 18 18:31:48 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:48.233 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.1.100:6008/db2?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
Wed Aug 18 18:31:48 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:48.255 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.1.100:6008/db3?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
Wed Aug 18 18:31:48 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:48.281 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.1.100:6008/db4?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
Wed Aug 18 18:31:48 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:48.306 [job-0] INFO OriginalConfPretreatmentUtil - table:[t1] has columns:[id,name].
2021-08-18 18:31:48.320 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:48.320 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:48.579 [job-0] INFO DbUtils - value for query [SHOW VARIABLES LIKE 'ob_compatibility_mode'] is [MYSQL]
2021-08-18 18:31:48.587 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:48.587 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:48.601 [job-0] INFO OriginalConfPretreatmentUtil - table:[t_sum] all columns:[
id,name
].
2021-08-18 18:31:48.602 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:48.602 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:48.615 [job-0] INFO OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (name) VALUES(?)
], which jdbcUrl like:[||_dsc_ob10_dsc_||myob_test:test_tenant_1||_dsc_ob10_dsc_||jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]
2021-08-18 18:31:48.616 [job-0] INFO JobContainer - jobContainer starts to do prepare ...
2021-08-18 18:31:48.616 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2021-08-18 18:31:48.616 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] do prepare work .
2021-08-18 18:31:48.617 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:48.617 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:48.626 [job-0] INFO CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table t_sum;]. context info:||_dsc_ob10_dsc_||myob_test:test_tenant_1||_dsc_ob10_dsc_||jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2021-08-18 18:31:48.730 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:48.730 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:48.743 [job-0] INFO DbUtils - value for query [show variables like 'version'] is [3.1.0]
2021-08-18 18:31:48.743 [job-0] INFO JobContainer - jobContainer starts to do split ...
2021-08-18 18:31:48.743 [job-0] INFO JobContainer - Job set Channel-Number to 1 channels.
2021-08-18 18:31:48.752 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [8] tasks.
2021-08-18 18:31:48.753 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] splits to [8] tasks.
2021-08-18 18:31:48.779 [job-0] INFO JobContainer - jobContainer starts to do schedule ...
2021-08-18 18:31:48.789 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2021-08-18 18:31:48.792 [job-0] INFO JobContainer - Running by standalone Mode.
2021-08-18 18:31:48.801 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [8] tasks.
2021-08-18 18:31:48.807 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2021-08-18 18:31:48.807 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2021-08-18 18:31:48.867 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2021-08-18 18:31:48.869 [0-0-0-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2021-08-18 18:31:48.871 [0-0-0-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2021-08-18 18:31:48.871 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select name from t1
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db1?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
Wed Aug 18 18:31:48 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:48.880 [0-0-0-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2021-08-18 18:31:48.897 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select name from t1
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db1?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:48.900 [0-0-0-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2021-08-18 18:31:48.901 [0-0-0-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO t_sum (name) VALUES(?)
2021-08-18 18:31:48.902 [0-0-0-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:48.902 [0-0-0-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:48.912 [0-0-0-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2021-08-18 18:31:48.912 [0-0-0-writer] INFO ConcurrentTableWriterTask - start 1 insert task.
2021-08-18 18:31:48.915 [0-0-0-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:48.915 [0-0-0-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:48.931 [0-0-0-writer] INFO ColumnMetaCache - fetch columnMeta of table t_sum success
2021-08-18 18:31:48.949 [0-0-0-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false
2021-08-18 18:31:48.950 [0-0-0-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 1, finished = 0
2021-08-18 18:31:49.068 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[203]ms
2021-08-18 18:31:49.070 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2021-08-18 18:31:49.070 [0-0-1-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2021-08-18 18:31:49.070 [0-0-1-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select name from t2
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db1?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.070 [0-0-1-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
Wed Aug 18 18:31:49 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:49.079 [0-0-1-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2021-08-18 18:31:49.088 [0-0-1-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2021-08-18 18:31:49.089 [0-0-1-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO t_sum (name) VALUES(?)
2021-08-18 18:31:49.089 [0-0-1-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.089 [0-0-1-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.091 [0-0-1-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select name from t2
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db1?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.099 [0-0-1-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2021-08-18 18:31:49.099 [0-0-1-writer] INFO ConcurrentTableWriterTask - start 1 insert task.
2021-08-18 18:31:49.099 [0-0-1-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.100 [0-0-1-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.124 [0-0-1-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false
2021-08-18 18:31:49.124 [0-0-1-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 1, finished = 0
2021-08-18 18:31:49.170 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[100]ms
2021-08-18 18:31:49.172 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[2] attemptCount[1] is started
2021-08-18 18:31:49.172 [0-0-2-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2021-08-18 18:31:49.172 [0-0-2-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select name from t3
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db2?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.173 [0-0-2-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
Wed Aug 18 18:31:49 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:49.181 [0-0-2-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2021-08-18 18:31:49.191 [0-0-2-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2021-08-18 18:31:49.192 [0-0-2-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO t_sum (name) VALUES(?)
2021-08-18 18:31:49.192 [0-0-2-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.192 [0-0-2-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.197 [0-0-2-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select name from t3
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db2?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.199 [0-0-2-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2021-08-18 18:31:49.200 [0-0-2-writer] INFO ConcurrentTableWriterTask - start 1 insert task.
2021-08-18 18:31:49.200 [0-0-2-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.200 [0-0-2-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.223 [0-0-2-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false
2021-08-18 18:31:49.224 [0-0-2-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 1, finished = 0
2021-08-18 18:31:49.272 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[2] is successed, used[100]ms
2021-08-18 18:31:49.274 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[3] attemptCount[1] is started
2021-08-18 18:31:49.274 [0-0-3-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2021-08-18 18:31:49.275 [0-0-3-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select name from t4
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db2?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.275 [0-0-3-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
Wed Aug 18 18:31:49 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:49.283 [0-0-3-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2021-08-18 18:31:49.290 [0-0-3-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2021-08-18 18:31:49.291 [0-0-3-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO t_sum (name) VALUES(?)
2021-08-18 18:31:49.291 [0-0-3-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.291 [0-0-3-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.295 [0-0-3-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select name from t4
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db2?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.299 [0-0-3-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2021-08-18 18:31:49.300 [0-0-3-writer] INFO ConcurrentTableWriterTask - start 1 insert task.
2021-08-18 18:31:49.300 [0-0-3-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.300 [0-0-3-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.319 [0-0-3-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false
2021-08-18 18:31:49.319 [0-0-3-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 1, finished = 0
2021-08-18 18:31:49.375 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[3] is successed, used[101]ms
2021-08-18 18:31:49.377 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[4] attemptCount[1] is started
2021-08-18 18:31:49.377 [0-0-4-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2021-08-18 18:31:49.377 [0-0-4-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select name from t5
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db3?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.378 [0-0-4-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
Wed Aug 18 18:31:49 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:49.388 [0-0-4-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2021-08-18 18:31:49.397 [0-0-4-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2021-08-18 18:31:49.397 [0-0-4-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO t_sum (name) VALUES(?)
2021-08-18 18:31:49.397 [0-0-4-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.397 [0-0-4-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.398 [0-0-4-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select name from t5
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db3?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.406 [0-0-4-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2021-08-18 18:31:49.407 [0-0-4-writer] INFO ConcurrentTableWriterTask - start 1 insert task.
2021-08-18 18:31:49.407 [0-0-4-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.407 [0-0-4-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.428 [0-0-4-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false
2021-08-18 18:31:49.428 [0-0-4-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 1, finished = 0
2021-08-18 18:31:49.478 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[4] is successed, used[101]ms
2021-08-18 18:31:49.480 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[5] attemptCount[1] is started
2021-08-18 18:31:49.480 [0-0-5-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2021-08-18 18:31:49.480 [0-0-5-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select name from t6
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db3?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.480 [0-0-5-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
Wed Aug 18 18:31:49 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:49.489 [0-0-5-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2021-08-18 18:31:49.498 [0-0-5-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2021-08-18 18:31:49.498 [0-0-5-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO t_sum (name) VALUES(?)
2021-08-18 18:31:49.498 [0-0-5-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.498 [0-0-5-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.503 [0-0-5-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select name from t6
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db3?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.507 [0-0-5-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2021-08-18 18:31:49.508 [0-0-5-writer] INFO ConcurrentTableWriterTask - start 1 insert task.
2021-08-18 18:31:49.508 [0-0-5-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.508 [0-0-5-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.532 [0-0-5-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false
2021-08-18 18:31:49.532 [0-0-5-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 1, finished = 0
2021-08-18 18:31:49.580 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[5] is successed, used[100]ms
2021-08-18 18:31:49.582 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[6] attemptCount[1] is started
2021-08-18 18:31:49.582 [0-0-6-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2021-08-18 18:31:49.583 [0-0-6-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select name from t7
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db4?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.583 [0-0-6-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
Wed Aug 18 18:31:49 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:49.592 [0-0-6-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2021-08-18 18:31:49.600 [0-0-6-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2021-08-18 18:31:49.600 [0-0-6-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO t_sum (name) VALUES(?)
2021-08-18 18:31:49.600 [0-0-6-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.600 [0-0-6-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.602 [0-0-6-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select name from t7
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db4?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.608 [0-0-6-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2021-08-18 18:31:49.608 [0-0-6-writer] INFO ConcurrentTableWriterTask - start 1 insert task.
2021-08-18 18:31:49.608 [0-0-6-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.608 [0-0-6-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.630 [0-0-6-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false
2021-08-18 18:31:49.630 [0-0-6-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 1, finished = 0
2021-08-18 18:31:49.683 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[6] is successed, used[101]ms
2021-08-18 18:31:49.685 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[7] attemptCount[1] is started
2021-08-18 18:31:49.685 [0-0-7-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2021-08-18 18:31:49.685 [0-0-7-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select name from t8
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db4?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.686 [0-0-7-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
Wed Aug 18 18:31:49 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-18 18:31:49.693 [0-0-7-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2021-08-18 18:31:49.703 [0-0-7-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2021-08-18 18:31:49.703 [0-0-7-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO t_sum (name) VALUES(?)
2021-08-18 18:31:49.703 [0-0-7-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.703 [0-0-7-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.705 [0-0-7-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select name from t8
] jdbcUrl:[jdbc:mysql://192.168.1.100:6008/db4?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2021-08-18 18:31:49.710 [0-0-7-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2021-08-18 18:31:49.710 [0-0-7-writer] INFO ConcurrentTableWriterTask - start 1 insert task.
2021-08-18 18:31:49.711 [0-0-7-writer] INFO DBUtil - this is ob1_0 jdbc url.
2021-08-18 18:31:49.711 [0-0-7-writer] INFO DBUtil - this is ob1_0 jdbc url. user=myob_test:test_tenant_1:root :url=jdbc:oceanbase://192.168.1.100:12881/db1?yearIsDateType=false&ZeroDateTimeBehavior=convertToNull&tinyIntlisBit=false&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2021-08-18 18:31:49.732 [0-0-7-writer] INFO CommonRdbmsWriter$Task - isMemstoreFull=false
2021-08-18 18:31:49.732 [0-0-7-writer] INFO ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0, total = 1, finished = 0
2021-08-18 18:31:49.785 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[7] is successed, used[100]ms
2021-08-18 18:31:49.786 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks.
2021-08-18 18:31:58.868 [job-0] INFO StandAloneJobContainerCommunicator - Total 16 records, 16 bytes | Speed 1B/s, 1 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2021-08-18 18:31:58.868 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2021-08-18 18:31:58.868 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] do post work.
2021-08-18 18:31:58.868 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do post work.
2021-08-18 18:31:58.868 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
2021-08-18 18:31:58.869 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /opt/datax_code2/DataX/target/datax/datax/hook
2021-08-18 18:31:58.870 [job-0] INFO JobContainer -
[total cpu info] =>
averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00%
[total gc info] =>
NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCTime
PS MarkSweep | 1 | 1 | 1 | 0.027s | 0.027s | 0.027s
PS Scavenge | 1 | 1 | 1 | 0.020s | 0.020s | 0.020s
2021-08-18 18:31:58.870 [job-0] INFO JobContainer - PerfTrace not enable!
2021-08-18 18:31:58.871 [job-0] INFO StandAloneJobContainerCommunicator - Total 16 records, 16 bytes | Speed 1B/s, 1 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2021-08-18 18:31:58.872 [job-0] INFO JobContainer -
任務啟動時刻 : 2021-08-18 18:31:47
任務結束時刻 : 2021-08-18 18:31:58
任務總計耗時 : 11s
任務平均流量 : 1B/s
記錄寫入速度 : 1rec/s
讀出記錄總數 : 16
讀寫失敗總數 : 0
[root@test bin]#
4、目標端OB租戶下驗證數據
MySQL [db1]> show tenant; +---------------------+ | Current_tenant_name | +---------------------+ | test_tenant_1 | +---------------------+ 1 row in set (0.003 sec) MySQL [db1]> select * from t_sum; +----+------+ | id | name | +----+------+ | 1 | a | | 2 | b | | 3 | c | | 4 | d | | 5 | e | | 6 | f | | 7 | g | | 8 | h | | 9 | i | | 10 | j | | 11 | k | | 12 | l | | 13 | m | | 14 | n | | 15 | o | | 16 | p | +----+------+ 16 rows in set (0.003 sec)