clickhouse 輸入輸出格式

TabSeparated、TabSeparatedRaw、TabSeparatedWithNames和TabSeparatedWithNamesAndTypes

TabSeparated

默認格式，縮寫:TSV，換行\n、制表符\t等符號依然顯示為\n、\t

列之間使用tab制表符分隔，數據按行寫入，每行結尾必須有換行符

可用於查詢和插入

CREATE TABLE tsv_demo(srcip String, destip String, time String) ENGINE = TinyLog;

 

clickhouse-client --query="INSERT INTO test.tsv_demo FORMAT TabSeparated" --max_insert_block_size=100000 < tab.txt

# cat tab.txt 
127.0.0.1    192.168.1.11    2020-08-24 17:43:00
192.168.1.100    192.168.1.101    2020-08-24 17:44:00

[root@ch2 tmp]# clickhouse-client 
ClickHouse client version 20.3.9.70 (official build).
Connecting to localhost:9300 as user default.
Connected to ClickHouse server version 20.3.9 revision 54433.

ch2 :) select * from test.tsv_demo;

SELECT *
FROM test.tsv_demo

┌─srcip─────────┬─destip────────┬─time────────────────┐
│ 127.0.0.1     │ 192.168.1.11  │ 2020-08-24 17:43:00 │
│ 192.168.1.100 │ 192.168.1.101 │ 2020-08-24 17:44:00 │
└───────────────┴───────────────┴─────────────────────┘

2 rows in set. Elapsed: 0.005 sec. 

ch2 :) exit
Bye.
[root@ch2 tmp]# myc
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 20.3.9.70-ClickHouse 

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> select * from test.tsv_demo;
+---------------+---------------+---------------------+
| srcip         | destip        | time                |
+---------------+---------------+---------------------+
| 127.0.0.1     | 192.168.1.11  | 2020-08-24 17:43:00 |
| 192.168.1.100 | 192.168.1.101 | 2020-08-24 17:44:00 |
+---------------+---------------+---------------------+
2 rows in set (0.00 sec)
Read 2 rows, 139.00 B in 0.001 sec., 1790 rows/sec., 121.53 KiB/sec.

 

TabSeparated可以縮寫為TSV

clickhouse-client --query="INSERT INTO test.tsv_demo FORMAT TSV" --max_insert_block_size=100000 < tab.txt

 

ch2 :) select * from test.tsv_demo format TSV;

SELECT *
FROM test.tsv_demo
FORMAT TSV

127.0.0.1    192.168.1.11    2020-08-24 17:43:00
192.168.1.100    192.168.1.101    2020-08-24 17:44:00
127.0.0.1    192.168.1.11    2020-08-24 17:43:00
192.168.1.100    192.168.1.101    2020-08-24 17:44:00

 

 

TabSeparatedRaw

TabSeparatedRaw格式可以解析制表、轉行符，縮寫TSVRaw，TSV不解析
只能在數據查詢的時候使用

查詢時如果列值中包括\n，則默認為該格式，即select * from a 的效果等價於 select * from a format TSVRaw;

 

建表語句：
create table escape_demo(name String, addr String, age UInt8, desc String) ENGINE=TinyLog;

 

# cat rw.txt

xiaomi nanjing\tjiangsu 23 From nanjing
xiaohong zheng\tzhou 23 \x41 gril

 

clickhouse-client --query="insert into test.escape_demo format TSV" < rw.txt

ch2 :) select * from escape_demo;

SELECT *
FROM escape_demo

┌─name─────┬─addr───────────────┬─age─┬─desc─────────┐
│ xiaomi   │ nanjing    jiangsu │  23 │ From nanjing │
│ xiaohong │ zheng    zhou    │  23 │ A gril       │
└──────────┴────────────────────┴─────┴──────────────┘

 

ch2 :) select * from escape_demo format TSVRaw;

SELECT *
FROM escape_demo
FORMAT TSVRaw

xiaomi    nanjing    jiangsu    23    From nanjing
xiaohong    zheng    zhou    23    A gril

2 rows in set. Elapsed: 0.005 sec. 

ch2 :) select * from escape_demo format TSV;

SELECT *
FROM escape_demo
FORMAT TSV

xiaomi    nanjing\tjiangsu    23    From nanjing
xiaohong    zheng\tzhou    23    A gril

2 rows in set. Elapsed: 0.004 sec. 

 

 

TabSeparatedWithNames

在查詢時，TabSeparatedWithNames格式的第一行顯示列的名稱
在數據導入時，第一行完全被忽略，不會解析第一行為表頭。
簡稱：TSVWithNames
在數據查詢和數據導入均可使用。

ch2 :) select * from escape_demo format TSVWithNames;

SELECT *
FROM escape_demo
FORMAT TSVWithNames

name    addr    age    desc
xiaomi    nanjing\tjiangsu    23    From nanjing
xiaohong    zheng\tzhou    23    A gril

 

 

數據導入：
clickhouse-client --query="INSERT INTO escape_demo FORMAT TSVWithNames" < rw.txt 

ch2 :) select * from escape_demo;

SELECT *
FROM escape_demo

┌─name─────┬─addr───────────────┬─age─┬─desc─────────┐
│ xiaomi   │ nanjing    jiangsu │  23 │ From nanjing │
│ xiaohong │ zheng    zhou    │  23 │ A gril       │
│ xiaohong │ zheng    zhou    │  23 │ A gril       │
└──────────┴────────────────────┴─────┴──────────────┘

 

rw.txt文件有兩行數據，再次導入應該有四行，但使用該格式會認為rw.txt文件的第一行是列的名稱，被忽略

 

TabSeparatedWithNamesAndTypes

在查詢時，TabSeparatedWithNamesAndTypes格式在主數據的前面額外顯示兩行數據，第一行顯示列的名稱， 第二行顯示列的數據類型。
在數據導入時，前面兩行的數據完全被忽略。
簡寫：TSVWithNamesAndTypes
在數據導入和查詢時均可使用。

ch2 :) select * from escape_demo format TSVWithNamesAndTypes;

SELECT *
FROM escape_demo
FORMAT TSVWithNamesAndTypes

name    addr    age    desc
String    String    UInt8    String
xiaomi    nanjing\tjiangsu    23    From nanjing
xiaohong    zheng\tzhou    23    A gril
xiaohong    zheng\tzhou    23    A gril

 

導入時前兩行被忽略 ，rw.txt只有兩行，所以導入后數據無變化

[root@ch2 tmp]# clickhouse-client --query="INSERT INTO escape_demo FORMAT TSVWithNamesAndTypes" < rw.txt 
[root@ch2 tmp]# clickhouse-client 
ClickHouse client version 20.3.9.70 (official build).
Connecting to localhost:9300 as user default.
Connected to ClickHouse server version 20.3.9 revision 54433.

ch2 :) select * from escape_demo;

SELECT *
FROM escape_demo

┌─name─────┬─addr───────────────┬─age─┬─desc─────────┐
│ xiaomi   │ nanjing    jiangsu │  23 │ From nanjing │
│ xiaohong │ zheng    zhou    │  23 │ A gril       │
│ xiaohong │ zheng    zhou    │  23 │ A gril       │
└──────────┴────────────────────┴─────┴──────────────┘