Datax將本地文件導入Hbase數據庫！！！酷酷酷

Hbase Writer的json文件鏈接：

https://github.com/alibaba/DataX/blob/master/hbase11xwriter/doc/hbase11xwriter.md

配置json文件

{
  "job": {
    "setting": {
      "speed": {
        "channel": 5
      }
    },
    "content": [
      {
        "reader": {
          "name": "txtfilereader",
          "parameter": {
            "path": "/home/baoshan/ml-1m/movies.dat",
            "charset": "UTF-8",
            "column": [
              {
                "index": 0,
                "type": "String"
              },
              {
                "index": 1,
                "type": "string"
              },
              {
                "index": 2,
                "type": "string"
              }
            ],
            "fieldDelimiter": ""
          }
        },
        "writer": {
          "name": "hbase11xwriter",
          "parameter": {
            "hbaseConfig": {
              "hbase.rootdir": "xxxxxx",
              "hbase.cluster.distributed": "true",
              "hbase.zookeeper.quorum": "xxxxxx"
            },
            "table": "writer",
            "mode": "normal",
            "rowkeyColumn": [
                {
                  "index":0,
                  "type":"string"
                }
            ],
            "column": [
              {
                "index":1,
                "name": "cf1:q1",
                "type": "string"
              },
              {
                "index":2,
                "name": "cf1:q2",
                "type": "string"
              }
            ],
            "versionColumn":{
              "index": -1,
              "value":"123456789"
            },
            "encoding": "utf-8"
          }
        }
      }
    ]
  }
}

可以同時導入多個本地文件，采用默認匹配符號*，例如有example開頭的多個文件，則寫為example*，則會並發導入多個文件

執行

python bin/datax.py local2hbase.json

 

親測，批量導入hbase數據，完美