如果是把數據放在了hdfs系統,那么我們如何訪問他們呢?
1.hdfs查看文件夾
./hdfs dfs -ls hdfs://mycluster/output/online/
2.hdfs創建目錄dfs創建文件夾
./hdfs dfs -mkdir hdfs://mycluster/output/online/
./hdfs dfs -mkdir hdfs://mycluster/output/online/pv
這里有個坑,如果你直接創建多級目錄,會得到錯誤提示,所以要一級一級的去建立目錄才行!
16/12/22 18:32:54 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
mkdir: `hdfs://mycluster/output/online/pv': No such file or directory
3.創建hive表
CREATE EXTERNAL TABLE `pv_table`(
`city_name` string,
`pv` string,
`product_line` string)
PARTITIONED BY (
`day` string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
'hdfs://mycluster/output/online/pv')