hive 中使用lzo
1 啟動hive 錯誤
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
解決方法是,安裝Hadoop時,修改Hadoop目錄下/conf/hadoop-env.sh時,添加HADOOP_CLASSPATH變量覆蓋了原有的變量,改成如下的形式即可:
HADOOP_CLASSPATH=$HADOOP_CLASSPATH:
2 創建測試表
create table lzo(name string) STORED AS INPUTFORMAT 'com.hadoop.mapred.DeprecatedLzoTextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat';
3 壓縮測試數據
lzop users.txt
4 導入數據
load data local inpath '/home/hadoop/tmp/tmplzo.txt.lzo' into table lzo;
4 測試
select * from lzo;