hive 將hive表數據查詢出來轉為json對象和json數組輸出

一、將hive表數據查詢出來轉為json對象輸出

1、將查詢出來的數據轉為一行一行，並指定分割符的數據

2、使用UDF函數，將每一行數據作為string傳入UDF函數中轉換為json再返回

1、准備數據

2、查詢出來的數據轉為一行一行，並指定分割符的數據

 

3、准備UDF函數

package com.laotou;

import org.apache.hadoop.hive.ql.exec.UDF;
import org.json.JSONException;
import org.json.JSONObject;

/**
 * @Author: 
 * @Date: 2019/8/9
 */
public class HiveJsonOut extends UDF{public static String evaluate(String jsonStr) throws JSONException {
        String[] split = jsonStr.split(",");
        JSONObject result = new JSONObject();
        result.put("key", split[0]);
        result.put("value", split[1]);
        return String.valueOf(result);
    }
}

package com.laotou;

import org.apache.hadoop.hive.ql.exec.UDF;
import org.json.JSONException;
import org.json.JSONObject;

/**
 * @Author:
 * string轉json:{"notifyType":13,"notifyEntity":{"school":"小學","name":"張三","age":"13"}}
 * @Date: 2019/8/14
 */
public class Record2Notify extends UDF {
    private static final String split_char = "!";
    private static final String null_char = "\002";

    public static String evaluate(int type, String line) throws JSONException {
        if (line == null) {
            return null;
        }
        JSONObject notify = new JSONObject();
        JSONObject entity = new JSONObject();
        notify.put("notifyType", type);
        String[] columns = line.split(split_char, -1);
        int size = columns.length / 2;
        for (int i = 0; i < size; i++) {
            String key = columns[i*2];
            String value = columns[i*2+1];
            if (isNull(key)) {
                throw new JSONException("Null key.1111111111");
            }
            if (!isNull(value)) {
                entity.put(key, value);
            }
        }
        notify.put("notifyEntity", entity);

        return notify.toString();
    }

    private static boolean isNull(String value) {
        return value == null || value.isEmpty() || value.equals(null_char);
    }

    public static void main(String[] args) throws JSONException {
        System.out.println(evaluate(13,"name!張三!age!13!school!小學"));
    }
}

 

 

二、將hive表數據查詢出來轉為json數組輸出

思路：

1、使用UDF函數（見上面內容）將查詢出來的每一條數據轉成json對象

select getJsonOut(concat_ws(',',key,value)) as content from test1

2、將第一步查詢的結果進行列轉行，並設置為逗號進行分割，得到如下字符串

select concat_ws('!!',collect_list(bb.content)) as new_value 
 from 
(select getJsonOut(concat_ws(',',key,value)) as content from test1) bb;

結果如圖：

3、使用UDF函數（JsonArray）將第2步中得到的字符串放入數組對象,准備UDF函數

package com.laotou;

import org.apache.hadoop.hive.ql.exec.UDF;
import org.json.JSONArray;
import org.json.JSONException;

/**
 * create temporary function getJsonArray as 'com.laotou.HiveJson';
 * @Author:
 * @Date: 2019/8/9
 */
public class HiveJson extends UDF{
    public static JSONArray evaluate(String jsonStr) throws JSONException {
        String[] split = jsonStr.split("!!");
        JSONArray jsonArray = new JSONArray();
        jsonArray.put(split[0]);
        jsonArray.put(split[1]);
        jsonArray.put(split[2]);
        jsonArray.put(split[3]);
        return jsonArray;
    }

}

 4、測試

select getJsonArray(new_value) from 
(select cast(concat_ws('!!',collect_list(bb.content)) as string) as new_value from 
(select getJsonOut(concat_ws(',',key,value)) as content from test1) bb) cc;