hivesql遷移spark2.4 cannot resolve '(ctime >= start_time) due to data type mismatch: differing types in '(ctime` >= `start_time`)'(timestamp and bigint).; line 99 pos 10

hivesql 一些默認類型轉換。但spark需要手動強轉

比如以下部分:

                         SELECT
                            1 as id,
                            avid as avid,
                            mid as mid,
                            TRIM(LOWER(tag)) as tag,
                            ctime as ctime
                          FROM
                            archive.dws_archive_daily
                          WHERE
                            log_date = '20191203'
                        ) av3
                        INNER JOIN ad_tag on av3.id = ad_tag.id
                      WHERE
                        av3.ctime >= ad_tag.start_time
                        AND av3.ctime <= ad_tag.end_time

報錯如下:

    Error in query: cannot resolve '(av2.`ctime` >= ad_sub_tid.`start_time`)' due to data type mismatch: differing types in '(av2.`ctime` >= ad_sub_tid.`start_time`)' (timestamp and bigint).; line 99 pos 10;

實際上ctime為timestamp類型，而start_time為bigint(long)類型。

那么我們將語句顯示強轉

類型轉換

cast(av2.ctime as bigint) >= ad_sub_tid.start_time

或者

av2.ctime >= cast(ad_sub_tid.start_time as timestamp)語句都能順利執行。

遷移sql中並不想修改sql語句。就修改下源碼好了

查看代碼報錯位置在checkAnalysis.class中

 

跟下checkInputDataTypes代碼

 

 

 

 

 

 新增一行類型可以轉換的判斷 timstamp->bigint 返回true

 

 

 再次運行提交sparksql就可以了