使用spark查詢hive表時,報錯如下:
20/03/23 11:24:10 INFO StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint Error in query: Table or view not found: bili_manga.homepage_mangacard_user_norm_onehot; line 19 pos 6 20/03/23 11:24:11 INFO SparkUI: Stopped Spark web UI at http://xxx:10005 20/03/23 11:24:11 INFO YarnClientSchedulerBackend: Interrupting monitor thread 20/03/23 11:24:12 INFO YarnClientSchedulerBackend: Shutting down all executors 20/03/23 11:24:12 INFO YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down 20/03/23 11:24:12 INFO SchedulerExtensionServices: Stopping SchedulerExtensionServices (serviceOption=None, services=List(),
但是使用hive引擎查詢表存在且有結果的,$SPARK_HOME/conf下 也已經存放hive-site.xml文件。
查詢sql部分如下
LEFT JOIN ( SELECT * FROM `bili_manga`.`homepage_mangacard_item_norm_onehot` WHERE dt = "<%=log_date-1%>" ) c ON a.manga_id = c.item_id LEFT JOIN ( SELECT
將反引號去掉即可正常查詢。