SPARK SQL ERROR: Detected cartesian product for INNER join between logical plans报错解决方法

本文转载自查看原文 2019-09-29 09:59 1781 spark/ SPARK SQL ERROR: Detected cartesian product for IN

SparkSql运行程序报错，

Exception in thread "main" org.apache.spark.sql.AnalysisException: Detected cartesian product for INNER join between logical plans

解决方式：设置spark.sql.crossJoin.enabled=true

因为，2.x中默认不支持笛卡尔积操作，需要通过参数spark.sql.crossJoin.enabled开启

程序代码里面开始笛卡尔积操作，如下示：

val sc: SparkSession = SparkSession.builder
  .appName("My Spark Application")  // optional and will be autogenerated if not specified
  .master("local[*]")               // avoid hardcoding the deployment environment
  .config("spark.debug.maxToStringFields", "200")
 .config("spark.sql.crossJoin.enabled","true")
  .getOrCreate

val rst = discountFinancial.run(sc)

免责声明！

本站转载的文章为个人学习借鉴使用，本站对版权不负任何法律责任。如果侵犯了您的隐私权益，请联系本站邮箱yoyou2525@163.com删除。

猜您在找 Detected cartesian product for LEFT OUTER join SQL ----------- join （inner join 内连接） sql语法：inner join on, left join on, right join on详细使用方法 sql语法：inner join on, left join on, right join on详细使用方法 sql语句的group by 与 inner join SQL中INNER JOIN的用法【其他】【PL/SQL Developer】【2】报错Initialization error Could not load ".../oci.dll"解决方法 SQL中LEFT JOIN 和 inner join 的区别 SQL中使用update inner join和delete inner join SQL中使用update inner join和delete inner join