Exception: Python in worker has different version 2.6 than that in driver 2.7, PySpark cannot run


Python版本不一致導致的,設置下環境變量即可。

Exception: Python in worker has different version 2.6 than that in driver 2.7, PySpark cannot run with different minor versions.Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set.

python --version 發現是2.6

echo $PYSPARK_PYTHON 值為空

echo $PYSPARK_DRIVER_PYTHON 值是ipython

which ipython發現路徑是

/home/work/envs/python_2.7.13/bin/ipython

設置環境變量,問題解決:

export PYSPARK_PYTHON=/home/work/envs/python_2.7.13/bin/ipython

 

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM