1.下載spark2.4.3 使用用戶的hadoop的版本,解壓並放到/usr/local下並改名為spark目錄
2.設置spark目錄為本用戶所有
3.設置環境變量
(1)#~/.bashrc
export SPARK_HOME=/usr/local/spark
source ~/.bashrc
(2)cp /usr/local/spark/conf/spark-env.sh.template /usr/local/spark/conf/spark-env.sh
(3)進入 /usr/local/spark/conf/spark-env.sh
export SPARK_DIST_CLASSPATH=$(/usr/local/hadoop/bin/hadoop classpath)
export HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop
SPARK_LOCAL_IP="127.0.0.1"
4.單機模式
/usr/local/spark/bin/spark-shell
5.增加內容到最外層標簽內,文件是/usr/local/hadoop/etc/hadoop/yarn-site.xml
執行(yarn模式)
/usr/local/spark/bin/spark-shell --master yarn --deploy-mode client