【原創】大數據基礎之Ambari(3)通過Ambari部署Airflow


ambari2.7.3(hdp3.1) 安裝 airflow1.10

ambari的hdp中原生不支持airflow安裝,下面介紹如何通過mpack方式使ambari支持airflow安裝:

1 下載

# wget https://github.com/miho120/ambari-airflow-mpack/raw/master/airflow-service-mpack.tar.gz

2 安裝

# ambari-server install-mpack --mpack airflow-service-mpack.tar.gz --verbose

3 重啟

# ambari-server restart

4 ambari頁面沒有出現ambari,檢查mpack.json

/var/lib/ambari-server/resources/mpacks/airflow-ambari-mpack-1.5.4/mpack.json

      "service_versions_map": [

        {

          "service_name" : "AIRFLOW",

          "service_version" : "1.10.0",

          "applicable_stacks" : [

            {

              "stack_name" : "HDP",

              "stack_version" : "2.4"

            },

            {

              "stack_name" : "HDP",

              "stack_version" : "2.5"

            },

            {

              "stack_name" : "HDP",

              "stack_version" : "2.6"

            },

            {

              "stack_name" : "HDP",

              "stack_version" : "3.0"

            }

          ]

        }

      ]

需要增加HDP3.1,修改完重新打包

4 卸載

ambari-server uninstall-mpack --mpack-name=airflow-ambari-mpack

5 安裝(同上)

6 重啟(同上)

打開ambari頁面

 

啟動worker時可能失敗,查看啟動命令

2019-01-18 19:09:48,159 - Execute['service airflow-worker start'] {}

查看日志

journalctl -xe

發現有報錯

Jan 18 19:04:05 cdp-test-server-01.bj python[15246]: detected unhandled Python exception in '/var/lib/ambari-agent/cache/common-services/AIRFLOW/1.10.0/package/scripts/airflow_worker_control.py'
Jan 18 19:04:05 cdp-test-server-01.bj abrt-server[15254]: Executable '/var/lib/ambari-agent/cache/common-services/AIRFLOW/1.10.0/package/scripts/airflow_worker_control.py' doesn't belong to any package and ProcessUnpackaged is set t
Jan 18 19:04:05 cdp-test-server-01.bj abrt-server[15254]: 'post-create' on '/var/spool/abrt/Python-2019-01-18-19:04:05-15246' exited with 1
Jan 18 19:04:05 cdp-test-server-01.bj abrt-server[15254]: Deleting problem directory '/var/spool/abrt/Python-2019-01-18-19:04:05-15246'

可以通過修改配置解決

# grep ProcessUnpackaged /etc/abrt/abrt-action-save-package-data.conf
no

改為yes

 

 

 

ambari具體使用詳見:https://www.cnblogs.com/barneywill/p/10268501.html

 

參考:

https://github.com/miho120/ambari-airflow-mpack

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM