(一) flink 1.11.2 下載地址 :
https://mirrors.tuna.tsinghua.edu.cn/apache/flink/flink-1.11.2/python/apache_flink-1.11.2-cp37-cp37m-manylinux1_x86_64.whl
https://mirrors.tuna.tsinghua.edu.cn/apache/flink/flink-1.11.2/
( 二) 關聯包
下載以下依賴 jar 包,並拷貝到 flink-1.11.0/lib/
目錄下。因為我們運行時需要依賴各個 connector 實現。可以在GitHub上面下載 https://github.com/peigenxiao/my-flink-project/tree/master/flinklib
(三)設置
export FLINK_HOME=/home/hadoop/flink-1.11.2 export PATH=$PATH:$HADOOP_HOME/bin source /etc/profile # sh /home/hadoop/flink-1.11.2/bin/start-cluster.sh export FLINK_HOME=/home/hadoop/flink-1.11.2 cd /home/hadoop/flink-1.11.2/conf/flink-conf.yaml vi flink-conf.yaml jobmanager.rpc.address:master rest.port: 8091 #8091端口設置后不起作用? #jobmanager.memory.process.size: 300m #taskmanager.memory.process.size: 300m vi master master:8091 vi workers slave1 slave2 scp -r /home/hadoop/flink-1.11.2/ hadoop@slave1:/home/hadoop/ scp -r /home/hadoop/flink-1.11.2/ hadoop@slave2:/home/hadoop/ scp /home/hadoop/flink-1.11.2/conf/flink-conf.yaml hadoop@slave1:/home/hadoop/flink-1.11.2/conf/
(四)測試
cd /home/hadoop/flink-1.11.2/bin
啟動
start-cluster.sh
./flink run /home/hadoop/flink-1.11.2/examples/streaming/WordCount.jar
登錄web控制台查看結果。
http://192.168.91.112:8091
停止 start-cluster.sh
(五)CDC 測試使用:
Flink1.11中的CDC Connectors操作實踐 【https://blog.csdn.net/jmx_bigdata/article/details/107993482】
Flink 1.11 SQL 使用攻略 【 https://www.jianshu.com/p/8f2a40ec226d】
flink 1.11 sqlclint 官方文檔【https://ci.apache.org/projects/flink/flink-docs-master/zh/dev/table/sqlClient.html#environment-files】
Hive Integration 【https://ci.apache.org/projects/flink/flink-docs-master/dev/table/hive/】
Hive Read & Write 【https://ci.apache.org/projects/flink/flink-docs-master/dev/table/hive/hive_read_write.html】
hive與flink的流批一體設計方式【https://mp.weixin.qq.com/s?__biz=MzU3Mzg4OTMyNQ==&mid=2247489376&idx=1&sn=000fda2c695c88a6542f5cebb3207b64&chksm=fd3b9922ca4c1034b8bc0bff6d091c31c13b41443c3a71f5420007d91bc24b60bb102d6a9a94&xtrack=1&scene=90&subscene=93&sessionid=1601205791&clicktime=1601205953&enterid=1601205953&ascene=56&devicetype=android-26&version=3.0.31.2998&nettype=cmnet&abtest_cookie=AAACAA%3D%3D&lang=zh_CN&exportkey=AeaDMVfzSL04VBtRUGcfL2Q%3D&pass_ticket=%2BVHG5JT6TAPRJ0%2BEgPbDiMJd0tzudJm5cBMmrmzu1D1RF1aDyY2nyEIuVQLSQS23&wx_header=1&platform=win】
flink 先加載 child class的設置【https://blog.csdn.net/weixin_44904816/article/details/108250776】