Flink1.10.1編譯hadoop2.7.2 編譯flink-shaded-hadoop-2-uber

Flink1.10.1編譯hadoop2.7.2 編譯flink-shaded-hadoop-2-uber

從Flink 1.11開始，flink-shaded-hadoop-2-uberFlink項目不再正式支持使用發行版。
如果想建立flink-shaded對供應商特定的Hadoop版本，您必須首先描述配置特定供應商的Maven倉庫在本地Maven安裝在這里。
運行以下命令以flink-shaded針對所需的Hadoop版本（例如對於version 2.6.5-custom）進行構建和安裝：

mvn clean install -Dhadoop.version=2.6.5-custom

1. 下載/flink-shaded

前往git 下載/flink-shaded 下載relese-10.0 因為10.0是包含hadoop-bin的，1.11 不包含hadoop

https://github.com/apache/flink-shaded.git

2. 解壓至linux

tar -zxvf flink-shaded-release-10.0.tar.gz

3. 進入flink-shaded-hadoop-2-parent，修改pom

[ryxiong@node01 flink-shaded-release-10.0]$ cd flink-shaded-hadoop-2-parent/

修改flink-shaded-hadoop-2-uber下pom.xml文件, 執行zookeeper的版本

[ryxiong@node01 flink-shaded-hadoop-2-parent]$ cd flink-shaded-hadoop-2-uber/
[ryxiong@node01 flink-shaded-hadoop-2-uber]$ vim pom.xml 

4. 進入flink-shaded-release-10.0下 修改pom

[ryxiong@node01 job]$ cd flink-shaded-release-10.0/
[ryxiong@node01 flink-shaded-release-10.0]$ vim pom.xml 

在profiles標簽中添加如下內容

 <profile>
    <id>vendor-repos</id>
    <activation>
        <property>
            <name>vendor-repos</name>
        </property>
    </activation>   <!-- Add vendor maven repositories -->
    <repositories>       <!-- Cloudera -->
        <repository>
            <id>cloudera-releases</id>
            <url>https://maven.aliyun.com/repository/central</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>
</profile>

添加阿里雲鏡像, 編譯過程中下載塊

5. 配置完畢，運行前提是需要linux配置maven

在flink-shaded-release-10.0執行

mvn clean install -DskipTests -Dhadoop.version=2.7.2 -Drat.skip=true

進入漫長的編譯過程, 沒有ERROR即成功

[INFO] Reactor Summary:
[INFO]
[INFO] flink-shaded ...................................... SUCCESS [1.266s]
[INFO] flink-shaded-force-shading ........................ SUCCESS [0.383s]
[INFO] flink-shaded-asm-7 ................................ SUCCESS [0.464s]
[INFO] flink-shaded-guava-18 ............................. SUCCESS [0.882s]
[INFO] flink-shaded-netty-4 .............................. SUCCESS [4.089s]
[INFO] flink-shaded-netty-tcnative-dynamic ............... SUCCESS [0.457s]
[INFO] flink-shaded-jackson-parent ....................... SUCCESS [0.015s]
[INFO] flink-shaded-jackson-2 ............................ SUCCESS [0.816s]
[INFO] flink-shaded-jackson-module-jsonSchema-2 .......... SUCCESS [0.595s]
[INFO] flink-shaded-hadoop-2-parent ...................... SUCCESS [0.017s]
[INFO] flink-shaded-hadoop-2 ............................. SUCCESS [8.014s]
[INFO] flink-shaded-hadoop-2-uber ........................ SUCCESS [10.657s]
[INFO] flink-shaded-zookeeper-parent ..................... SUCCESS [0.011s]
[INFO] flink-shaded-zookeeper-3.4 ........................ SUCCESS [41.310s]
[INFO] flink-shaded-zookeeper-3.5 ........................ SUCCESS [33.234s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:42.440s
[INFO] Finished at: Fri Apr 09 16:56:24 CST 2021
[INFO] Final Memory: 45M/1112M
[INFO] ------------------------------------------------------------------------
[mac@ryxiong flink-shaded-release-10.0]$ll

編譯好的路徑

[mac@ryxiong target]$pwd
~/flink-shaded-release-10.0/flink-shaded-hadoop-2-parent/flink-shaded-hadoop-2-uber/target

6. 編譯好的jar包 放入flinke1.10.1的lib下

[ryxiong@node01 flink-shaded-release-10.0]$ cp flink-shaded-hadoop-2-uber-2.7.2-10.0.jar /opt/module/flink-1.10.1/lib/

我編譯好的jar包,有需要的可以下載.

https://download.csdn.net/download/Ryxiong728/16593328