Flink 1.4沒出來以前,一直使用Flink 1.3.2,感覺還算穩定,最近將運行環境升級到1.4,遇到了一些坑:
1.需要將可運行程序,基於1.4.0重新編譯一次
2.對比了一下flink-conf.yml中的配置,發現一處不同點:
# The port under which the web-based runtime monitor listens.
# A value of -1 deactivates the web server.
1.3.2:jobmanager.web.port: 8081
1.4.0:web.port: 8081
1.4.0 少了前面的“jobmanager”,但是查看官網1.4.0的文檔,配置項卻是:jobmanager.web.port(https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html)
查看源代碼,原來是兩個都行,不過后面應該建議都用web開頭的,改了還不少:
3.在Configuration的Common Options部分,1.4多了如下的東西:
classloader.resolve-order: Whether Flink should use a child-first ClassLoader when loading
user-code classes or a parent-first ClassLoader. Can be one of parent-first or child-first. (default: child-first) classloader.parent-first-patterns: A (semicolon-separated) list of patterns that specifies
which classes should always be resolved through the parent ClassLoader first.
A pattern is a simple prefix that is checked against the fully qualified class name.
By default, this is set to java.;org.apache.flink.;javax.annotation;org.slf4j;org.apache.log4j;org.apache.logging.log4j;ch.qos.logback.
If you want to change this setting you have to make sure to also include the default patterns in your list of patterns if you want to keep that default behaviour.
Flink程序,原來在1.3.2上跑着沒問題,換到1.4后,總是報:
java.lang.LinkageError: loader constraint violation:
loader (instance of org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader)
previously initiated loading for a different type with name "scala/collection/Iterable"
查看發布說明,才知道默認類加載順序變了:
https://flink.apache.org/news/2017/12/12/release-1.4.0.html
在flink-conf.yml中配置了
classloader.resolve-order: parent-first
后問題解決
所以每個版本的發布說明,得仔細看清楚。