flink異常

Task xxx did not react to cancelling signal in the last 30 seconds, but is stuck in method

兩種可能

第一種是程序30秒內沒有做出反應導致taskmanager掛掉

解決辦法  在flink-conf.yaml中配置task.cancellation.timeout: 0

第二種是任務被阻塞在某個方法里

解決辦法看日志找到被阻塞的方法進行解決

 

 

解決zookeeper中的org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss問題

修改flink conf下的zoo.cfg文件

 

 

804: Could not resolve substitution to a value: ${akka.stream.materializer}

 解決：

<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
     <resource>reference.conf</resource>
</transformer>

<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.3</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <!--打jar包-->
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>com.allen.capturewebdata.Main</mainClass>
                        </manifest>
                    </archive>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.3</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <transformers>
                                <!--<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>flink.KafkaDemo1</mainClass>
                                </transformer>-->
                                <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                                    <resource>reference.conf</resource>
                                </transformer>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

　　

 

2021-07-23 13:32:38,844 ERROR org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Unhandled exception.
akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://flink@ip-hd:43085/user/dispatcher#-1808307521]] after [150000 ms]. Message of type [org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply.