码上快乐

相关内容   简体   繁体

flink-playgrounds官方案例使用注意事项

本文转载自  查看原文  2021-11-25 15:57  855 

一、案例

地址：https://nightlies.apache.org/flink/flink-docs-release-1.14/zh/docs/try-flink/table_api/

二、问题

2.1 代码中"$"符号问题

 

 

 解决方案： import static org.apache.flink.table.api.Expressions.$; 

2.2  flink-playgrounds-master\table-walkthrough\Dockerfile 

（1）在宿主机修改maven的settings.xml文件，修改镜像仓库和代理，并通过 COPY 命令拷贝到容器中

（2）修改wget的代理

（3）修改wget的镜像

FROM maven:3.6-jdk-8-slim AS builder

COPY ./settings.xml /root/.m2/settings.xml
COPY ./pom.xml /opt/pom.xml
COPY ./src /opt/src

RUN cd /opt; mvn clean install -Dmaven.test.skip

FROM apache/flink:1.13.1-scala_2.12-java8

# Download connector libraries
# RUN wget -P /opt/flink/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-kafka_2.12/1.13.1/flink-sql-connector-kafka_2.12-1.13.1.jar; \
#    wget -P /opt/flink/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-connector-jdbc_2.12/1.13.1/flink-connector-jdbc_2.12-1.13.1.jar; \
#    wget -P /opt/flink/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-csv/1.13.1/flink-csv-1.13.1.jar; \
#    wget -P /opt/flink/lib/ https://repo.maven.apache.org/maven2/mysql/mysql-connector-java/8.0.19/mysql-connector-java-8.0.19.jar;

RUN wget  -e use_proxy=yes -e https_proxy=proxy_ip:proxy_port/ -P /opt/flink/lib/ https://maven.aliyun.com/nexus/content/groups/public/org/apache/flink/flink-sql-connector-kafka_2.12/1.13.1/flink-sql-connector-kafka_2.12-1.13.1.jar; \
    wget  -e use_proxy=yes -e https_proxy= -P /opt/flink/lib/ https://maven.aliyun.com/nexus/content/groups/public/org/apache/flink/flink-connector-jdbc_2.12/1.13.1/flink-connector-jdbc_2.12-1.13.1.jar; \
    wget  -e use_proxy=yes -e https_proxy= -P /opt/flink/lib/ https://maven.aliyun.com/nexus/content/groups/public/org/apache/flink/flink-csv/1.13.1/flink-csv-1.13.1.jar; \
    wget  -e use_proxy=yes -e https_proxy= -P /opt/flink/lib/ https://maven.aliyun.com/nexus/content/groups/public/mysql/mysql-connector-java/8.0.19/mysql-connector-java-8.0.19.jar;

2.3 flink-playgrounds-master\docker\data-generator\Dockerfile 

（1）在宿主机修改maven的settings.xml文件，修改镜像仓库和代理，并通过COPY命令拷贝到容器中

（2）给赋予 docker-entrypoint.sh 执行权限，否则执行 docker-compose up -d 报错,，提示：

ERROR: for data-generator  Cannot start service data-generator: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/docker-entrypoint.sh": permission denied: unknown

 

FROM maven:3.6-jdk-8-slim AS builder


# Get data producer code and compile it
COPY ./settings.xml /root/.m2/settings.xml
COPY ./src /opt/data-producer/src
COPY ./pom.xml /opt/data-producer/pom.xml

RUN cd /opt/data-producer; \
    mvn clean install

FROM openjdk:8-jre

COPY --from=builder /opt/data-producer/target/data-generator-*.jar /opt/data-generator.jar

RUN cd /opt

COPY docker-entrypoint.sh /

# 添加执行权限
RUN chmod +x /docker-entrypoint.sh

ENTRYPOINT ["/docker-entrypoint.sh"]

2.4 编译 data-generator 报错

Nov 25, 2021 3:43:23 AM com.diffplug.spotless.FormatExceptionPolicyLegacy error
SEVERE: Step 'google-java-format' found problem in 'src/main/java/org/apache/flink/playground/datagen/DataGenerator.java':
Unable to resolve dependencies
com.diffplug.spotless.maven.ArtifactResolutionException: Unable to resolve dependencies
        at com.diffplug.spotless.maven.ArtifactResolver.resolveDependencies(ArtifactResolver.java:95)
        at com.diffplug.spotless.maven.ArtifactResolver.resolve(ArtifactResolver.java:81)
        at com.diffplug.spotless.JarState.provisionWithTransitives(JarState.java:87)
        at com.diffplug.spotless.JarState.from(JarState.java:76)
        at com.diffplug.spotless.JarState.from(JarState.java:71)
        at com.diffplug.spotless.java.GoogleJavaFormatStep$State.<init>(GoogleJavaFormatStep.java:99)
        at com.diffplug.spotless.java.GoogleJavaFormatStep.lambda$create$0(GoogleJavaFormatStep.java:73)
        at com.diffplug.spotless.FormatterStepImpl.calculateState(FormatterStepImpl.java:56)
        at com.diffplug.spotless.LazyForwardingEquality.state(LazyForwardingEquality.java:56)
        at com.diffplug.spotless.FormatterStep$Strict.format(FormatterStep.java:76)
        at com.diffplug.spotless.Formatter.compute(Formatter.java:230)
        at com.diffplug.spotless.Formatter.isClean(Formatter.java:167)
        at com.diffplug.spotless.maven.SpotlessCheckMojo.process(SpotlessCheckMojo.java:51)
        at com.diffplug.spotless.maven.AbstractSpotlessMojo.execute(AbstractSpotlessMojo.java:117)
        at com.diffplug.spotless.maven.AbstractSpotlessMojo.execute(AbstractSpotlessMojo.java:110)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:193)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
Caused by: org.eclipse.aether.resolution.DependencyResolutionException: Failed to collect dependencies at com.google.googlejavaformat:google-java-format:jar:1.7 -> com.google.errorprone:javac-shaded:jar:9+181-r4173-1
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:353)
        at com.diffplug.spotless.maven.ArtifactResolver.resolveDependencies(ArtifactResolver.java:93)
        ... 36 more
Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at com.google.googlejavaformat:google-java-format:jar:1.7 -> com.google.errorprone:javac-shaded:jar:9+181-r4173-1
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:291)
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:309)
        ... 37 more
Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for com.google.errorprone:javac-shaded:jar:9+181-r4173-1
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:255)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.resolveCachedArtifactDescriptor(DefaultDependencyCollector.java:541)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.getArtifactDescriptorResult(DefaultDependencyCollector.java:524)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:412)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:365)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:352)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:509)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:461)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:365)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:352)
        at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:254)
        ... 38 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.google.errorprone:javac-shaded:pom:9+181-r4173-1 from/to alimaven (https://maven.aliyun.com/nexus/content/groups/public/): Authorization failed for https://maven.aliyun.com/nexus/content/groups/public/com/google/errorprone/javac-shaded/9+181-r4173-1/javac-shaded-9+181-r4173-1.pom 403 Forbidden
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:424)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:229)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:207)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:240)
        ... 49 more
Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact com.google.errorprone:javac-shaded:pom:9+181-r4173-1 from/to alimaven (https://maven.aliyun.com/nexus/content/groups/public/): Authorization failed for https://maven.aliyun.com/nexus/content/groups/public/com/google/errorprone/javac-shaded/9+181-r4173-1/javac-shaded-9+181-r4173-1.pom 403 Forbidden
        at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:52)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:369)
        at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:644)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:262)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:499)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:401)
        ... 52 more
Caused by: org.apache.maven.wagon.authorization.AuthorizationException: Authorization failed for https://maven.aliyun.com/nexus/content/groups/public/com/google/errorprone/javac-shaded/9+181-r4173-1/javac-shaded-9+181-r4173-1.pom 403 Forbidden
        at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1180)
        at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1138)
        at org.apache.maven.wagon.StreamWagon.getInputStream(StreamWagon.java:126)
        at org.apache.maven.wagon.StreamWagon.getIfNewer(StreamWagon.java:88)
        at org.apache.maven.wagon.StreamWagon.get(StreamWagon.java:61)
        at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunner.run(WagonTransporter.java:567)
        at org.eclipse.aether.transport.wagon.WagonTransporter.execute(WagonTransporter.java:435)
        at org.eclipse.aether.transport.wagon.WagonTransporter.get(WagonTransporter.java:412)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:457)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:364)
        ... 57 more

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:20 min
[INFO] Finished at: 2021-11-25T03:43:23Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:1.20.0:check (spotless-check) on project data-generator: Execution spotless-check of goal com.diffplug.spotless:spotless-maven-plugin:1.20.0:check failed: Unable to resolve dependencies: Failed to collect dependencies at com.google.googlejavaformat:google-java-format:jar:1.7 -> com.google.errorprone:javac-shaded:jar:9+181-r4173-1: Failed to read artifact descriptor for com.google.errorprone:javac-shaded:jar:9+181-r4173-1: Could not transfer artifact com.google.errorprone:javac-shaded:pom:9+181-r4173-1 from/to alimaven (https://maven.aliyun.com/nexus/content/groups/public/): Authorization failed for https://maven.aliyun.com/nexus/content/groups/public/com/google/errorprone/javac-shaded/9+181-r4173-1/javac-shaded-9+181-r4173-1.pom 403 Forbidden -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
The command '/bin/sh -c cd /opt/data-producer;     mvn clean install' returned a non-zero code: 1
ERROR: Service 'data-generator' failed to build : Build failed

解决方案：注释pom文件中相关插件

 <plugin>
                <groupId>com.diffplug.spotless</groupId>
                <artifactId>spotless-maven-plugin</artifactId>
                <version>${spotless-maven-plugin.version}</version>
                <configuration>
                    <java>
                        <googleJavaFormat>
                            <version>1.7</version>
                            <style>GOOGLE</style>
                        </googleJavaFormat>
                        <removeUnusedImports/>
                    </java>
                </configuration>
<!--
                <executions>
                    <execution>
                        <id>spotless-check</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>check</goal>
                        </goals>
                    </execution>
                </executions>
-->
            </plugin>

三、编译&启动

$ docker-compose build
$ docker-compose up -d

 

 


免责声明！

本站转载的文章为个人学习借鉴使用，本站对版权不负任何法律责任。如果侵犯了您的隐私权益，请联系本站邮箱yoyou2525@163.com删除。



猜您在找 mysqldump 使用及其注意事项 SqlBulkCopy使用注意事项 视图的使用及注意事项 In和Not In的使用注意事项和区别 dvaJs使用注意事项 NSTimer使用注意事项 mycat使用注意事项 MyBatis使用注意事项 使用SSZipArchive 注意事项 Bitmap使用注意事项
 
粤ICP备18138465号  © 2018-2025 CODEPRJ.COM