【功能模块】【操作步骤&问题现象】1、flink消费kafka数据输出到hbase中，写不进hbase中，报错信息如下：com.huawei.bigdata.flink.examples.outputs.HbaseSink2 (HbaseSink2.java:132) java.io.InterruptedIOException: Giving up trying to location region in meta: thread is interrupted.    at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:973)    at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:790)    at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:64)    at org.apache.hadoop.hbase.client.RegionLocator.getRegionLocation(RegionLocator.java:58)    at org.apache.hadoop.hbase.client.RegionLocator.getRegionLocation(RegionLocator.java:47)    at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:223)    at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)    at org.apache.hadoop.hbase.client.HTable.put(HTable.java:540)    at com.huawei.bigdata.flink.examples.outputs.HbaseSink2.invoke(HbaseSink2.java:128)    at com.huawei.bigdata.flink.examples.outputs.HbaseSink2.invoke(HbaseSink2.java:29)    at org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink.java:56)    at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:641)    at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:616)    at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:596)    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:734)    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:712)    at org.apache.flink.streaming.api.operators.StreamSourceContexts$NonTimestampContext.collect(StreamSourceContexts.java:104)    at org.apache.flink.streaming.api.operators.StreamSourceContexts$NonTimestampContext.collectWithTimestamp(StreamSourceContexts.java:111)    at org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher.emitRecordWithTimestamp(AbstractFetcher.java:398)    at org.apache.flink.streaming.connectors.kafka.internal.KafkaFetcher.emitRecord(KafkaFetcher.java:185)    at org.apache.flink.streaming.connectors.kafka.internal.KafkaFetcher.runFetchLoop(KafkaFetcher.java:150)    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:715)    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:100)    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:63)    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:196)2、【截图信息】【日志信息】（可选，上传日志内容或者附件）

【功能模块】【操作步骤&问题现象】1、我在原有flink连接kafka的example中写了一个sinkHbase的代码(说明：运行原来flink连接kafka样例代码没有任何问题)，之后进行测试，我将所需要的jar包都上传到flink/lib下面了，然后运行代码报：Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration，具体报错信息如下所示：2、【截图信息】【日志信息】（可选，上传日志内容或者附件）

【功能模块】【DLI】【Flink作业】【问题】 问题1：    在LiveData向Topic “T_IO_PARKING_FREE_SPACE” 发送消息，此topic作为“dli_vehicle_all”作业的其中一个数据源，发送消息内容如下：    Flink作业数据源：    Flink作业目标表：        到最后，到DAYU中查询这张目标表，确实新增了一条记录，但是所有字段值都为空。    问题：MQS到Flink作业的数据源 需要配置消息与字段的映射关系吗？否则怎么解释这个现象：发送消息后，确实在目标表实时新增了记录，但值都为空。问题2： DLI中有能看到完整SQL日志的地方吗？（能看到参数值的）【截图信息】【日志信息】（可选，上传日志内容或者附件）

运行flink任务时，日志显示 内部连接失败，如下图所示：麻烦帮忙看下是什么原因导致的。

问题场景客户作业场景如下图所示，从DMS kafka通过DLI Flink将业务数据实时清洗存储到DWS。其中，DMS Kafka 目标Topic 6个分区，DLI Flink作业配置taskmanager数量为12，并发数为1。问题现象客户在DLI服务共有三个相同规格的队列，该作业在其中003号队列上运行正常，在001和002号队列上都存在严重的反压导致数据处理缓慢。作业列表显示如下图，可以看到Sink反压状态正常，Souce和Map反压状态为HIGH。问题分析根据反压情况分析，该作业的性能瓶颈在Sink，由于Sink处理数据缓慢导致上游反压严重。该作业所定义的Sink类型为DwsCsvSink，该Sink的工作原理如下图所示：Sink将结果数据分片写入到OBS，每一分片写入完成后，调用DWS insert select sql将obs路径下该分片数据load到dws。因此性能瓶颈出现在分片数据写入到OBS这一步。但问题来了，写同一个桶，为什么在不同队列上的表现不一致？为此，我们排查了各个队列的CPU、内存和网络带宽情况，结果显示负载都很低。这种情况下，只能继续分析FlinkUI和TaskManager日志。数据倾斜？然后我们在FlinkUI任务情况页面，看到如下情况：Map阶段的12个TaskManager并不是所有反压都很严重，而是只有一半是HIGH状态，难道有数据倾斜导致分配到不同TaskManager的数据不均匀？然后看Source subTask详情，发现有两个TaskManager读取的数据量是其他几个的几十倍，这说明源端Kafka分区流入的数据量不均匀。难道就是这么简单的问题？很不幸并不是，通过进一步分析源端数据我们发现Kafka 6个分区数据流入记录数相差并不大。这两个Task只是多消费了部分存量数据，接收数据增长的速度各TaskManager保持一致。时钟同步进一步分析TaskManager日志，我们发现单个分片数据写入OBS竟然耗费3min以上。这非常异常，要知道单个分片数据才500000条而已。进一步通过分析代码发现如下问题：在写OBS数据时，其中一个taskmanager写分片目录后获取该目录的最后修改时间，作为处理该分片的开始时间，该时间为OBS服务端的时间。后续其他taskmanager向该分片目录写数据时，会获取本地时间与分片开始时间对比，间隔大于所规定的转储周期才会写分片数据。如果集群节点NTP时间与OBS服务端不同步，本地时间晚于OBS服务端时间，则会造成写入OBS等待。后续排查集群节点，发现6个节点中一半时间同步有问题，这也和只有一半taskmanager反压严重的现象相对应。问题修复在集群节点上执行如下命令，强制时间同步。systemctl stop ntp ntpdate ntp.myhuaweicloud.com systemctl start ntp systemctl status ntpdateNTP同步后，作业反压很快消失，故障恢复。

【功能模块】【操作步骤&问题现象】1、运行样例代码FlinkKafkaJavaExample.jar代码，生产者可以生产数据，但是消费者不能消费数据，运行消费者命令会报错，如附件所示：2、【截图信息】【日志信息】（可选，上传日志内容或者附件）

flink实时计算中，如果部分数据格式转换异常，怎么处理

【功能模块】【操作步骤&问题现象】ambari部署大数据平台缺失Flink组件，源是hdp3.1,有没编译好的Flink安装包【截图信息】【日志信息】（可选，上传日志内容或者附件）

【功能模块】【操作步骤&问题现象】1、将flinkkafkaexample样例程序打包运行，报无法提交作业的错误。2、【截图信息】【日志信息】（可选，上传日志内容或者附件） The program finished with the following exception:org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: java.util.concurrent.ExecutionException: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:700)        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:219)        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:932)        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1005)        at java.security.AccessController.doPrivileged(Native Method)        at javax.security.auth.Subject.doAs(Subject.java:422)        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)        at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1005)Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.        at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:199)        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1772)        at org.apache.flink.streaming.api.environment.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:94)        at org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:63)        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1651)        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1633)        at com.huawei.bigdata.flink.examples.WriteIntoKafka.main(WriteIntoKafka.java:48)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:498)        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)        ... 11 moreCaused by: java.util.concurrent.ExecutionException: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.        at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1767)        ... 21 moreCaused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.        at org.apache.flink.client.program.rest.RestClusterClient.lambda$submitJob$7(RestClusterClient.java:359)        at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884)        at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866)        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)        at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)        at org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$8(FutureUtils.java:287)        at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)        at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)        at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)        at org.apache.flink.runtime.rest.RestClient$ClientHandler.channelInactive(RestClient.java:483)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)        at org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:81)        at org.apache.flink.shaded.netty4.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:278)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)        at org.apache.flink.shaded.netty4.io.netty.handler.stream.ChunkedWriteHandler.channelInactive(ChunkedWriteHandler.java:141)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)        at org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:81)        at org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageAggregator.channelInactive(MessageAggregator.java:438)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)        at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelInactive(CombinedChannelDuplexHandler.java:420)        at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:393)        at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:358)        at org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpClientCodec$Decoder.channelInactive(HttpClientCodec.java:282)        at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelInactive(CombinedChannelDuplexHandler.java:223)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:236)        at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1416)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:257)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:243)        at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:912)        at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:816)        at org.apache.flink.shaded.netty4.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)        at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:416)        at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:515)        at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)        at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)        at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.flink.runtime.concurrent.FutureUtils$RetryException: Could not complete the operation. Number of retries has been exhausted.        at org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$8(FutureUtils.java:284)        ... 41 moreCaused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.rest.ConnectionClosedException: Channel became inactive.        at java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:326)        at java.util.concurrent.CompletableFuture.completeRelay(CompletableFuture.java:338)        at java.util.concurrent.CompletableFuture.uniRelay(CompletableFuture.java:925)        at java.util.concurrent.CompletableFuture$UniRelay.tryFire(CompletableFuture.java:913)        ... 39 moreCaused by: org.apache.flink.runtime.rest.ConnectionClosedException: Channel became inactive.        ... 37 more

【功能模块】【操作步骤&问题现象】1、2、【截图信息】【日志信息】（可选，上传日志内容或者附件）报错信息如下： The program finished with the following exception:org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: org.apache.flink.client.program.ProgramInvocationException: Job failed (JobID: 9e3c78adac04823062e5328231935728)        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:700)        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:219)        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:932)        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1005)        at java.security.AccessController.doPrivileged(Native Method)        at javax.security.auth.Subject.doAs(Subject.java:422)        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)        at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1005)Caused by: java.util.concurrent.ExecutionException: org.apache.flink.client.program.ProgramInvocationException: Job failed (JobID: 9e3c78adac04823062e5328231935728)        at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)        at org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:83)        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1651)        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1633)        at com.huawei.bigdata.flink.examples.ReadFromKafka.main(ReadFromKafka.java:59)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:498)        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)        ... 11 moreCaused by: org.apache.flink.client.program.ProgramInvocationException: Job failed (JobID: 9e3c78adac04823062e5328231935728)        at org.apache.flink.client.deployment.ClusterClientJobClientAdapter.lambda$null$6(ClusterClientJobClientAdapter.java:112)        at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)        at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)        at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)        at org.apache.flink.client.program.rest.RestClusterClient.lambda$pollResourceAsync$21(RestClusterClient.java:565)        at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)        at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)        at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)        at org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$8(FutureUtils.java:291)        at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)        at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)        at java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:575)        at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:943)        at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)        at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.        at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:147)        at org.apache.flink.client.deployment.ClusterClientJobClientAdapter.lambda$null$6(ClusterClientJobClientAdapter.java:110)        ... 19 moreCaused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy        at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:110)        at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:76)        at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:192)        at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:186)        at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:180)        at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:484)        at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:380)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:498)        at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:279)        at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:194)        at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)        at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)        at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)        at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)        at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)        at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)        at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)        at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)        at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)        at akka.actor.Actor$class.aroundReceive(Actor.scala:517)        at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)        at akka.actor.ActorCell.invoke(ActorCell.scala:561)        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)        at akka.dispatch.Mailbox.run(Mailbox.scala:225)        at akka.dispatch.Mailbox.exec(Mailbox.scala:235)        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata

因公司需要，需要在华为FIC80环境下执行Flink作业，但是目前遇到了问题，过程如下：1、先将Flink 1.12.0 版本解压后配置conf/flink-conf.yml 的安全认证，如下：security.kerberos.login.use-ticket-cache: truesecurity.kerberos.login.keytab: /home/kgp/user.keytabsecurity.kerberos.login.principal: kgp@HADOOP.COMsecurity.kerberos.login.contexts: Client,KafkaClient环境变量中的hadoop_home 和 hadoop_classpath都配置了，flink是可以找到yarn的2、然后尝试执行官网的demo案例：先执行了kinit命令：kinit -kt user.keytab kgp然后提交作业./bin/flink run -t yarn-per-job --detached ./examples/streaming/TopSpeedWindowing.jar3、结果出现异常：此时yarn UI 界面没有此任务，按照提示执行yarn命令来查看日志命令：yarn logs -applicationId application_1616467846343_00314、执行后得到的日志为：所用到的票据皆是从FI管理界面上下载的，可以保证正确求助

【功能模块】消费kafka消息写hdfs，flink 1.10，FI8.0.2【操作步骤&问题现象】1、操作步骤source-sink在一个task，并行度240，FsStateBackend1.1 RollingPolicy：batchSize=1024000000rolloverInterval=6000001.2 CheckpointConfig：chkInterval=600000chkMinPause=30000chkTimeout=500000chkMaxConcurrentNum=1FsStateBackend1.3 数据量大小 1.7T/10mins1.4 分配的yarn队列资源，内存、CPU均未用完2、问题现象2.1 怎么排查checkpoint为什么慢2.2 checkpoint sync和async分别是做什么的【截图信息】【日志信息】（可选，上传日志内容或者附件）对应时间点没做checkpoint的subtask的tm的log日志已保存，因为一些关键信息就不贴在这了。希望有大神指导！！！

【功能模块】【fi  华为flink  sql】【XXX功能】flink sql 是怎么读和写Gauss高斯数据库的?【操作步骤&问题现象】1、我想迁移这部分功能到开源的flink 1.12上, 使得  开源的flink  sql也支持  读和写 Gauss高斯数据库 ?2、【截图信息】【日志信息】（可选，上传日志内容或者附件）

【功能模块】【flink sql】【 尝试 使用 huaweicloud-mrs-example-mrs-2.1】调flinksql???【操作步骤&问题现象】1、基于 huaweicloud-mrs-example-mrs-2.1 在 flink 1.7.2版本的 环境 运行 flink  sql报错了?? 2、 测试 代码 package com.huawei.flink.example.sqljoin; import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; import org.apache.flink.table.api.TableEnvironment; import org.apache.flink.table.api.java.StreamTableEnvironment; //flink run -c   com.huawei.flink.example.sqljoin.SqlSim   -m yarn-cluster    /srv/BigData/data1/task_script_dir/onedata_ec_analyse/flink/s5.jar public class SqlSim {     public static final String KAFKA_TABLE_SOURCE_DDL = "" +             "CREATE SOURCE STREAM stream_id (attr_name attr_type (',' attr_name attr_type)* )\n" +             "  WITH (\n" +             "    type = \"kafka\",\n" +             "    kafka_bootstrap_servers = \"\",\n" +             "    kafka_group_id = \"\",\n" +             "    kafka_topic = \"\",\n" +             "    encode = \"json\"\n" +             "  )\n" +             "  (TIMESTAMP BY timeindicator (',' timeindicator)?);timeindicator:PROCTIME '.' PROCTIME| ID '.' ROWTIME";     public static void main(String[] args) throws Exception{         System.err.println("--77777*--SqlSim--- ");         StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();         StreamTableEnvironment tEnv = TableEnvironment.getTableEnvironment(env); //        ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); //        BatchTableEnvironment tableEnv = BatchTableEnvironment.create(env);         System.err.println("--35 *--SqlSim--- ");         tEnv.sqlUpdate(KAFKA_TABLE_SOURCE_DDL);         System.err.println("--38  38*--SqlSim--- ");         env.execute();         System.err.println("--42 42*--SqlSim--- ");     } }测试官网的例子" CREATE SOURCE STREAM stream_id (attr_name attr_type (',' attr_name attr_type)* )   WITH (     type = "kafka",     kafka_bootstrap_servers = "",     kafka_group_id = "",     kafka_topic = "",     encode = "json"   )   (TIMESTAMP BY timeindicator (',' timeindicator)?);timeindicator:PROCTIME '.' PROCTIME| ID '.' ROWTIME【截图信息】【日志信息】（可选，上传日志内容或者附件）apache.flink.client.program.ProgramInvocationException: The main method caused an error.        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:546)        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)        at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:430)        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:814)        at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:288)        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1051)        at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1127)        at java.security.AccessController.doPrivileged(Native Method)        at javax.security.auth.Subject.doAs(Subject.java:422)        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)        at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1127)Caused by: org.apache.flink.table.api.SqlParserException: SQL parse failed. Encountered "CREATE" at line 1, column 1.Was expecting one of:    "SET" ...    "RESET" ...    "ALTER" ...    "WITH" ...    "+" ...    "-" ...    "NOT" ...    "EXISTS" ...    <UNSIGNED_INTEGER_LITERAL> ...    <DECIMAL_NUMERIC_LITERAL> ...    <APPROX_NUMERIC_LITERAL> ...    <BINARY_STRING_LITERAL> ...    <PREFIXED_STRING_LITERAL> ...    <QUOTED_STRING> ...    <UNICODE_STRING_LITERAL> ...    "TRUE" ...    "FALSE" ...    "UNKNOWN" ...    "NULL" ...    <LBRACE_D> ...    <LBRACE_T> ...    <LBRACE_TS> ...    "DATE" ...    "TIME" ...

【功能模块】bin/flink run --class com.huawei.flink.example.sqljoin.WriteIntoKafka4SQLJoin /opt/Flink_test/flink-examples-1.0.jar --topic topic-test --bootstrap.servers xxx.xxx.xxx.xxx:21005运行测试 例子   com.huawei.flink.example.sqljoin.WriteIntoKafka4SQLJoin运行不通过  【操作步骤&问题现象】1、2、【截图信息】【日志信息】（可选，上传日志内容或者附件）--sasl.kerberos.service.name kafka --ssl.truststore.location /home/truststore.jks --ssl.truststore.password huawei******************************************************************************************<topic> is the kafka topic name<bootstrap.servers> is the ip:port list of brokers******************************************************************************************java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Callback        at com.huawei.flink.example.sqljoin.WriteIntoKafka4SQLJoin.main(WriteIntoKafka4SQLJoin.java:53)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:498)        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)        at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:430)        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:814)        at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:288)        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1051)        at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1127)        at java.security.AccessController.doPrivileged(Native Method)        at javax.security.auth.Subject.doAs(Subject.java:422)        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)        at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1127)Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.producer.Callback        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)        ... 18 more