需求：       想后台方式批量查找FusionInsight HD的管理平台Manager 上的role 角色，都授权了哪些服务，哪些权限。例， 查询A 角色授权了哪些服务和权限，A权限拥有哪些组件权限，Hive组件，有哪些库权限，哪些表/视图权限；HDFS组件，哪些目录有读写权限等。

mrs 提交任务到yarn报错   Could not load service provider for table factories

能否使用flink-sql-connector-oceanbase-cdc 进行连接，请问mrs flink是否支持这么做，如果支持，我该把jar放入那个位置才能生效

flinkSQL client中select 还是报错的，请帮忙指点下，哪里有问题？谢谢org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$SessionClosedRequireAuthException: KeeperErrorCode = Session closed because client failed to authenticate for /flink_base/flink或者org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /flink_base/flinkzookeeper已经启动，192.168.0.82:24002 ，而且zookeeper中的ACL权限已经设置，但是在设置配额失败[zk: 192.168.0.82:24002(CONNECTED) 5] setquota -n 1000000 /flink_base/flink Insufficient permission : /flink_base/flink tail -f /home/dmp/app/ficlient/Flink/flink/log/flink-root-sql-client-192-168-0-85.log  中的日志如下flink-conf.yaml中的全部配置如下akka.ask.timeout: 120 s akka.client-socket-worker-pool.pool-size-factor: 1.0 akka.client-socket-worker-pool.pool-size-max: 2 akka.client-socket-worker-pool.pool-size-min: 1 akka.framesize: 10485760b akka.log.lifecycle.events: false akka.lookup.timeout: 30 s akka.server-socket-worker-pool.pool-size-factor: 1.0 akka.server-socket-worker-pool.pool-size-max: 2 akka.server-socket-worker-pool.pool-size-min: 1 akka.ssl.enabled: true akka.startup-timeout: 10 s akka.tcp.timeout: 60 s akka.throughput: 15 blob.fetch.backlog: 1000 blob.fetch.num-concurrent: 50 blob.fetch.retries: 50 blob.server.port: 32456-32520 blob.service.ssl.enabled: true classloader.check-leaked-classloader: false classloader.resolve-order: child-first client.rpc.port: 32651-32720 client.timeout: 120 s compiler.delimited-informat.max-line-samples: 10 compiler.delimited-informat.max-sample-len: 2097152 compiler.delimited-informat.min-line-samples: 2 env.hadoop.conf.dir: /home/dmp/app/ficlient/Flink/flink/conf env.java.opts.client: -Djava.io.tmpdir=/home/dmp/app/ficlient/Flink/tmp env.java.opts.jobmanager: -Djava.security.krb5.conf=/opt/huawei/Bigdata/common/runtime/krb5.conf -Djava.io.tmpdir=${PWD}/tmp -Des.security.indication=true env.java.opts.taskmanager: -Djava.security.krb5.conf=/opt/huawei/Bigdata/common/runtime/krb5.conf -Djava.io.tmpdir=${PWD}/tmp -Des.security.indication=true env.java.opts: -Xloggc:<LOG_DIR>/gc.log -XX:+PrintGCDetails -XX:-OmitStackTraceInFastThrow -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=20 -XX:GCLogFileSize=20M -Djdk.tls.ephemeralDHKeySize=3072 -Djava.library.path=${HADOOP_COMMON_HOME}/lib/native -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false -Dbeetle.application.home.path=/opt/huawei/Bigdata/common/runtime/security/config -Dwcc.configuration.path=/opt/huawei/Bigdata/common/runtime/security/config -Dscc.configuration.path=/opt/huawei/Bigdata/common/runtime/securityforscc/config -Dscc.bigdata.common=/opt/huawei/Bigdata/common/runtime env.yarn.conf.dir: /home/dmp/app/ficlient/Flink/flink/conf flink.security.enable: true flinkserver.alarm.cert.skip: true flinkserver.host.ip: fs.output.always-create-directory: false fs.overwrite-files: false heartbeat.interval: 10000 heartbeat.timeout: 120000 high-availability.job.delay: 10 s high-availability.storageDir: hdfs://hacluster/flink/recovery high-availability.zookeeper.client.acl: creator high-availability.zookeeper.client.connection-timeout: 90000 high-availability.zookeeper.client.max-retry-attempts: 5 high-availability.zookeeper.client.retry-wait: 5000 high-availability.zookeeper.client.session-timeout: 90000 high-availability.zookeeper.client.tolerate-suspended-connections: true high-availability.zookeeper.path.root: /flink high-availability.zookeeper.path.under.quota: /flink_base high-availability.zookeeper.quorum: 192.168.0.82:24002,192.168.0.81:24002,192.168.0.80:24002 high-availability.zookeeper.quota.enabled: true high-availability: zookeeper job.alarm.enable: true jobmanager.heap.size: 1024mb jobmanager.web.403-redirect-url: https://192.168.0.82:28443/web/pages/error/403.html jobmanager.web.404-redirect-url: https://192.168.0.82:28443/web/pages/error/404.html jobmanager.web.415-redirect-url: https://192.168.0.82:28443/web/pages/error/415.html jobmanager.web.500-redirect-url: https://192.168.0.82:28443/web/pages/error/500.html jobmanager.web.access-control-allow-origin: * jobmanager.web.accesslog.enable: true jobmanager.web.allow-access-address: * jobmanager.web.backpressure.cleanup-interval: 600000 jobmanager.web.backpressure.delay-between-samples: 50 jobmanager.web.backpressure.num-samples: 100 jobmanager.web.backpressure.refresh-interval: 60000 jobmanager.web.cache-directive: no-store jobmanager.web.checkpoints.disable: false jobmanager.web.checkpoints.history: 10 jobmanager.web.expires-time: 0 jobmanager.web.history: 5 jobmanager.web.logout-timer: 600000 jobmanager.web.pragma-value: no-cache jobmanager.web.refresh-interval: 3000 jobmanager.web.ssl.enabled: false jobmanager.web.x-frame-options: DENY library-cache-manager.cleanup.interval: 3600 metrics.internal.query-service.port: 28844-28943 metrics.reporter.alarm.factory.class: com.huawei.mrs.flink.alarm.FlinkAlarmReporterFactory metrics.reporter.alarm.interval: 30 s metrics.reporter.alarm.job.alarm.checkpoint.consecutive.failures.num: 5 metrics.reporter.alarm.job.alarm.failure.restart.rate: 80 metrics.reporter.alarm.job.alarm.task.backpressure.duration: 180 s metrics.reporter: alarm nettyconnector.message.delimiter: $_ nettyconnector.registerserver.topic.storage: /flink/nettyconnector nettyconnector.sinkserver.port.range: 28444-28843 nettyconnector.ssl.enabled: false parallelism.default: 1 query.client.network-threads: 0 query.proxy.network-threads: 0 query.proxy.ports: 32541-32560 query.proxy.query-threads: 0 query.server.network-threads: 0 query.server.ports: 32521-32540 query.server.query-threads: 0 resourcemanager.taskmanager-timeout: 300000 rest.await-leader-timeout: 30000 rest.bind-port: 32261-32325 rest.client.max-content-length: 104857600 rest.connection-timeout: 15000 rest.idleness-timeout: 300000 rest.retry.delay: 3000 rest.retry.max-attempts: 20 rest.server.max-content-length: 104857600 rest.server.numThreads: 4 restart-strategy.failure-rate.delay: 10 s restart-strategy.failure-rate.failure-rate-interval: 60 s restart-strategy.failure-rate.max-failures-per-interval: 1 restart-strategy.fixed-delay.attempts: 3 restart-strategy.fixed-delay.delay: 10 s restart-strategy: none security.cookie: 9477298cd52a3e409ed0bc570bdc795179fcc7c301a1225e22f47fe0a3db47c2 security.enable: true security.kerberos.login.contexts: Client,KafkaClient security.kerberos.login.keytab: security.kerberos.login.principal: security.kerberos.login.use-ticket-cache: true security.networkwide.listen.restrict: true security.ssl.algorithms: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 security.ssl.enabled: false security.ssl.encrypt.enabled: false security.ssl.key-password: Bapuser@9000 security.ssl.keystore-password: Bapuser@9000 security.ssl.keystore: ssl/flink.keystore security.ssl.protocol: TLSv1.2 security.ssl.rest.enabled: false security.ssl.truststore-password: Bapuser@9000 security.ssl.truststore: ssl/flink.truststore security.ssl.verify-hostname: false slot.idle.timeout: 50000 slot.request.timeout: 300000 state.backend.fs.checkpointdir: hdfs://hacluster/flink/checkpoints state.backend.fs.memory-threshold: 20kb state.backend.incremental: true state.backend: rocksdb state.savepoints.dir: hdfs://hacluster/flink/savepoint task.cancellation.interval: 30000 task.cancellation.timeout: 180000 taskmanager.data.port: 32391-32455 taskmanager.data.ssl.enabled: false taskmanager.debug.memory.logIntervalMs: 0 taskmanager.debug.memory.startLogThread: false taskmanager.heap.size: 1024mb taskmanager.initial-registration-pause: 500 ms taskmanager.max-registration-pause: 30 s taskmanager.maxRegistrationDuration: 5 min taskmanager.memory.fraction: 0.7 taskmanager.memory.off-heap: false taskmanager.memory.preallocate: false taskmanager.memory.segment-size: 32768 taskmanager.network.detailed-metrics: false taskmanager.network.memory.buffers-per-channel: 2 taskmanager.network.memory.floating-buffers-per-gate: 8 taskmanager.network.memory.fraction: 0.1 taskmanager.network.memory.max: 1gb taskmanager.network.memory.min: 64mb taskmanager.network.netty.client.connectTimeoutSec: 300 taskmanager.network.netty.client.numThreads: -1 taskmanager.network.netty.num-arenas: -1 taskmanager.network.netty.sendReceiveBufferSize: 4096 taskmanager.network.netty.server.backlog: 0 taskmanager.network.netty.server.numThreads: -1 taskmanager.network.netty.transport: nio taskmanager.network.numberOfBuffers: 2048 taskmanager.network.request-backoff.initial: 100 taskmanager.network.request-backoff.max: 10000 taskmanager.numberOfTaskSlots: 1 taskmanager.refused-registration-pause: 10 s taskmanager.registration.timeout: 5 min taskmanager.rpc.port: 32326-32390 taskmanager.runtime.hashjoin-bloom-filters: false taskmanager.runtime.max-fan: 128 taskmanager.runtime.sort-spilling-threshold: 0.8 use.path.filesystem: true use.smarterleaderlatch: true web.submit.enable: false web.timeout: 10000 yarn.application-attempt-failures-validity-interval: 600000 yarn.application-attempts: 5 yarn.application-master.port: 32586-32650 yarn.heap-cutoff-min: 384 yarn.heap-cutoff-ratio: 0.25 yarn.heartbeat-delay: 5 yarn.heartbeat.container-request-interval: 500 yarn.maximum-failed-containers: 5 yarn.per-job-cluster.include-user-jar: ORDER zk.ssl.enabled: false zookeeper.clientPort.quorum: 192.168.0.82:24002,192.168.0.81:24002,192.168.0.80:24002 zookeeper.root.acl: OPEN zookeeper.sasl.disable: false zookeeper.sasl.login-context-name: Client zookeeper.sasl.service-name: zookeeper zookeeper.secureClientPort.quorum: 192.168.0.82:24002,192.168.0.81:24002,192.168.0.80:24002 

flinkSQL client中select 还是报错的，请帮忙指点下，哪里有问题？谢谢org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$SessionClosedRequireAuthException: KeeperErrorCode = Session closed because client failed to authenticate for /flink_base/flink或者org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /flink_base/flinkzookeeper已经启动，192.168.0.82:24002 ，而且zookeeper中的ACL权限已经设置，但是在设置配额失败[zk: 192.168.0.82:24002(CONNECTED) 2] create /flink_base/flink_base Created /flink_base/flink_base [zk: 192.168.0.82:24002(CONNECTED) 3] ls /flink_base/ Path must not end with / character [zk: 192.168.0.82:24002(CONNECTED) 4] ls /flink_base [flink, flink_base] [zk: 192.168.0.82:24002(CONNECTED) 5] [zk: 192.168.0.82:24002(CONNECTED) 5] [zk: 192.168.0.82:24002(CONNECTED) 5] [zk: 192.168.0.82:24002(CONNECTED) 5] setquota -n 1000000 /flink_base/flink Insufficient permission : /flink_base/flink [zk: 192.168.0.82:24002(CONNECTED) 6] getAcl /flink_base/flink 'world,'anyone : cdrwa [zk: 192.168.0.82:24002(CONNECTED) 7] setAcl /flink_base/flink world:anyone:rwcda [zk: 192.168.0.82:24002(CONNECTED) 8] setquota -n 1000000 /flink_base/flink Insufficient permission : /flink_base/flink [zk: 192.168.0.82:24002(CONNECTED) 9] getAcl /flink_base/ Path must not end with / character [zk: 192.168.0.82:24002(CONNECTED) 10] getAcl /flink_base 'world,'anyone : cdrwa [zk: 192.168.0.82:24002(CONNECTED) 11] getAcl /flink_base/flink 'world,'anyone : cdrwa [zk: 192.168.0.82:24002(CONNECTED) 12] ls /zookeeper/quota [beeline, elasticsearch, flink_base, graphbase, hadoop, hadoop-adapter-data, hadoop-flag, hadoop-ha, hbase, hdfs-acl-log, hive, hiveserver2, kafka, loader, mr-ha, rmstore, sparkthriftserver, sparkthriftserver2x, sparkthriftserver2x_sparkInternal_HAMode, yarn-leader-election] [zk: 192.168.0.82:24002(CONNECTED) 13] ls /zookeeper/quota/flink_base [zookeeper_limits, zookeeper_stats] [zk: 192.168.0.82:24002(CONNECTED) 5] setquota -n 1000000 /flink_base/flink Insufficient permission : /flink_base/flink tail -f /home/dmp/app/ficlient/Flink/flink/log/flink-root-sql-client-192-168-0-85.log  中的日志如下flink-conf.yaml中的全部配置如下akka.ask.timeout: 120 s akka.client-socket-worker-pool.pool-size-factor: 1.0 akka.client-socket-worker-pool.pool-size-max: 2 akka.client-socket-worker-pool.pool-size-min: 1 akka.framesize: 10485760b akka.log.lifecycle.events: false akka.lookup.timeout: 30 s akka.server-socket-worker-pool.pool-size-factor: 1.0 akka.server-socket-worker-pool.pool-size-max: 2 akka.server-socket-worker-pool.pool-size-min: 1 akka.ssl.enabled: true akka.startup-timeout: 10 s akka.tcp.timeout: 60 s akka.throughput: 15 blob.fetch.backlog: 1000 blob.fetch.num-concurrent: 50 blob.fetch.retries: 50 blob.server.port: 32456-32520 blob.service.ssl.enabled: true classloader.check-leaked-classloader: false classloader.resolve-order: child-first client.rpc.port: 32651-32720 client.timeout: 120 s compiler.delimited-informat.max-line-samples: 10 compiler.delimited-informat.max-sample-len: 2097152 compiler.delimited-informat.min-line-samples: 2 env.hadoop.conf.dir: /home/dmp/app/ficlient/Flink/flink/conf env.java.opts.client: -Djava.io.tmpdir=/home/dmp/app/ficlient/Flink/tmp env.java.opts.jobmanager: -Djava.security.krb5.conf=/opt/huawei/Bigdata/common/runtime/krb5.conf -Djava.io.tmpdir=${PWD}/tmp -Des.security.indication=true env.java.opts.taskmanager: -Djava.security.krb5.conf=/opt/huawei/Bigdata/common/runtime/krb5.conf -Djava.io.tmpdir=${PWD}/tmp -Des.security.indication=true env.java.opts: -Xloggc:<LOG_DIR>/gc.log -XX:+PrintGCDetails -XX:-OmitStackTraceInFastThrow -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=20 -XX:GCLogFileSize=20M -Djdk.tls.ephemeralDHKeySize=3072 -Djava.library.path=${HADOOP_COMMON_HOME}/lib/native -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false -Dbeetle.application.home.path=/opt/huawei/Bigdata/common/runtime/security/config -Dwcc.configuration.path=/opt/huawei/Bigdata/common/runtime/security/config -Dscc.configuration.path=/opt/huawei/Bigdata/common/runtime/securityforscc/config -Dscc.bigdata.common=/opt/huawei/Bigdata/common/runtime env.yarn.conf.dir: /home/dmp/app/ficlient/Flink/flink/conf flink.security.enable: true flinkserver.alarm.cert.skip: true flinkserver.host.ip: fs.output.always-create-directory: false fs.overwrite-files: false heartbeat.interval: 10000 heartbeat.timeout: 120000 high-availability.job.delay: 10 s high-availability.storageDir: hdfs://hacluster/flink/recovery high-availability.zookeeper.client.acl: creator high-availability.zookeeper.client.connection-timeout: 90000 high-availability.zookeeper.client.max-retry-attempts: 5 high-availability.zookeeper.client.retry-wait: 5000 high-availability.zookeeper.client.session-timeout: 90000 high-availability.zookeeper.client.tolerate-suspended-connections: true high-availability.zookeeper.path.root: /flink high-availability.zookeeper.path.under.quota: /flink_base high-availability.zookeeper.quorum: 192.168.0.82:24002,192.168.0.81:24002,192.168.0.80:24002 high-availability.zookeeper.quota.enabled: true high-availability: zookeeper job.alarm.enable: true jobmanager.heap.size: 1024mb jobmanager.web.403-redirect-url: https://192.168.0.82:28443/web/pages/error/403.html jobmanager.web.404-redirect-url: https://192.168.0.82:28443/web/pages/error/404.html jobmanager.web.415-redirect-url: https://192.168.0.82:28443/web/pages/error/415.html jobmanager.web.500-redirect-url: https://192.168.0.82:28443/web/pages/error/500.html jobmanager.web.access-control-allow-origin: * jobmanager.web.accesslog.enable: true jobmanager.web.allow-access-address: * jobmanager.web.backpressure.cleanup-interval: 600000 jobmanager.web.backpressure.delay-between-samples: 50 jobmanager.web.backpressure.num-samples: 100 jobmanager.web.backpressure.refresh-interval: 60000 jobmanager.web.cache-directive: no-store jobmanager.web.checkpoints.disable: false jobmanager.web.checkpoints.history: 10 jobmanager.web.expires-time: 0 jobmanager.web.history: 5 jobmanager.web.logout-timer: 600000 jobmanager.web.pragma-value: no-cache jobmanager.web.refresh-interval: 3000 jobmanager.web.ssl.enabled: false jobmanager.web.x-frame-options: DENY library-cache-manager.cleanup.interval: 3600 metrics.internal.query-service.port: 28844-28943 metrics.reporter.alarm.factory.class: com.huawei.mrs.flink.alarm.FlinkAlarmReporterFactory metrics.reporter.alarm.interval: 30 s metrics.reporter.alarm.job.alarm.checkpoint.consecutive.failures.num: 5 metrics.reporter.alarm.job.alarm.failure.restart.rate: 80 metrics.reporter.alarm.job.alarm.task.backpressure.duration: 180 s metrics.reporter: alarm nettyconnector.message.delimiter: $_ nettyconnector.registerserver.topic.storage: /flink/nettyconnector nettyconnector.sinkserver.port.range: 28444-28843 nettyconnector.ssl.enabled: false parallelism.default: 1 query.client.network-threads: 0 query.proxy.network-threads: 0 query.proxy.ports: 32541-32560 query.proxy.query-threads: 0 query.server.network-threads: 0 query.server.ports: 32521-32540 query.server.query-threads: 0 resourcemanager.taskmanager-timeout: 300000 rest.await-leader-timeout: 30000 rest.bind-port: 32261-32325 rest.client.max-content-length: 104857600 rest.connection-timeout: 15000 rest.idleness-timeout: 300000 rest.retry.delay: 3000 rest.retry.max-attempts: 20 rest.server.max-content-length: 104857600 rest.server.numThreads: 4 restart-strategy.failure-rate.delay: 10 s restart-strategy.failure-rate.failure-rate-interval: 60 s restart-strategy.failure-rate.max-failures-per-interval: 1 restart-strategy.fixed-delay.attempts: 3 restart-strategy.fixed-delay.delay: 10 s restart-strategy: none security.cookie: 9477298cd52a3e409ed0bc570bdc795179fcc7c301a1225e22f47fe0a3db47c2 security.enable: true security.kerberos.login.contexts: Client,KafkaClient security.kerberos.login.keytab: security.kerberos.login.principal: security.kerberos.login.use-ticket-cache: true security.networkwide.listen.restrict: true security.ssl.algorithms: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 security.ssl.enabled: false security.ssl.encrypt.enabled: false security.ssl.key-password: Bapuser@9000 security.ssl.keystore-password: Bapuser@9000 security.ssl.keystore: ssl/flink.keystore security.ssl.protocol: TLSv1.2 security.ssl.rest.enabled: false security.ssl.truststore-password: Bapuser@9000 security.ssl.truststore: ssl/flink.truststore security.ssl.verify-hostname: false slot.idle.timeout: 50000 slot.request.timeout: 300000 state.backend.fs.checkpointdir: hdfs://hacluster/flink/checkpoints state.backend.fs.memory-threshold: 20kb state.backend.incremental: true state.backend: rocksdb state.savepoints.dir: hdfs://hacluster/flink/savepoint task.cancellation.interval: 30000 task.cancellation.timeout: 180000 taskmanager.data.port: 32391-32455 taskmanager.data.ssl.enabled: false taskmanager.debug.memory.logIntervalMs: 0 taskmanager.debug.memory.startLogThread: false taskmanager.heap.size: 1024mb taskmanager.initial-registration-pause: 500 ms taskmanager.max-registration-pause: 30 s taskmanager.maxRegistrationDuration: 5 min taskmanager.memory.fraction: 0.7 taskmanager.memory.off-heap: false taskmanager.memory.preallocate: false taskmanager.memory.segment-size: 32768 taskmanager.network.detailed-metrics: false taskmanager.network.memory.buffers-per-channel: 2 taskmanager.network.memory.floating-buffers-per-gate: 8 taskmanager.network.memory.fraction: 0.1 taskmanager.network.memory.max: 1gb taskmanager.network.memory.min: 64mb taskmanager.network.netty.client.connectTimeoutSec: 300 taskmanager.network.netty.client.numThreads: -1 taskmanager.network.netty.num-arenas: -1 taskmanager.network.netty.sendReceiveBufferSize: 4096 taskmanager.network.netty.server.backlog: 0 taskmanager.network.netty.server.numThreads: -1 taskmanager.network.netty.transport: nio taskmanager.network.numberOfBuffers: 2048 taskmanager.network.request-backoff.initial: 100 taskmanager.network.request-backoff.max: 10000 taskmanager.numberOfTaskSlots: 1 taskmanager.refused-registration-pause: 10 s taskmanager.registration.timeout: 5 min taskmanager.rpc.port: 32326-32390 taskmanager.runtime.hashjoin-bloom-filters: false taskmanager.runtime.max-fan: 128 taskmanager.runtime.sort-spilling-threshold: 0.8 use.path.filesystem: true use.smarterleaderlatch: true web.submit.enable: false web.timeout: 10000 yarn.application-attempt-failures-validity-interval: 600000 yarn.application-attempts: 5 yarn.application-master.port: 32586-32650 yarn.heap-cutoff-min: 384 yarn.heap-cutoff-ratio: 0.25 yarn.heartbeat-delay: 5 yarn.heartbeat.container-request-interval: 500 yarn.maximum-failed-containers: 5 yarn.per-job-cluster.include-user-jar: ORDER zk.ssl.enabled: false zookeeper.clientPort.quorum: 192.168.0.82:24002,192.168.0.81:24002,192.168.0.80:24002 zookeeper.root.acl: OPEN zookeeper.sasl.disable: false zookeeper.sasl.login-context-name: Client zookeeper.sasl.service-name: zookeeper zookeeper.secureClientPort.quorum: 192.168.0.82:24002,192.168.0.81:24002,192.168.0.80:24002 

MapReduce服务（MRS）MRS产品文档https://support.huaweicloud.com/mrs/index.htmlMRS视频介绍https://support.huaweicloud.com/mrs_video/index.htmlMRS产品体验https://lab.huaweicloud.com/experiment-list?ticket=ST-82337421-kKgD75jMOxdSAkr1TN3LuLVT-ssoMRS入门实践https://support.huaweicloud.com/qs-mrs/mrs_09_0027.html开发者论坛https://bbs.huaweicloud.com/forum/forum-612-1.html华为云大数据产品职业认证HCIA/HCIP 免费视频学习https://edu.huaweicloud.com/training华为云大数据开发者认证HCCDP 免费视频学习https://edu.huaweicloud.com/certificationindex

版本现象使用开源Hive相关依赖（hive-exec、hive-metastore等）无法连接MRS Hive MetaStore 问题1：MRS Hive MetaStore 是否支持外部访问  问题2：如果支持外部访问需要哪些必要操作？（必须强制要MRS Hive相同的依赖？或者其他注意事项？）  其他：我看到一个类似的问题，https://bbs.huaweicloud.com/forum/thread-99927-1-1.html，是属于同一类问题吗？ 

mrs320版本，hudi0.11。场景是静态表离线跑批。使用spark-sql每天向hudi cow表里insert select 0条数据，timeline文件无法archive，导致小文件越来越多希望大佬给个解决方案spark-sql复现步骤如下：--创建源表CREATE TABLE emp_test ( empno int, ename string, job string, mgr int, hiredate string, sal int, comm int, deptno int, tx_date string)using hudioptions( type='cow' ,primaryKey='empno' ,payloadclass='org.apache.hudi.common.model.OverwriteNonDefaultWithLatestAvroPayLoad' ,preCombineField='tx_date' ,hoodie.cleaner.commits.retained='1' ,hoodie.keep.min.commits='2' ,hoodie.keep.max.commits='3' ,hoodie.index.type='SIMPLE');insert into emp_test values(7369,'SMITH','CLERK',7902,'1980-12-17',800,100,20,'2022-11-17'),(7499,'ALLEN','SALESMAN',7698,'1981-02-20',1600,300,30,'2022-11-17'),(5233,'ANDY','DEVELOPER',9192,'1996-05-30',5000,3000,10,'2022-11-13');--创建2表create table emp_test2 using hudioptions ( type='cow' ,primaryKey='empno' ,payloadclass='org.apache.hudi.common.model.OverwriteNonDefaultWithLatestAvroPayLoad' ,preCombineField='tx_date' ,hoodie.cleaner.commits.retained='1' ,hoodie.keep.min.commits='2' ,hoodie.keep.max.commits='3' ,hoodie.index.type='SIMPLE' ) as select * from emp_test where 1<>1;--初始化2表insert into emp_test2 select * from emp_test;--2表每天无新增数据insert into emp_test2 select * from emp_test limit 0;insert into emp_test2 select * from emp_test limit 0;insert into emp_test2 select * from emp_test limit 0;insert into emp_test2 select * from emp_test limit 0;insert into emp_test2 select * from emp_test limit 0;insert into emp_test2 select * from emp_test limit 0;insert into emp_test2 select * from emp_test limit 0;insert into emp_test2 select * from emp_test limit 0;insert into emp_test2 select * from emp_test limit 0;观察hdfs2表/.hoodie下timeline instant文件一直新增，不归档/archived(无归档文件)

一、MRS流式集群购买按照华为云用户手册（cid:link_0）购买。二、Kafka客户端安装FusionInsight Manager登陆界面下载集群客户端客户端生成成功安装客户端至集群内其他节点解压并校验软件包安装客户端完成三、Kafka消息传递登陆master1节点，并配置环境变量开启IAM用户同步查看并记录一个ZooKeeper角色实例的IP地址创建名为“77”的Kafka topic查看并记录Kafka角色实例中任意一个的IP在topic test中收发消息，内容为姓名全拼（liuchengjie）和学号（201250125）四、Python使用Kafka结果如下，内容为姓名全拼（liuchengjie）和学号（201250125）

在配置MRS与开源大数据集群互信时，分别在两边创建好用户，配置对端的kerbserver和hdfs，配置mrs界面上的互信并重启后，在两个集群分别进行互信访问时都会报同样的错：[root@bdtl-vm-1652 ~]# hdfs dfs -ls /2023-11-09 09:58:10,209 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 60 seconds before. Last Login=16994950888392023-11-09 09:58:15,102 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 60 seconds before. Last Login=16994950888392023-11-09 09:58:15,719 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 60 seconds before. Last Login=16994950888392023-11-09 09:58:17,278 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 60 seconds before. Last Login=16994950888392023-11-09 09:58:17,728 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 60 seconds before. Last Login=16994950888392023-11-09 09:58:21,389 WARN ipc.Client: Couldn't setup connection for hdpuser@HDP.COM to xx.xx.xx.xx:25000javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Generic error (description in e-text) (60) - PROCESS_TGS)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:410) at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:627) at org.apache.hadoop.ipc.Client$Connection.access$2400(Client.java:418) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:855) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:851) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1890) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:851) at org.apache.hadoop.ipc.Client$Connection.access$3800(Client.java:418) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1694) at org.apache.hadoop.ipc.Client.call(Client.java:1519) at org.apache.hadoop.ipc.Client.call(Client.java:1472) at org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:245) at org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:131) at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:1008) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:435) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:170) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:162) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:366) at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1892) at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1805) at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1802) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1817) at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:115) at org.apache.hadoop.fs.Globber.doGlob(Globber.java:367) at org.apache.hadoop.fs.Globber.glob(Globber.java:205) at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:2196) at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:345) at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:252) at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:235) at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:107) at org.apache.hadoop.fs.shell.Command.run(Command.java:179) at org.apache.hadoop.fs.FsShell.run(FsShell.java:343) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:95) at org.apache.hadoop.fs.FsShell.main(FsShell.java:410)Caused by: GSSException: No valid credentials provided (Mechanism level: Generic error (description in e-text) (60) - PROCESS_TGS) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:772) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 45 moreCaused by: KrbException: Generic error (description in e-text) (60) - PROCESS_TGS at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:226) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:237) at sun.security.krb5.internal.CredentialsUtil.serviceCredsSingle(CredentialsUtil.java:477) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:340) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:314) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:169) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:490) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:695) ... 48 moreCaused by: KrbException: Identifier doesn't match expected value (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65) at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ... 56 morels: DestHost:destPort xx.xx.xx.xx:25000 , LocalHost:localPort xx.xx.xx.xx:0. Failed on local exception: java.io.IOException: Couldn't setup connection for hdpuser@HDP.COM to xx.xx.xx.xx:25000请问这个是哪里缺少配置吗？

是多花费时间与精力去学习、考取HCIE-Cloud，bigdata类似的认证还是早点找实习工作通过实际项目积累经验

什么是DataArts Insight(一)智能数据洞察（DataArts Insight）是华为云新一代BI服务，提供可视、实时、易用、安全的企业智能分析数据服务，以最自然高效的方式获取业务见解，支撑业务实时高效决策。适配云上云下多种数据源，提供丰富多样的可视化组件，采用拖拽式自由布局，轻松实现数据分析和报表搭建，快速定制专属数据大屏。产品架构(二)DataArts Insight的产品架构如图所示产品功能(三)01自助式分析DataArts Insight提供的智能图表可以帮助您直观、清晰地展示数据分析结果。DataArts Insight提供了多种图表样式，覆盖了表格、线图/面图、柱状图/条形图、指标图、圆盘图、散点图、气泡图等分析图表，满足您灵活多样的可视化分析需求。02数据大屏内置丰富的行业模板和素材内容，支持一键安装应用，快速搭建大屏。将可视化与叙事技术结合，支持多场景、多页面的故事性大屏。图表配置精细化程度再提升，支持动画效果，更有助于气氛渲染。数据指标、分析加工一键复用，加工效率高。03盘古 for BI将智能报表转化为智能工具，提供更加直观和高效的数据分析方式。通过机器学习和数据挖掘，自动发现数据中的关联与趋势，提供有效的洞察与建议。04数据接入支持多种数据源接入能力，包括DWS、ClickHouse、API、本地文件作为现代商业智能分析的数据源。支持公网连接、支持数据源的连通性测试。05数据加工支持在工作空间新建数据集，通过数据源导入、图形化和SQL形式创建数据集。数据集支持度量和维度的设置，支持新建分组维度，层次维度和计算字段，支持数据集字段隐藏。更多产品信息可进入产品主页查看：智能数据洞察 DataArts Insight

HBase介绍​ HBase是Hadoop Database的简称，是建立在Hadoop文件系统之上的分布式面向列的数据库，它具有高可靠、高性能、面向列和可伸缩的特性，提供快速随机访问海量数据能力。​ HBase采用Master/Slave架构，由HMaster节点、RegionServer节点、ZooKeeper集群组成，底层数据存储在HDFS上。​ 整体架构如图所示：HMaster主要负责：在HA模式下，包含主用Master和备用Master。主用Master：负责HBase中RegionServer的管理，包括表的增删改查；RegionServer的负载均衡，Region分布调整；Region分裂以及分裂后的Region分配；RegionServer失效后的Region迁移等。备用Master：当主用Master故障时，备用Master将取代主用Master对外提供服务。故障恢复后，原主用Master降为备用。RegionServer主要负责：存放和管理本地HRegion。RegionServer负责提供表数据读写等服务，是HBase的数据处理和计算单元，直接与Client交互。RegionServer一般与HDFS集群的DataNode部署在一起，实现数据的存储功能。读写HDFS，管理Table中的数据。ZooKeeper集群主要负责：存放整个 HBase集群的元数据以及集群的状态信息。实现HMaster主从节点的Failover。HDFS集群主要负责：HDFS为HBase提供高可靠的文件存储服务，HBase的数据全部存储在HDFS中。结构说明：Store一个Region由一个或多个Store组成，每个Store对应图中的一个Column Family。MemStore一个Store包含一个MemStore，MemStore缓存客户端向Region插入的数据，当RegionServer中的MemStore大小达到配置的容量上限时，RegionServer会将MemStore中的数据“flush”到HDFS中。StoreFileMemStore的数据flush到HDFS后成为StoreFile，随着数据的插入，一个Store会产生多个StoreFile，当StoreFile的个数达到配置的阈值时，RegionServer会将多个StoreFile合并为一个大的StoreFile。HFileHFile定义了StoreFile在文件系统中的存储格式，它是当前HBase系统中StoreFile的具体实现。HLog（WAL）HLog日志保证了当RegionServer故障的情况下用户写入的数据不丢失，RegionServer的多个Region共享一个相同的HLog。HBase提供两种API来写入数据。Put：数据直接发送给RegionServer。BulkLoad：直接将HFile加载到表存储路径。HBase 冷热分离诉求HBase是Hadoop Database的简称，是建立在Hadoop文件系统之上的分布式面向列的数据库，它具有高可靠、高性能、面向列和可伸缩的特性，提供快速随机访问海量数据能力。在海量大数据场景下，表中的部分业务数据随着时间的推移仅作为归档数据或者访问频率很低，同时这部分历史数据体量非常大，比如订单数据或者监控数据，如果降低这部分数据的存储成本将会极大的节省企业的成本。冷热分离功能支持将冷热数据存储在不同的介质上，冷数据的存储类型为普通IO存储，热数据的存储类型为超高IO存储。普通IO存储的价格仅为超高IO存储的30%，大大降低了存储成本。HBase 冷热分离介绍HBase支持对同一张表的数据进行冷热分离存储。用户在表上配置数据冷热时间分界点后，HBase会依赖用户写入数据的时间戳（毫秒）和时间分界点来判断数据的冷热。数据开始存储在热存储上，随着时间的推移慢慢往冷存储上迁移。同时用户可以任意变更数据的冷热分界点，数据可以从热存储到冷存储，也可以从冷存储到热存储。整体架构如图所示：命令介绍设置表的冷热分界线创建冷热分离表：hbase(main):002:0> create 'hot_cold_table', {NAME=>'f', COLD_BOUNDARY=>'86400'}参数说明：NAME：需要冷热分离的列族。COLD_BOUNDARY：冷热分离时间点，单位为秒（s）。例如COLD_BOUNDARY为86400，代表86400秒（一天）前写入的数据会被自动归档到冷存储。取消冷热分离。hbase(main):004:0> alter 'hot_cold_table', {NAME=>'f', COLD_BOUNDARY=>""}为已经存在的表设置冷热分离,或者修改冷热分离分界线，单位为秒。hbase(main):005:0> alter 'hot_cold_table', {NAME=>'f', COLD_BOUNDARY=>'86400'}查询冷热分离是否设置或者修改成功hbase(main):005:0> desc 'hot_cold_table'数据写入冷热分离的表与普通表的数据写入方式完全一致，数据会先存储在热存储（超高IO）中。随着时间的推移，如果一行数据满足当前时间-时间列值>COLD_BOUNDARY设置的值条件，则会在执行Compaction时被归档到冷存储（普通IO）中。插入记录执行“put”命令往指定表插入一条记录，需要指定表的名称，主键，自定义列，以及插入的具体值。hbase(main):004:0> put 'hot_cold_table','row1','cf:a','value1'参数说明：hot_cold_table：表的名称。row1：主键。cf：a：自定义的列。value1：插入的值。数据查询由于冷热数据都在同一张表中，因此用户所有的查询操作都只需在一张表内进行。在查询时，建议通过配置TimeRange来指定查询的时间范围，系统将会根据指定的时间范围决定查询模式，即仅查询热存储、仅查询冷存储或同时查询冷存储和热存储。如果查询时未限定时间范围，则会导致查询冷数据。在这种情况下，查询吞吐量会受到冷存储的限制。随机查询不指定HOT_ONLY参数来查询数据。在这种情况下，将会查询冷存储中的数据。hbase(main):001:0> get 'hot_cold_table', 'row1'通过指定HOT_ONLY参数来查询数据。在这种情况下，只会查询热存储中的数据。hbase(main):002:0> get 'hot_cold_table', 'row1', {HOT_ONLY=>true}通过指定TimeRange参数来查询数据。在这种情况下，CloudTable将会比较TimeRange和冷热边界值，以确定是只查询热存储还是冷存储中的数据，还是同时查询热冷存储中的数据。hbase(main):003:0> get 'hot_cold_table', 'row1', {TIMERANGE => [0, 1568203111265]}范围查询不指定HOT_ONLY参数来查询数据。在这种情况下，将会查询冷存储中的数据。hbase(main):001:0> scan 'hot_cold_table', {STARTROW =>'row1', STOPROW=>'row9'}通过指定HOT_ONLY参数来查询数据。在这种情况下，只会查询热存储中的数据。hbase(main):002:0> scan 'hot_cold_table', {STARTROW =>'row1', STOPROW=>'row9', HOT_ONLY=>true}通过指定TimeRange参数来查询数据。在这种情况下，CloudTable将会比较TimeRange和冷热边界值，以确定是只查询热存储还是冷存储中的数据，还是同时查询热冷存储中的数据。hbase(main):003:0> scan 'hot_cold_table', {STARTROW =>'row1', STOPROW=>'row9', TIMERANGE => [0, 1568203111265]}数据合并合并表所有分区的热数据区。hbase(main):002:0> major_compact 'hot_cold_table', nil, 'NORMAL', 'HOT'合并表所有分区的冷数据区。hbase(main):002:0> major_compact 'hot_cold_table', nil, 'NORMAL', 'COLD'合并表所有分区的热冷数据区。hbase(main):002:0> major_compact 'hot_cold_table', nil, 'NORMAL', 'ALL'HBase 冷热分离效果

1.一个CDL任务，源端是mysql，目标端是hive，填写了binlog。那么假设，在运行一个月后，这个CDL任务失败了，那么重新启动后，读源端mysql的数据是从哪里开始读，是从填写的binglog位置重新读，还是从失败时间点起读？

背景：flink （flink on yarn ）作业通过MRS 客户端提交(flink run -m yarn-cluster)到yarn 上后，这面想自己监控flink  作业状态。 但是 MRS 的yarn 跳转到flink web UI的地址是需要登录的。目的：怎么通过java 代码可以请求WEB UI 里面的接口啊 比如这种接口。这个需要验证。。。