【功能模块】【操作步骤&问题现象】1、在数据转发规则里面选择数据转发目标的时候，没有LTS云日志服务。2、【截图信息】【日志信息】（可选，上传日志内容或者附件）

架构信息和系统  安装mysql及tomcat运行所需依赖 [root@a1 ~]# yum install -y java-1.8.0-openjdk mysql [root@a1 mysql]# mkdir -p /data/mysql/data/ mysql初始化 [root@a1 mysql]# /usr/local/mysql/bin/mysqld --initialize --basedir=/usr/local/mysql --datadir=/data/mysql/data --user=mysql 注意此时密码已经生成，用此密码登录 [root@a1 mysql]# systemctl start mysql  安装tomcat [root@a1 local]# wget https://archive.apache.org/dist/tomcat/tomcat-8/v8.0.21/bin/apache-tomcat-8.0.21.tar.gz  [root@a1 local]# tar zxvf apache-tomcat-8.0.21.tar.gz [root@a1 local]# wget https://downloads.mysql.com/archives/get/p/3/file/mysql-connector-java-8.0.21.tar.gz [root@a1 local]# tar zxvf mysql-connector-java-8.0.21.tar.gz 安装mysql-connect [root@a1 local]# cp mysql-connector-java-8.0.21/mysql-connector-java-8.0.21.jar apache-tomcat-8.0.21/lib/  编写连接mysql文件 [root@a1 local]# cd apache-tomcat-8.0.21/webapps/ [root@a1 webapps]# mkdir test [root@a1 webapps]# vim test/mysql.jsp  [root@a1 apache-tomcat-8.0.21]# cat webapps/test/mysql.jsp  测试连接  自此tomcat连接mysql成功 ps 报错无关连接问题，更改mysql初始password后正常 

架构信息和系统  安装编译需要的依赖 [root@localhost ~]# yum install wget apr-devel apr-util-devel pcre-devel -y  获取httpd源码包 [root@localhost ~]# wget http://archive.apache.org/dist/httpd/httpd-2.4.46.tar.gz [root@localhost ~]# tar zxvf httpd-2.4.46.tar.gz  编译安装 [root@localhost ~]# cd httpd-2.4.46/ [root@localhost httpd-2.4.46]# mkdir /usr/local/httpd-2.4.46  [root@localhost httpd-2.4.46]# ./configure --prefix=/usr/local/httpd-2.4.46/   [root@localhost httpd-2.4.46]# make -j16   [root@localhost httpd-2.4.46]# make install   运行 [root@localhost httpd-2.4.46]# /usr/local/httpd-2.4.46/bin/apachectl start  通过浏览器输入服务器地址，本次采用默认80端口  自此httpd在openEuler-20.03-LTS-SP1安装完成

部署说明本文以root权限执行以下操作，如非root权限用户可加sudo后或切换root权限用户执行命令。一、基础环境属性说明OSopenEuler-20.03-LTS-SP1CPU架构x86_64/aarch64规格4C/8G/40G二、确认架构环境以X86架构为例，执行如下命令查看操作系统及系统架构：cat /etc/*-release arch三、安装Hive-3.1.1详细步骤①yum安装jdk-1.8.0，并配置环境变量yum install java-1.8.0-openjdk* vim /etc/profile export JAVA_HOME=/usr/lib/jvm/java export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/jre/lib/rt.jar export PATH=$PATH:$JAVA_HOME/bin # 环境变量生效 source /etc/profile校验java是否安装完成及其版本java -version②部署Hadoop3.3.1注意hive3.x和hadoop3.x适配#下载hadoop-3.1.1源码 wget https://archive.apache.org/dist/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz解压至/usr/local目录tar zxf hadoop-3.3.1.tar.gz -C /usr/local配置环境变量vim /etc/profile在文件中添加以下内容HADOOP_HOME=/usr/local/hadoop-3.3.1 PATH=$HADOOP_HOME/bin:$PATH export HADOOP_HOME PATH使环境变量立即生效source /etc/profile检查Hadoop是否安装成功hadoop version③部署mariadbyum安装mariadb，启动服务并设置开机自启动yum install mariadb* systemctl start mariadb systemctl enable mariadb查看mariadb服务状态systemctl status mariadb登陆mysql配置密码并授权mysql -uroot -p # 首次登陆无密码，直接回车登陆 set password=password('你的密码'); grant all on *.* to root@'%' identified by '你的密码'; flush privileges;配置my.cnfvim /etc/my.cnf添加以下内容[mysqld] init_connect='SET collation_connection = utf8_unicode_ci' init_connect='SET NAMES utf8' character-set-server=utf8 collation-server=utf8_unicode_ci skip-character-set-client-handshake重启mariadbsystemctl restart mariadb④安装Hive-3.1.1下载apache hive-3.1.1安装包wget https://archive.apache.org/dist/hive/hive-3.1.1/apache-hive-3.1.1-bin.tar.gz解压至/usr/local目录tar zxf apache-hive-3.1.1-bin.tar.gz -C /usr/local/配置环境变量vim /etc/profile添加以下内容# hive export HIVE_HOME=/usr/local/apache-hive-3.1.1-bin export PATH=$HIVE_HOME/bin:$PATH使环境变量立即生效source /etc/profile检查hive是否安装成功hive --version复制配置文件模板cd /usr/local/apache-hive-3.1.1-bin/conf cp hive-env.sh.template hive-env.sh vim hive-env.sh添加以下内容export JAVA_HOME=/usr/lib/jvm/java export HADOOP_HOME=/usr/local/hadoop-3.3.1 export HIVE_CONF_DIR=/usr/local/apache-hive-3.1.1-bin/conf复制hive-site.xml模板文件cp hive-default.xml.template hive-site.xml sed -i 's/for&#/for/g' hive-site.xml vim hive-site.xml修改为下列的value的值，注意格式<name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value> <name>javax.jdo.option.ConnectionDriverName</name> <value>org.mariadb.jdbc.Driver</value> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> <name>javax.jdo.option.ConnectionPassword</name> <value>数据库密码</value> <name>hive.exec.local.scratchdir</name> <value>/tmp/hive-3.1.1</value> <name>hive.downloaded.resources.dir</name> <value>/tmp/${hive.session.id}_resources</value> <name>hive.querylog.location</name> <value>/tmp/hive-3.1.1</value>下载JDBC驱动，并放置/usr/local/apache-hive-3.1.1-bin/lib目录下cd /usr/local/apache-hive-3.1.1-bin/lib wget https://downloads.mariadb.com/Connectors/java/connector-java-2.3.0/mariadb-java-client-2.3.0.jar创建hive数据目录并配置文件夹权限/usr/local/hadoop-3.3.1/bin/hadoop fs -mkdir /tmp /usr/local/hadoop-3.3.1/bin/hadoop fs -mkdir -p /user/hive/warehouse /usr/local/hadoop-3.3.1/bin/hadoop fs -chmod g+w /tmp /usr/local/hadoop-3.3.1/bin/hadoop fs -chmod g+w /user/hive/warehouse创建Hive日志目录mkdir -p /usr/local/apache-hive-3.1.1-bin/log/ touch /usr/local/apache-hive-3.1.1-bin/log/hiveserver.log touch /usr/local/apache-hive-3.1.1-bin/log/hiveserver.err初始化Hiveschematool -dbType mysql -initSchema启动Hive-3.1.1至此，在openEuler-20.03-LTS-SP1上面部署hive-3.1.1已完成。

编译调试插件是鲲鹏开发框架的一个子工具。编译调试插件即插即用，支持一键安装服务器鲲鹏编译器，支持单机下Nvidia GPU应用调试能力，通过统一调试界面调试GPU应用，实现cuda-gdb调试能力，以及鲲鹏平台远程调试能力，支持的功能特性如下：一键式部署支持从Visual Studio Marketplace下载并在线安装插件，同时支持一键部署服务端GCC for openEuler、毕昇编译器和毕昇JDK编译调试一键式安装GCC for openEuler、毕昇编译器和毕昇JDK可视化编译配置任务，一键式任务运行远程单步调试C/C++代码编译调试过程信息实时展示gtest框架用例树渲染及状态展示CUDA程序调试一键式安装部署工具调试信息实时交互

操作系统和软件介绍操作系统为openEuler 20.03 (LTS-SP1)   可用如下命令查询：cat /etc/os-release uname -iCodeViz介绍CodeViz是《Understanding The Linux Virtual Memory Manager》的作者 Mel Gorman 写的一款分析C/C++源代码中函数调用关系的open source工具（类似的open source软件有 egypt、ncc）。其基本原理是给 GCC 打个补丁（如果你的gcc版本不符合它的要求还得先下载正确的gcc版本），让它在编译每个源文件时 dump 出其中函数的 call graph，然后用 Perl 脚本收集并整理调用关系，转交给Graphviz绘制图形（Graphviz属于后端，CodeViz属于前端）。Graphviz + CodeViz编译安装1. 安装 GraphViz调用图的生成依赖于 GraphViz，所以首先要安装 GraphViz。可以下载源码包编译、安装。可以直接yum安装：[root@1ocalhost ~]# yum install graphviz2. 安装 CodeViz下载CodeVize源码包： 很多源都失效了，还好下面这个还在 https://github.com/petersenna/codeviz 进入/root目录下载[root@1ocalhost ~]# cd /root/ [root@1ocalhost ~]# wget https://github.com/petersenna/codeviz/archive/refs/heads/master.zip --2021-08-24 00:12:07-- https://github.com/petersenna/codeviz/archive/refs/heads/master.zip Resolving github.com (github.com)... 52.74.223.119 Connecting to github.com (github.com)|52.74.223.119|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://codeload.github.com/petersenna/codeviz/zip/refs/heads/master [following] --2021-08-24 00:12:10-- https://codeload.github.com/petersenna/codeviz/zip/refs/heads/master Resolving codeload.github.com (codeload.github.com)... 20.205.243.165 Connecting to codeload.github.com (codeload.github.com)|20.205.243.165|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [application/zip] Saving to: ‘master.zip’ master.zip [ <=> ] 2.62M 5.12MB/s in 0.5s 2021-08-24 00:12:11 (5.12 MB/s) - ‘master.zip’ saved [2753095]复制一份并解压[root@1ocalhost ~]# cp master.zip codeviz.zip [root@1ocalhost ~]# unzip codeviz.zip 进入解压后的目录：[root@1ocalhost ~]# cd codeviz-master/CodeViz 使用了一个 patch 版本的 GCC 编译器，而且不同的 CodeViz 版本使用的GCC 版本也不同，可以下载 CodeViz 的源码包后查看 Makefile 文件来确定要使用的 GCC 版本，codeviz-1.0.12 使用 GCC-4.6.2。实际上安装 CodeViz 时安装脚本make会检查当前的GCC版本如果不符合则会自动下载对应的 GCC并打 patch，但由于GCC较大如果网速不好且在虚拟机中的话容易下载失败或系统错误什么的，因此这里我们还是分步安装比较好，先安装gcc再回来安装 CodeViz。1）安装 GCC下载gcc-4.6.2.tar.gz到/root/codeviz-master目录下的compilers里。下载地址：ftp://ftp.gnu.org/pub/gnu/gcc/gcc-4.6.2/gcc-4.6.2.tar.gz[root@1ocalhost codeviz-master]# cd compilers/ [root@1ocalhost compilers]# wget ftp://ftp.gnu.org/pub/gnu/gcc/gcc-4.6.2/gcc-4.6.2.tar.gzCodeViz 的安装脚本 compilers/install_gcc-4.6.2.sh 会自动检测 compilers 目录下是否有 gcc 的源码包，若没有则自动下载并打 patch。这里前面已经下载，则剩下的就是解压安装了。install_gcc-3.4.6.sh 会解压缩 gcc打 patch，并将其安装到指定目录。安装： ./install_gcc-4.6.2.sh[root@1ocalhost compilers]# ./install_gcc-4.6.2.sh注意：这里可能安装时有些错误，具体错误及解决方案见文末后面。 GMP， MPFR， MPC这三个库安装好之后，我们修改一下install_gcc-4.6.2.sh文件，在# Configure and compile的地方指定三个库安装路径：--with-gmp=/usr/local/gmp-4.3.2/ --with-mpfr=/usr/local/mpfr-2.4.2/ --with-mpc=/usr/local/mpc-0.8.1/ [root@1ocalhost compilers]# vim install_gcc-4.6.2.sh 再次安装： ./install_gcc-4.6.2.sh 需要耐心等待一段时间。[root@1ocalhost compilers]# ./install_gcc-4.6.2.sh这次报错，是由于缺少32位兼容包。/usr/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory解决办法：[root@1ocalhost compilers]# wget http://www.rpmfind.net/linux/centos/7.9.2009/os/x86_64/Packages/glibc-devel-2.17-317.el7.i686.rpm --2021-08-24 17:37:54-- http://www.rpmfind.net/linux/centos/7.9.2009/os/x86_64/Packages/glibc-devel-2.17-317.el7.i686.rpm Resolving www.rpmfind.net (www.rpmfind.net)... 195.220.108.108 Connecting to www.rpmfind.net (www.rpmfind.net)|195.220.108.108|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1128200 (1.1M) [application/x-rpm] Saving to: ‘glibc-devel-2.17-317.el7.i686.rpm’ glibc-devel-2.17-317.el7.i686 100%[=================================================>] 1.08M 694KB/s in 1.6s 2021-08-24 17:37:59 (694 KB/s) - ‘glibc-devel-2.17-317.el7.i686.rpm’ saved [1128200/1128200] [root@1ocalhost compilers]# rpm -i --force --nodeps glibc-devel-2.17-317.el7.i686.rpm warning: glibc-devel-2.17-317.el7.i686.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY/usr/include/gnu/下查看，文件stubs-32.h已经存在安装过程发现还需要做如下修改：/root/codeviz-master/compilers/gcc-graph/gcc-4.6.2/gcc/config/i386/linux-unwind.h文件中的struct siginfo 改成 siginfo_t，struct ucontext 改为 ucontext_t将struct ucontext *uc_ = context->cfa; 改为 struct ucontext_t *uc_ = context->cfa;[root@1ocalhost compilers]# vi /root/codeviz-master/compilers/gcc-graph/gcc-4.6.2/gcc/config/i386/linux-unwind.h 设置环境变量：vi /etc/profile export LD_LIBRARY_PATH=/usr/local/gmp-4.3.2/lib:/usr/local/mpfr-2.4.2/lib:/usr/local/mpc-0.8.1/lib //使其生效即可 source /etc/profile最后一次安装使用如下命令：[root@1ocalhost gcc-4.6.2]# cd /root/codeviz-master/compilers/gcc-graph/gcc-4.6.2 [root@1ocalhost gcc-4.6.2]# ./configure --prefix=/usr/local/gcc-4.6.2 --enable-threads=posix --disable-checking --disable-multilib --enable-languages=c,c++ --with-gmp=/usr/local/gmp-4.3.2 --with-mpfr=/usr/local/mpfr-2.4.2 --with-mpc=/usr/local/mpc-0.8.1make终于编译成功了,效果图入下:makemake install gcc4.6.2安装成功如下 /usr/local/gcc-4.6.2/bin/gcc -v Using built-in specs. COLLECT_GCC=/usr/local/gcc-4.6.2/bin/gcc COLLECT_LTO_WRAPPER=/usr/local/gcc-4.6.2/libexec/gcc/x86_64-unknown-linux-gnu/4.6.2/lto-wrapper Target: x86_64-unknown-linux-gnu Configured with: ./configure --prefix=/usr/local/gcc-4.6.2 --enable-threads=posix --disable-checking --disable-multilib --enable-languages=c,c++ --with-gmp=/usr/local/gmp-4.3.2 --with-mpfr=/usr/local/mpfr-2.4.2 --with-mpc=/usr/local/mpc-0.8.1 Thread model: posix gcc version 4.6.2 (GCC) 2）安装 Codeviz4.6.2[root@1ocalhost gcc-4.6.2]# cd /root/codeviz-master/ [root@1ocalhost codeviz-master]# ./configure && make install-codevizchecking for dot.../usr/bin/dot checking for make.../usr/bin/make checking for gcc.../usr/bin/gcc checking dot can generate graphs...yes checking perl library path.../usr/local/lib64/perl5 Creating Makefile Creating testsuite/c/Makefile Configured. Running "make" will compile a patched version of gcc suitable for use with codeviz. Running "make install" when it completes will install the patched version of gcc to and the other codeviz scripts under /usr/local. When compiling other programs for use with codeviz, make sure you use the patched version of gcc. For example, to build a project based on C, the following should work; make CC=/bin/gcc Configuration: PREFIX = /usr/local GCC version to patch = 4.6.2 Patched GCC install path = /usr/local/gccgraph mkdir -p /usr/local/bin cp /root/codeviz-master/bin/* /usr/local/bin mkdir -p /usr/local/lib64/perl5 cp -rv lib/CodeViz /usr/local/lib64/perl5 'lib/CodeViz/Collect.pm' -> '/usr/local/lib64/perl5/CodeViz/Collect.pm' 'lib/CodeViz/CollectCNcc.pm' -> '/usr/local/lib64/perl5/CodeViz/CollectCNcc.pm' 'lib/CodeViz/CollectCObjdump.pm' -> '/usr/local/lib64/perl5/CodeViz/CollectCObjdump.pm' 'lib/CodeViz/CollectCXref.pm' -> '/usr/local/lib64/perl5/CodeViz/CollectCXref.pm' 'lib/CodeViz/CollectCppDepn.pm' -> '/usr/local/lib64/perl5/CodeViz/CollectCppDepn.pm' 'lib/CodeViz/CollectPPStack.pm' -> '/usr/local/lib64/perl5/CodeViz/CollectPPStack.pm' 'lib/CodeViz/Format.pm' -> '/usr/local/lib64/perl5/CodeViz/Format.pm' 'lib/CodeViz/Graph.pm' -> '/usr/local/lib64/perl5/CodeViz/Graph.pm' 'lib/CodeViz/IPC.pm' -> '/usr/local/lib64/perl5/CodeViz/IPC.pm' 'lib/CodeViz/Layout.pm' -> '/usr/local/lib64/perl5/CodeViz/Layout.pm' 'lib/CodeViz/Output.pm' -> '/usr/local/lib64/perl5/CodeViz/Output.pm' 'lib/CodeViz/PPCStack.pm' -> '/usr/local/lib64/perl5/CodeViz/PPCStack.pm' 'lib/CodeViz/PPOprofile.pm' -> '/usr/local/lib64/perl5/CodeViz/PPOprofile.pm' 'lib/CodeViz/PPStack.pm' -> '/usr/local/lib64/perl5/CodeViz/PPStack.pm' 'lib/CodeViz/SourceMarkup.pm' -> '/usr/local/lib64/perl5/CodeViz/SourceMarkup.pm' 'lib/CodeViz/VRML.pm' -> '/usr/local/lib64/perl5/CodeViz/VRML.pm'不需要 make ，因为make的作用就是检测是否有gcc若没有则下载源码包，所以这里只要安装 codeviz 即可。具体查看 Makefile 文件。注意：这里为什么不是通常用的make install，因为这里make install的作用是先安装gcc再安装codeviz，而前面已经安装了 gcc，所以这里只需要安装 codeviz ，即make install-codeviz脚本，该脚本也就是将genfull 和 gengraph 复制到/usr/local/bin 目录下。目前为止,CodeViz 安装完成了。安装过程出现的错误及解决方案1. 在运行./install_gcc-4.6.2.sh时出现下面错误：gcc configure: error: Building GCC requires GMP 4.2+, MPFR 2.3.1+ and MPC 0.8.0+从错误中可以看出：GCC编译需要GMP， MPFR， MPC这三个库（有的系统已经安装了就没有这个提示，我的没有安装），gcc 源码包中自带了一个gcc依赖库安装脚本download_prerequisites，位置在gcc源码目录中的 contrib/download_prerequisites，因此只需要进入该目录，直接运行脚本安装即可：. /download_prerequisitescd /root/codeviz-master/compilers/gcc-graph/gcc-4.6.2/contrib ./download_prerequisites[root@1ocalhost contrib]# pwd /root/codeviz-master/compilers/gcc-graph/gcc-4.6.2/contrib [root@1ocalhost contrib]# ./download_prerequisites --2021-08-24 01:03:37-- ftp://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2 => ‘mpfr-2.4.2.tar.bz2’ Resolving gcc.gnu.org (gcc.gnu.org)... 8.43.85.97, 2620:52:3:1:0:246e:9693:128c Connecting to gcc.gnu.org (gcc.gnu.org)|8.43.85.97|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /pub/gcc/infrastructure ... done. ==> SIZE mpfr-2.4.2.tar.bz2 ... 1077886 ==> PASV ... done. ==> RETR mpfr-2.4.2.tar.bz2 ... done. Length: 1077886 (1.0M) (unauthoritative) mpfr-2.4.2.tar.bz2 100%[================================>] 1.03M 20.2KB/s in 84s 2021-08-24 01:05:09 (12.6 KB/s) - ‘mpfr-2.4.2.tar.bz2’ saved [1077886] --2021-08-24 01:05:09-- ftp://gcc.gnu.org/pub/gcc/infrastructure/gmp-4.3.2.tar.bz2 => ‘gmp-4.3.2.tar.bz2’ Resolving gcc.gnu.org (gcc.gnu.org)... 8.43.85.97, 2620:52:3:1:0:246e:9693:128c Connecting to gcc.gnu.org (gcc.gnu.org)|8.43.85.97|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /pub/gcc/infrastructure ... done. ==> SIZE gmp-4.3.2.tar.bz2 ... 1897483 ==> PASV ... done. ==> RETR gmp-4.3.2.tar.bz2 ... done. Length: 1897483 (1.8M) (unauthoritative) gmp-4.3.2.tar.bz2 100%[================================>] 1.81M 14.0KB/s in 2m 17s 2021-08-24 01:07:32 (13.5 KB/s) - ‘gmp-4.3.2.tar.bz2’ saved [1897483] --2021-08-24 01:07:32-- ftp://gcc.gnu.org/pub/gcc/infrastructure/mpc-0.8.1.tar.gz => ‘mpc-0.8.1.tar.gz’ Resolving gcc.gnu.org (gcc.gnu.org)... 8.43.85.97, 2620:52:3:1:0:246e:9693:128c Connecting to gcc.gnu.org (gcc.gnu.org)|8.43.85.97|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /pub/gcc/infrastructure ... done. ==> SIZE mpc-0.8.1.tar.gz ... 544950 ==> PASV ... done. ==> RETR mpc-0.8.1.tar.gz ... done. Length: 544950 (532K) (unauthoritative) mpc-0.8.1.tar.gz 100%[================================>] 532.18K 27.0KB/s in 18s 2021-08-24 01:07:55 (28.8 KB/s) - ‘mpc-0.8.1.tar.gz’ saved [544950] [root@1ocalhost contrib]# PS：该脚本内容如下： 1234567891011121314151617181920212223242526272829303132333435363738#! /bin/sh # Download some prerequisites needed by gcc.# Run this from the top level of the gcc source tree and the gcc# build will do the right thing.## (C) 2010 Free Software Foundation## This program is free software: you can redistribute it and/or modify# it under the terms of the GNU General Public License as published by# the Free Software Foundation, either version 3 of the License, or# (at your option) any later version.## This program is distributed in the hope that it will be useful, but# WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU# General Public License for more details.## You should have received a copy of the GNU General Public License# along with this program. If not, see http://www.gnu.org/licenses/. MPFR=mpfr-2.4.2GMP=gmp-4.3.2MPC=mpc-0.8.1 wget ftp://gcc.gnu.org/pub/gcc/infrastructure/$MPFR.tar.bz2 || exit 1tar xjf $MPFR.tar.bz2 || exit 1ln -sf $MPFR mpfr || exit 1 wget ftp://gcc.gnu.org/pub/gcc/infrastructure/$GMP.tar.bz2 || exit 1tar xjf $GMP.tar.bz2  || exit 1ln -sf $GMP gmp || exit 1 wget ftp://gcc.gnu.org/pub/gcc/infrastructure/$MPC.tar.gz || exit 1tar xzf $MPC.tar.gz || exit 1ln -sf $MPC mpc || exit 1 rm $MPFR.tar.bz2 $GMP.tar.bz2 $MPC.tar.gz  exit 1可见是通过wget的方式下载安装，因此如果没有安装wget则需要先安装下。大家仔细看下这个脚本，发现非常简单，就是从网上自动下载三个依赖库并解压，然后建立三个改名后的软链接分别指向这三个库，这里建立软链接过程中也可能出错，具体看问题2这里我们自行安装，根据如下顺序分别安装GMP，MPFR和MPC（mpfr依赖gmp，mpc依赖gmp和mpfr）安装gmp: 编译gmp时需要m4文件，这里我们提前安装m4[root@1ocalhost contrib]# yum install m4 Last metadata expiration check: 0:23:04 ago on Tue 24 Aug 2021 10:56:09 AM CST. Dependencies resolved. ================================================================================================== Package Architecture Version Repository Size ================================================================================================== Installing: m4 x86_64 1.4.18-13.oe1 OS 90 k Transaction Summary ================================================================================================== Install 1 Package Total download size: 90 k Installed size: 203 k Is this ok [y/N]: y Downloading Packages: m4-1.4.18-13.oe1.x86_64.rpm 97 kB/s | 90 kB 00:00 -------------------------------------------------------------------------------------------------- Total 97 kB/s | 90 kB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : m4-1.4.18-13.oe1.x86_64 1/1 Verifying : m4-1.4.18-13.oe1.x86_64 1/1 Installed: m4-1.4.18-13.oe1.x86_64 Complete![root@1ocalhost contrib]# cd gmp-4.3.2/ [root@1ocalhost gmp-4.3.2]# ./configure --prefix=/usr/local/gmp-4.3.2make install安装mpfr:[root@1ocalhost gmp-4.3.2]# cd ../mpfr-2.4.2/ [root@1ocalhost mpfr-2.4.2]# pwd /root/codeviz-master/compilers/gcc-graph/gcc-4.6.2/contrib/mpfr-2.4.2 [root@1ocalhost mpfr-2.4.2]# ./configure --prefix=/usr/local/mpfr-2.4.2 --with-gmp=/usr/local/gmp-4.3.2/make install安装mpc: [root@1ocalhost mpfr-2.4.2]# cd ../mpc-0.8.1/ [root@1ocalhost mpc-0.8.1]# pwd /root/codeviz-master/compilers/gcc-graph/gcc-4.6.2/contrib/mpc-0.8.1 [root@1ocalhost mpc-0.8.1]# ./configure --prefix=/usr/local/mpc-0.8.1 --with-gmp=/usr/local/gmp-4.3.2/ --with-mpfr=/usr/local/mpfr-2.4.2/make install2.  解决ln -s 软链接产生Too many levels of symbolic links错误从网上查找了一下原因，原来是建立软连接的时候采用的是相对路径，所以才会产生这样的错误，解决方式是采用绝对路径建立软链接：这样问题就解决了。

一、查看系统信息二、安装Java注意：请勿安装jdk1.7及之前的版本，否则hive启动时会报错。jdk下载链接：https://www.oracle.com/cn/java/technologies/javase/javase-jdk8-downloads.html下载jdk软件包，解压至/usr/java目录tar zxf jdk-8u301-linux-x64.tar.gz -C /usr/java/配置环境变量vim /etc/porfile文件中添加以下内容export JAVA_HOME=/usr/java/jdk1.8.0_301 export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=.$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar执行以下命令使环境变量立即生效source /etc/profile检查Java是否安装成功java -version三、安装Hadoop下载Hadoopwget https://archive.apache.org/dist/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz解压至/usr/local目录，并在目录下创建软链接tar zxf hadoop-3.3.1.tar.gz -C /usr/local cd /usr/local ln -sv hadoop-3.3.1 hadoop配置环境变量vim /etc/profile在文件中添加以下内容HADOOP_HOME=/usr/local/hadoop PATH=$HADOOP_HOME/bin:$PATH export HADOOP_HOME PATH使环境变量立即生效source /etc/profile检查Hadoop是否安装成功hadoop version四、安装hive首先安装并启动mariadbyum install mariadb* systemctl start mariadb.service2、登陆mysql配置密码并授权mysql -uroot -p # 首次登陆无密码，直接回车登陆 set password=password('你的密码'); grant all on *.* to root@'%' identified by '你的密码'; flush privileges;输入quit退出配置my.cnfvim /etc/my.cnf添加以下内容[mysqld] init_connect='SET collation_connection = utf8_unicode_ci' init_connect='SET NAMES utf8' character-set-server=utf8 collation-server=utf8_unicode_ci skip-character-set-client-handshake重启mariadbsystemctl restart mariadb.service下载apache hive 3.0.0安装包wget https://archive.apache.org/dist/hive/hive-3.0.0/apache-hive-3.0.0-bin.tar.gz解压至/usr/local目录并创建软链接tar zxf apache-hive-3.0.0-bin.tar.gz -C /usr/local/ cd /usr/local ln -sv apache-hive-3.0.0-bin hive配置环境变量vim /etc/profile添加以下内容# hive export HIVE_HOME=/usr/local/hive export PATH=$HIVE_HOME/bin:$PATH使环境变量立即生效source /etc/profile检查hive是否安装成功hive --version复制配置文件模板cd /usr/local/hive/conf cp hive-env.sh.template hive-env.sh vim hive-env.sh添加以下内容export JAVA_HOME=/usr/java/jdk1.8.0_301 export HADOOP_HOME=/usr/local/hadoop export HIVE_CONF_DIR=/usr/local/hive/conf复制hive-site.xml模板文件cp hive-default.xml.template hive-site.xml sed -i 's/for&#/for/g' hive-site.xml vim hive-site.xml找到以下对应name标签，修改value的值<name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value> <name>javax.jdo.option.ConnectionDriverName</name> <value>org.mariadb.jdbc.Driver</value> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> <name>javax.jdo.option.ConnectionPassword</name> <value>数据库密码</value> <name>hive.exec.local.scratchdir</name> <value>/tmp/hive</value> <name>hive.downloaded.resources.dir</name> <value>/tmp/${hive.session.id}_resources</value> <name>hive.querylog.location</name> <value>/tmp/hive</value>下载JDBC驱动，并放置/usr/local/hive/lib目录下cd /usr/local/hive/lib wget https://downloads.mariadb.com/Connectors/java/connector-java-2.3.0/mariadb-java-client-2.3.0.jar创建hive数据目录并配置文件夹权限/usr/local/hadoop/bin/hadoop fs -mkdir /tmp /usr/local/hadoop/bin/hadoop fs -mkdir -p /user/hive/warehouse /usr/local/hadoop/bin/hadoop fs -chmod g+w /tmp /usr/local/hadoop/bin/hadoop fs -chmod g+w /user/hive/warehouse创建Hive日志目录mkdir -p /usr/local/hive/log/ touch /usr/local/hive/log/hiveserver.log touch /usr/local/hive/log/hiveserver.err初始化Hiveschematool -dbType mysql -initSchema启动hive至此，hive3.0.0安装完成。

一、查看系统信息二、安装Java注意：请勿安装jdk1.7及之前的版本，否则hive启动时会报错。jdk下载链接：https://www.oracle.com/cn/java/technologies/javase/javase-jdk8-downloads.html下载jdk软件包，解压至/usr/java目录tar zxf jdk-8u301-linux-x64.tar.gz -C /usr/java/配置环境变量vim /etc/porfile文件中添加以下内容export JAVA_HOME=/usr/java/jdk1.8.0_301 export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=.$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar执行以下命令使环境变量立即生效source /etc/profile检查Java是否安装成功java -version三、安装Hadoop下载Hadoopwget http://archive.apache.org/dist/hadoop/core/hadoop-2.9.2/hadoop-2.9.2.tar.gz解压至/usr/local目录，并在目录下创建软链接tar zxf hadoop-2.9.2.tar.gz -C /usr/local cd /usr/local ln -sv hadoop-2.9.2 hadoop配置环境变量vim /etc/profile在文件中添加以下内容HADOOP_HOME=/usr/local/hadoop PATH=$HADOOP_HOME/bin:$PATH export HADOOP_HOME PATH使环境变量立即生效source /etc/profile检查Hadoop是否安装成功hadoop version四、安装hive安装并启动mariadbyum install mariadb* systemctl start mariadb.service登陆mysql配置密码并授权mysql -uroot -p # 首次登陆无密码，直接回车登陆 set password=password('你的密码'); grant all on *.* to root@'%' identified by '你的密码'; flush privileges;配置my.cnfvim /etc/my.cnf添加以下内容[mysqld] init_connect='SET collation_connection = utf8_unicode_ci' init_connect='SET NAMES utf8' character-set-server=utf8 collation-server=utf8_unicode_ci skip-character-set-client-handshake重启mariadbsystemctl restart mariadb.service下载apache hive 2.3.1安装包wget https://archive.apache.org/dist/hive/hive-2.3.1/apache-hive-2.3.1-bin.tar.gz解压至/usr/local目录并创建软链接tar zxf apache-hive-2.3.1-bin.tar.gz -C /usr/local/ cd /usr/local ln -sv apache-hive-2.3.1-bin hive配置环境变量vim /etc/profile添加以下内容# hive export HIVE_HOME=/usr/local/hive export PATH=$HIVE_HOME/bin:$PATH使环境变量立即生效source /etc/profile检查hive是否安装成功hive --version复制配置文件模板cd /usr/local/hive/conf cp hive-env.sh.template hive-env.sh vim hive-env.sh添加以下内容export JAVA_HOME=/usr/java/jdk1.8.0_301 export HADOOP_HOME=/usr/local/hadoop export HIVE_CONF_DIR=/usr/local/hive/conf复制hive-site.xml模板文件cp hive-default.xml.template hive-site.xml sed -i 's/for&#/for/g' hive-site.xml vim hive-site.xml找到以下对应name标签，修改value的值<name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value> <name>javax.jdo.option.ConnectionDriverName</name> <value>org.mariadb.jdbc.Driver</value> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> <name>javax.jdo.option.ConnectionPassword</name> <value>数据库密码</value> <name>hive.exec.local.scratchdir</name> <value>/tmp/hive</value> <name>hive.downloaded.resources.dir</name> <value>/tmp/${hive.session.id}_resources</value> <name>hive.querylog.location</name> <value>/tmp/hive</value>下载JDBC驱动，并放置/usr/local/hive/lib目录下cd /usr/local/hive/lib wget https://downloads.mariadb.com/Connectors/java/connector-java-2.3.0/mariadb-java-client-2.3.0.jar创建hive数据目录并配置文件夹权限/usr/local/hadoop/bin/hadoop fs -mkdir /tmp /usr/local/hadoop/bin/hadoop fs -mkdir -p /user/hive/warehouse /usr/local/hadoop/bin/hadoop fs -chmod g+w /tmp /usr/local/hadoop/bin/hadoop fs -chmod g+w /user/hive/warehouse创建Hive日志目录mkdir -p /usr/local/hive/log/ touch /usr/local/hive/log/hiveserver.log touch /usr/local/hive/log/hiveserver.err初始化Hiveschematool -dbType mysql -initSchema启动hive至此，hive2.3.1安装完成。

一.环境信息二.安装jdkyum install java-1.8.0-openjdk*配置环境变量vim /etc/profile export JAVA_HOME=/usr/lib/jvm/java export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/jre/lib/rt.jar export PATH=$PATH:$JAVA_HOME/bin环境变量生效source /etc/profile查看jdk版本java -version三.安装部署hadoop注意hive2.x和hadoop2.x适配下载wget http://archive.apache.org/dist/hadoop/core/hadoop-2.6.5/hadoop-2.6.5.tar.gz解压至/usr/local目录下tar -xzvf hadoop-2.6.5.tar.gz -C /usr/local配置环境变量vim /etc/profile export HADOOP_HOME=/usr/local/hadoop export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH环境变量生效source /etc/profile创建软连接cd /usr/local/ ln -s hadoop-2.6.5/ hadoop查看hadoop版本hadoop version四.安装部署apache hive首先安装并启动mariadbyum install mariadb* systemctl start mariadb.service查看启动状态systemctl status mariadb.service登陆mysql配置密码并授权,密码不可过于简单mysql -uroot -p # 首次登陆无密码，直接回车登陆 set password=password('qwer1234!@#$'); grant all on *.* to root@'%' identified by 'qwer1234!@#$'; flush privileges;编辑配置my.cnfvim /etc/my.cnf [mysqld] init_connect='SET collation_connection = utf8_unicode_ci' init_connect='SET NAMES utf8' character-set-server=utf8 collation-server=utf8_unicode_ci skip-character-set-client-handshake重启mariadbsystemctl restart mariadb获取hive 2.0.1版本的软件安装包wget https://archive.apache.org/dist/hive/hive-2.0.1/apache-hive-2.0.1-bin.tar.gz解压至/usr/local目录下tar -xzvf apache-hive-2.0.1-bin.tar.gz -C /usr/local/创建软连接cd /usr/local ln -s apache-hive-2.0.1-bin hive配置环境变量vim /etc/profile export HIVE_HOME=/usr/local/hive export PATH=$HIVE_HOME/bin:$PATH环境变量生效source /etc/profile查看hive版本hive --version修改配置文件cd /usr/local/hive/conf cp hive-env.sh.template hive-env.sh vim hive-env.sh export JAVA_HOME=/usr/lib/jvm/java export HADOOP_HOME=/usr/local/hadoop export HIVE_CONF_DIR=/usr/local/hive/conf复制hive-site.xml这样才会生效cp hive-default.xml.template hive-site.xml sed -i 's/for&#/for/g' hive-site.xml vim hive-site.xml添加如下内容，注意格式 <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>qwer1234!@#$</value> </property> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>org.mariadb.jdbc.Driver</value> </property> <property> <name>hive.metastore.schema.verification</name> <value>false</value> </property>如果后面启动有报错，需要把其他内容删除，只保留这一部分。下载JDBC驱动，并放置/usr/local/hive/lib目录下cd /usr/local/hive/lib wget https://downloads.mariadb.com/Connectors/java/connector-java-2.3.0/mariadb-java-client-2.3.0.jar创建Hive数据目录/usr/local/hadoop/bin/hadoop fs -mkdir /tmp /usr/local/hadoop/bin/hadoop fs -mkdir -p /user/hive/warehouse /usr/local/hadoop/bin/hadoop fs -chmod g+w /tmp /usr/local/hadoop/bin/hadoop fs -chmod g+w /user/hive/warehouse创建Hive日志目录mkdir -p /usr/local/hive/log/ touch /usr/local/hive/log/hiveserver.log touch /usr/local/hive/log/hiveserver.err初始化Hiveschematool -dbType mysql -initSchema启动hive至此Apache Hive2.0.X部署完成。五.Hive基础操作1.查看数据库show databases;2.创建数据库create database if not exists euler;3.创建测试表create table test( name string, friends array<string>, children map<string, int>, address struct<street:string, city:string> ) row format delimited fields terminated by ',' collection items terminated by '_' map keys terminated by ':' lines terminated by '\n';字段解释：row format delimited fields terminated by ‘,’ – 列分隔符collection items terminated by ‘_’ --MAP STRUCT 和 ARRAY 的分隔符(数据分割符号)map keys terminated by ‘:’ – MAP中的key与value的分隔符lines terminated by ‘\n’; – 行分隔符4.创建本地测试文件test.txtongsong,bingbing_lili,xiao song:18_xiaoxiao song:19,hui long guan_beijing yangyang,caicai_susu,xiao yang:18_xiaoxiao yang:19,chao yang_beijing5.导入文本数据到测试表 load data local inpath "/root/test.txt"into table test;6.访问三种集合列里的数据，以下分别是ARRAY，MAP，STRUCT的访问方式select friends[1],children['xiao song'],address.city from test where name="ongsong";

一、操作系统和软件版本介绍1.操作系统为openEuler 20.03 (LTS-SP1)   可用如下命令查询：[root@1ocalhost ~]# cat /etc/os-release [root@1ocalhost ~]# uname -i2.JDK依赖   安装ElasticSearch需要Java，因此在安装ElasticSearch之前需要先安装JDK。openEuler 20.03 (LTS-SP1) 默认没有安装JDK环境，需要自己自行安装。 这里安装JDK环境做简单说明，请开发者自行安装。[root@1ocalhost ~]# yum install java输入Y确认  查看JDK版本[root@1ocalhost ~]# java -version openjdk version "1.8.0_242" OpenJDK Runtime Environment (build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)二、详细安装步骤     下载elasticsearch[root@1ocalhost ~]# cd /opt/ [root@1ocalhost opt]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.1-linux-x86_64.tar.gz  解压[root@1ocalhost opt]# tar -zvxf elasticsearch-7.5.1-linux-x86_64.tar.gz   启动  elasticsearch不需要特别的配置，只需要启动即可。 elasticsearch不能以root账户启动，所以需新建一个用户启动。[root@1ocalhost opt]# useradd es [root@1ocalhost opt]# chown -R es:es /opt/elasticsearch-7.5.1  切换到es用户：su es后台启动[es@1ocalhost opt]$ cd elasticsearch-7.5.1 [es@1ocalhost elasticsearch-7.5.1]$ bin/elasticsearch > elasticsearch.log 2>&1 & [1] 53805 [es@1ocalhost elasticsearch-7.5.1]$ 启动完成后，可使用命令curl http://localhost:9200  验证服务是否开启成功标签：Elasticsearch 云日志服务 LTS

kafka

一、操作系统和软件版本介绍1.操作系统为openEuler 20.03 (LTS-SP1)   可用如下命令查询：[root@1ocalhost ~]# cat /etc/os-release [root@1ocalhost ~]# uname -i二.docker安装  下载软件rpm包wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-17.12.1.ce-1.el7.centos.x86_64.rpm 用yum命令安装[root@1ocalhost ~]# yum install docker-ce-17.12.1.ce-1.el7.centos.x86_64.rpm  加入自启动[root@1ocalhost ~]# systemctl enable docker 启动docker[root@1ocalhost ~]# systemctl start docker [root@1ocalhost ~]# docker run hello-world三.开启远程访问  查询状态[root@1ocalhost ~]# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2021-08-18 14:30:54 CST; 3min 21s ago Docs: https://docs.docker.com Main PID: 39341 (dockerd) Tasks: 19 Memory: 28.7M CGroup: /system.slice/docker.service ├─39341 /usr/bin/dockerd └─39347 docker-containerd --config /var/run/docker/containerd/containerd.toml Aug 18 14:30:53 1ocalhost dockerd[39341]: time="2021-08-18T14:30:53.976077406+08:00" level=info msg="L> Aug 18 14:30:54 1ocalhost dockerd[39341]: time="2021-08-18T14:30:54.043802823+08:00" level=info msg="D> Aug 18 14:30:54 1ocalhost dockerd[39341]: time="2021-08-18T14:30:54.070649796+08:00" level=info msg="L> Aug 18 14:30:54 1ocalhost dockerd[39341]: time="2021-08-18T14:30:54.105805696+08:00" level=info msg="D> Aug 18 14:30:54 1ocalhost dockerd[39341]: time="2021-08-18T14:30:54.105846846+08:00" level=info msg="D> Aug 18 14:30:54 1ocalhost dockerd[39341]: time="2021-08-18T14:30:54.110040371+08:00" level=info msg="A> Aug 18 14:30:54 1ocalhost systemd[1]: Started Docker Application Container Engine. Aug 18 14:31:05 1ocalhost dockerd[39341]: time="2021-08-18T14:31:05+08:00" level=info msg="shim docker> Aug 18 14:31:05 1ocalhost dockerd[39341]: time="2021-08-18T14:31:05+08:00" level=info msg="shim reaped> Aug 18 14:31:05 1ocalhost dockerd[39341]: time="2021-08-18T14:31:05.974762303+08:00" level=info msg="i> lines 1-21/21 (END) 开启远程端口，允许远程访问[root@1ocalhost ~]# vi /usr/lib/systemd/system/docker.service   刷新配置重启docker[root@1ocalhost ~]# systemctl daemon-reload [root@1ocalhost ~]# systemctl restart docker测试[root@1ocalhost ~]# curl http://localhost:2375/version [root@1ocalhost ~]# curl http://localhost:2375/info

说明：本文参考https://bbs.huaweicloud.com/forum/thread-147823-1-1.html一、操作系统和软件版本介绍1.操作系统为openEuler 20.03 (LTS-SP1)   可用如下命令查询：[root@1ocalhost ~]# cat /etc/os-release [root@1ocalhost ~]# uname -i2.软件版本   Kafka版本为：Kafka 2.7.03.JDK依赖   Kafka 是用Scala 语言开发的，运行在JVM上，因此在安装Kafka 之前需要先安装JDK。openEuler 20.03 (LTS-SP1) 默认没有安装JDK环境，需要自己自行安装。 这里安装JDK环境做简单说明，请开发者自行安装。[root@1ocalhost ~]# yum install java输入Y确认配置环境变量vim /etc/profile export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-1.h5.oe1.x86_64 export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/jre/lib/rt.jar export PATH=$PATH:$JAVA_HOME/bin环境变量生效source /etc/profile  查看JDK版本[root@1ocalhost ~]# java -version openjdk version "1.8.0_242" OpenJDK Runtime Environment (build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)二、详细安装步骤1.在/root目录创建data目录cd /root mkdir data2.进入data目录并下载Kafka 2.7.0 [root@1ocalhost ~]# cd data/ [root@1ocalhost data]# wget http://archive.apache.org/dist/kafka/2.7.0/kafka_2.12-2.7.0.tgz3.解压并进入到解压后的目录[root@1ocalhost data]# tar -zvxf kafka_2.12-2.7.0.tgz [root@1ocalhost data]# cd kafka_2.12-2.7.04.启动zookeeper服务和kafka服务，可以在命令的结尾加个&符号，这样服务就可以在后台运行   说明：单节点的 kafka 是不用修改配置文件，直接照官网的介绍就可以，见kafka官网入门[root@1ocalhost kafka_2.12-2.7.0]# /root/data/kafka_2.12-2.7.0/bin/zookeeper-server-start.sh /root/data/kafka_2.12-2.7.0/config/zookeeper.properties > zookeeper.log 2>&1 & [root@1ocalhost kafka_2.12-2.7.0]# /root/data/kafka_2.12-2.7.0/bin/kafka-server-start.sh /root/data/kafka_2.12-2.7.0/config/server.properties > kafkastart.log 2>&1 & 三、验证   kafka原理：Kafka集群将 Record 流存储在称为 Topic 的类别中，每个记录由一个键、一个值和一个时间戳组成。Kafka 中消息是以 Topic 进行分类的，生产者生产消息，消费者消费消息，面向的都是同一个 Topic。Topic 是逻辑上的概念，而 Partition 是物理上的概念，每个 Partition 对应于一个 log 文件，该 log 文件中存储的就是 Producer 生产的数据。Producer 生产的数据会不断追加到该 log 文件末端，且每条数据都有自己的 Offset。消费者组中的每个消费者，都会实时记录自己消费到了哪个 Offset，以便出错恢复时，从上次的位置继续消费。1.创建一个叫"itren"的topic，它只有一个分区，一个副本：[root@1ocalhost kafka_2.12-2.7.0]# /root/data/kafka_2.12-2.7.0/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic itren2.发送消息，运行producer并在控制台中输一些消息，这些消息将被发送到服务端[root@1ocalhost kafka_2.12-2.7.0]# /root/data/kafka_2.12-2.7.0/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic itren >it520 >itren666 >byebye >3.在另一个终端开启consumer，可以读取到刚才发出的消息并输出[root@1ocalhost ~]# /root/data/kafka_2.12-2.7.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic itren it520 itren666 byebye

配置文件是使用openEuler-20.03-LTS-SP2-everything-aarch64-dvd.iso装openEuler时自带的openEuler.repo，按照网络上类似问题的解决方法修改了网络源地址，仍然报相同的错

一、操作系统和软件版本介绍1.操作系统为openEuler 20.03 (LTS-SP1)   可用如下命令查询：[root@ecs-2e3d ~]# cat /etc/os-release [root@ecs-2e3d ~]# uname -i2.软件版本   Kafka版本为：Kafka 2.5.03.JDK依赖   Kafka 是用Scala 语言开发的，运行在JVM上，因此在安装Kafka 之前需要先安装JDK。openEuler 20.03 (LTS-SP1) 默认没有安装JDK环境，需要自己自行安装。 这里安装JDK环境做简单说明，请开发者自行安装。[root@ecs-2e3d ~]# yum install java输入y 配置环境变量  JDK默认安装路径/usr/lib/jvm在/etc/profile文件末尾配置jdk路径vi /etc/profile# set java environment JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-1.h5.oe1.x86_64 PATH=$PATH:$JAVA_HOME/bin CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export JAVA_HOME CLASSPATH PATH保存关闭profile文件，执行如下命令生效source /etc/profile 查看JDK版本[root@ecs-2e3d ~]# java -version openjdk version "1.8.0_242" OpenJDK Runtime Environment (build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)二、详细安装步骤1.在/root创建data目录cd /root mkdir data2.进入data目录并下载Kafka 2.5.0 [root@ecs-2e3d ~]# cd data [root@ecs-2e3d data]# wget http://archive.apache.org/dist/kafka/2.5.0/kafka_2.12-2.5.0.tgz3.解压并进入到解压后的目录[root@ecs-2e3d data]# tar -zvxf kafka_2.12-2.5.0.tgz [root@ecs-2e3d data]# cd kafka_2.12-2.5.04.启动zookeeper服务和kafka服务，可以在命令的结尾加个&符号，这样服务就可以在后台运行   说明：单节点的 kafka 是不用修改配置文件，直接照官网的介绍就可以，见kafka官网入门[root@ecs-2e3d kafka_2.12-2.5.0]# /root/data/kafka_2.12-2.5.0/bin/zookeeper-server-start.sh /root/data/kafka_2.12-2.5.0/config/zookeeper.properties > zookeeper.log 2>&1 & [1] 7742 [root@ecs-2e3d kafka_2.12-2.5.0]# /root/data/kafka_2.12-2.5.0/bin/kafka-server-start.sh /root/data/kafka_2.12-2.5.0/config/server.properties > kafkastart.log 2>&1 & [2] 8115 [root@ecs-2e3d kafka_2.12-2.5.0]# 三、验证Kafka工作流程Kafka将消息按Topic进行分类，每条message由三个属性组成。          offset：表示message在当前Partition（分区）中的偏移量，是一个逻辑上的值，唯一确定了Partition中的一条message，可以简单的认为是一个id；           MessageSize：表示message内容data的大小；           data：message的具体内容；在整个kafka架构中，生产者和消费者采用发布和订阅的模式，生产者生产消息，消费者消费消息，它俩各司其职，并且都是面向topic的。（需要注意：topic是逻辑上的概念，而partition是物理上的概念，每个partition对应于一个log文件，该log文件中存储的就是producer生产的数据）Producer生产的数据会被不断追加到该log文件末端，且每条数据都有自己的offset。消费者组中的每个消费者，都会实时记录自己消费到了哪个offset，这样当出现故障并恢复后，可从这个offset位置继续进行消费，避免漏掉数据或者重复消费。1.创建一个叫"demo"的topic，它只有一个分区，一个副本：[root@ecs-2e3d kafka_2.12-2.5.0]# /root/data/kafka_2.12-2.5.0/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic demo Created topic demo. [root@ecs-2e3d kafka_2.12-2.5.0]#2.发送消息，运行producer并在控制台中输一些消息，这些消息将被发送到服务端[root@ecs-2e3d kafka_2.12-2.5.0]# /root/data/kafka_2.12-2.5.0/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic demo >morning >nihao >goodday! >3.在另一个终端开启consumer，可以读取到刚才发出的消息并输出[root@ecs-2e3d kafka_2.12-2.5.0]# /root/data/kafka_2.12-2.5.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic demo morning nihao goodday!