【Elasticsearch入门】ElasticSearch安装

    xiaoxiao2025-06-01  91

    1、单机安装

    打开网址: https://www.elastic.co/downloads/past-releases, 选择 7.0.0 版本, 获取下载链接。下载之后, 直接解压即可。 运行 bin 目录下的 elasticsearch.bat 文件(linux 下需要在 root 用户下运行 elasticsearch文件) , 出现如下界面即启动成功。

    如果在 CentOS 中, 要做以下设置: 1) 系统参数修改脚本 注意 要以 root 身份执行下面的脚本, 执行后要重新登录普通账户启动 ES

    2) JVM 参数调整 ES 5.5.x 的 JVM 参数配置方式和以往 2.x 等版本不同, 它独立出了一个 jvm.options 的文件在 config 目录下, 我们可以通过修改 jvm.options 里的参数来指定 ES 启动需要的 JVM 环境, 比如 ES 默认是 2G 堆内存, 当内存不足时, 我们可以进行提升, 如下:

    除此之外, 我们还可以针对性的做各种调整, 这里就不再赘述 JVM 的各参数用法。 启动报错问题 在 CentOS 下运行 ES 5.5.x 版本时, 可能会遇到如下报错信息:

    遇到这些错误信息不要慌, 这是因为 CentOS 内核不支持 SecComp, 而 ES 5.5.x 默认是要执行检测命令的, 所以这里我们把这个环境检测禁掉, 就可以正常运行了, 参数如下:

    在配置文件 elasticsearch.yml 追加即可。 到此, 单实例安装就完成了。

    2、集群安装

    架构设计

    ES-Master:ES集群的主节点为10.150.55.94/10.150.55.95,我们设定只有这两台机器有机会称为Master节点。ES-Data:ES集群的数据节点为10.150.55.88/10.150.55.89/10.150.30.241,设定着三台机器为专门存储数据节点。ES-Client:ES集群的客户端节点为10.150.30.242,设定这台机器为为Client节点,主要为负载均衡,请求转发等功能。ES-File-Storage:ES集群专门的海量数据存储服务器。 关于怎么分配ES集群角色类型详情请看:https://www.dockerc.com/elasticsearch-master-or-data/

    IP地址CPU内存应用版本节点功能10.150.55.948c32Gelasticsearch-7.0.0-linux-x86_64.tar.gzES-Master10.150.55.958c32Gelasticsearch-7.0.0-linux-x86_64.tar.gzES-Master10.150.30.2468c16Gelasticsearch-7.0.0-linux-x86_64.tar.gzES-Master10.150.55.888c16Gelasticsearch-7.0.0-linux-x86_64.tar.gzES-Data10.150.55.898c16Gelasticsearch-7.0.0-linux-x86_64.tar.gzES-Data10.150.30.2418c16Gelasticsearch-7.0.0-linux-x86_64.tar.gzES-Data10.150.30.2428c16Gelasticsearch-7.0.0-linux-x86_64.tar.gzES-Client10.150.30.2468c16G文件存储服务器ES-File-Storage

    注意事项

    Elasticsearch原来是需要依赖JDK1.8+的,但是ES7中包含了ES7版本所需的JDK捆绑版本,绑定的JDK版本是ES此版本兼容较好的版本,所以我们只需要安装ES7即可。 ELastic官网:https://www.elastic.co/ Elasticsearch7.0版本下载:https://pan.baidu.com/s/1ewexoP_w9z7E-N4S9Bd-Tw Elasticsearch7.0其它版本:https://www.elastic.co/cn/downloads/past-releases/elasticsearch-7-0-0所有节点关闭防火墙所有节点时间同步一致所有节点关闭SELinux

    注:ES如果做集群的话Master节点至少三台服务器或者三个Master实例加入相同集群,三个Master节点最多只能故障一台Master节点,亲测,如果故障两个Master节点,ES将无法组成集群,会一直报错为“master not discovered or elected yet”,如果生产中发生了此类情况,后果将不堪设想!!! 因为公司测试环境服务空闲机器暂时找不出来很多,所以上面的一台Master服务器节点,我给复用了一台为数据存储节点,见谅!

    节点部署

    ES-File-Storage:10.150.30.246

    我们先把存储服务器给配置出来,我这里就简单的配置了下,但是如果上生产,存储服务器一定要用专用的存储设备!!!至少磁盘要是RAID由冗余功能

    mkdir /data chmod -Rf 777 /data/ vim /etc/exports /data 10.150.*(rw,sync,root_squash) systemctl restart rpcbind systemctl start nfs-server

    ES-Master:10.150.55.94

    先上传到服务器文件解压

    tar xf elasticsearch-7.0.0-linux-x86_64.tar.gz -C /usr/local/ ls /usr/local/elasticsearch-7.0.0/ bin config jdk lib LICENSE.txt logs modules NOTICE.txt plugins README.textile

    bin目录:二进制脚本启动es程序文件以及启动es插件目录 config目录:elasticsearch配置文件 data目录:默认Elasticsearch生成的索引/切片数据文件存放目录,可以指定多个位置来存储数据 lib目录:一些开发的jar包 logs目录:elasticsearch日志目录 modules目录:模块目录 plugins目录:elasticsearch插件目录,此版本tar包装后默认无插件

    bin目录文件介绍

    ll /usr/local/elasticsearch-7.0.0/bin/ total 18188 -rwxr-xr-x 1 root root 1845 Apr 6 06:52 elasticsearch #elasticsearch二进制启动脚本 -rwxr-xr-x 1 root root 121 Apr 6 06:52 elasticsearch-keystore #elasticsearch安全设置,详细用法请看官网:https://www.elastic.co/guide/en/elasticsearch/reference/7.0/secure-settings.html -rwxr-xr-x 1 root root 172 Apr 6 06:52 elasticsearch-plugin ##集成插件启动脚本 -rwxr-xr-x 1 root root 431 Apr 6 06:57 elasticsearch-saml-metadata -rwxr-xr-x 1 root root 438 Apr 6 06:57 elasticsearch-setup-passwords -rwxr-xr-x 1 root root 118 Apr 6 06:52 elasticsearch-shard -rwxr-xr-x 1 root root 427 Apr 6 06:57 elasticsearch-sql-cli -rwxr-xr-x 1 root root 18545268 Apr 6 06:57 elasticsearch-sql-cli-7.0.0.jar -rwxr-xr-x 1 root root 426 Apr 6 06:57 elasticsearch-syskeygen -rwxr-xr-x 1 root root 426 Apr 6 06:57 elasticsearch-users #如果es要基于文件的用户身份验证,那么此脚本可以添加、删除、管理用户角色,具体用法看官网:https://www.elastic.co/guide/en/elasticsearch/reference/7.x/users-command.html -rwxr-xr-x 1 root root 346 Apr 6 06:57 x-pack-env -rwxr-xr-x 1 root root 354 Apr 6 06:57 x-pack-security-env -rwxr-xr-x 1 root root 353 Apr 6 06:57 x-pack-watcher-env

    config目录介绍

    ll /usr/local/elasticsearch-7.0.0/config/ total 36 -rw-rw---- 1 root root 2831 Apr 6 06:52 elasticsearch.yml #ES节点集群等相关配置 -rw-rw---- 1 root root 3524 Apr 6 06:52 jvm.options #JVM堆内存及GC相关配置 -rw-rw---- 1 root root 17170 Apr 6 06:57 log4j2.properties #log4j2框架日志输出相关配置 -rw-rw---- 1 root root 473 Apr 6 06:57 role_mapping.yml #配置身份验证,角色映射文件,其中将elasticsearch角色作为键 映射到一个或多个用户或组专有名称,具体用法见官网:https://www.elastic.co/cn/search?q=role_mapping.yml&size=20 -rw-rw---- 1 root root 197 Apr 6 06:57 roles.yml #定义身份角色相关配置 -rw-rw---- 1 root root 0 Apr 6 06:57 users -rw-rw---- 1 root root 0 Apr 6 06:57 users_roles

    jdk目录介绍

    此目录里面就像是我们解压过的JDK一样,官网给出,es集成的JDK默认只为es服务,如果我们系统想要使用es所集成的jdk版本,我们需要添加环境变量至此目录。 当然我们也可以自己安装我们系统中所需的JDK版本。 ll /usr/local/elasticsearch-7.0.0/jdk/ total 20 drwxr-xr-x 2 root root 4096 Apr 6 06:57 bin drwxr-xr-x 5 root root 123 Apr 6 06:57 conf drwxr-xr-x 3 root root 132 Apr 6 06:57 include drwxr-xr-x 2 root root 4096 Apr 6 06:57 jmods drwxr-xr-x 72 root root 4096 Apr 6 06:57 legal drwxr-xr-x 5 root root 4096 Apr 6 06:57 lib -rw-r--r-- 1 root root 1186 Apr 6 06:57 release /usr/local/elasticsearch-7.0.0/jdk/bin/java -version openjdk version "12" 2019-03-19 OpenJDK Runtime Environment (build 12+33) OpenJDK 64-Bit Server VM (build 12+33, mixed mode, sharing)

    创建elasticsearch用户组 elasticsearch默认是不能以root身份去运行的,否则启动会报错误信息为“can not run elasticsearch as root Elasticsearch”,所以我们要创建一个普通用户来管理elasticsearch,当然这也是elasticsearch为安全着想的一种方式

    groupadd elasticsearch useradd elasticsearch -g elasticsearch echo '3edc#EDC' | passwd --stdin elasticsearch Changing password for user elasticsearch. passwd: all authentication tokens updated successfully. 更改elasticsearch属性 chown -Rf elasticsearch /usr/local/elasticsearch-7.0.0/

    修改文件描述符数量 因为elasticsearch对文件描述符的限制至少为65536,所以我们必须要修改系统的文件描述符数量

    echo '* soft nofile 65536' >> /etc/security/limits.conf echo '* hard nofile 65536' >> /etc/security/limits.conf ulimit -n 65535 ulimit -n如果未生效,请重新登录设备,让用户重载环境变量

    修改max_map_count值 在linux系统上,elasticsearch默认使用hybrid mmapfs / niofs来存储索引文件,因此操作系统主要会通过mmap来限制存储的空间,因此如果存储空间满了,那么会抛出异常,我们可以使用如下命令来更改设置。

    临时设置: sysctl -w vm.max_map_count=655360 vm.max_map_count = 655360 永久设置: echo 'vm.max_map_count=655360' >> /etc/sysctl.conf sysctl -p vm.max_map_count = 655360

    添加内存分配字段 当我们启动ES时,它会报错为无法分配内存字段

    echo '* soft memlock unlimited' >> /etc/security/limits.conf echo '* hard memlock unlimited' >> /etc/security/limits.conf tail -2 /etc/security/limits.conf * soft memlock unlimited * hard memlock unlimited

    elasticsearch7.0配置文件详解 此文件不是集群生成所用到的文件,只是给大家讲解ES7配置文件的详细说明

    vim /usr/local/elasticsearch-7.0.0/config/elasticsearch.yml cluster.name: ES-Cluster #ES集群名称,同一个集群内的所有节点集群名称必须保持一致 node.name: ES-master-10.150.55.94 #ES集群内的节点名称,同一个集群内的节点名称要具备唯一性 node.master: true #允许节点是否可以成为一个master节点,ES是默认集群中的第一台机器成为master,如果这台机器停止就会重新选举 node.data: false #允许该节点存储索引数据(默认开启) #关于Elasticsearch节点的角色功能详解,请看:https://www.dockerc.com/elasticsearch-master-or-data/ path.data: /data/ES-Cluster/master/ES-master-10.150.55.94/data1,/data/ES-Cluster/master/ES-master-10.150.55.94/data2 #ES是搜索引擎,会创建文档,建立索引,此路径是索引的存放目录,如果我们的日志数据较为庞大,那么索引所占用的磁盘空间也是不可小觑的 #这个路径建议是专门的存储系统,如果不是存储系统,最好也要有冗余能力的磁盘,此目录还要对elasticsearch的运行用户有写入权限 #path可以指定多个存储位置,分散存储,有助于性能提升,以至于怎么分散存储请看详解https://www.dockerc.com/elk-theory-elasticsearch/ path.logs: /data/ES-Cluster/master/ES-master-10.150.55.94/logs #elasticsearch专门的日志存储位置,生产环境中建议elasticsearch配置文件与elasticsearch日志分开存储 bootstrap.memory_lock: true #在ES运行起来后锁定ES所能使用的堆内存大小,锁定内存大小一般为可用内存的一半左右;锁定内存后就不会使用交换分区 #如果不打开此项,当系统物理内存空间不足,ES将使用交换分区,ES如果使用交换分区,那么ES的性能将会变得很差 network.host: 10.150.55.94 #es绑定地址,支持IPv4及IPv6,默认绑定127.0.0.1;es的HTTP端口和集群通信端口就会监听在此地址上 network.tcp.no_delay: true #是否启用tcp无延迟,true为启用tcp不延迟,默认为false启用tcp延迟 network.tcp.keep_alive: true #是否启用TCP保持活动状态,默认为true network.tcp.reuse_address: true #是否应该重复使用地址。默认true,在Windows机器上默认为false network.tcp.send_buffer_size: 128mb #tcp发送缓冲区大小,默认不设置 network.tcp.receive_buffer_size: 128mb #tcp接收缓冲区大小,默认不设置 transport.tcp.port: 9301 #设置集群节点通信的TCP端口,默认就是9300 transport.tcp.compress: true #设置是否压缩TCP传输时的数据,默认为false http.max_content_length: 200mb #设置http请求内容的最大容量,默认是100mb http.cors.enabled: true #是否开启跨域访问 http.cors.allow-origin: "*" #开启跨域访问后的地址限制,*表示无限制 http.port: 9201 #定义ES对外调用的http端口,默认是9200 discovery.zen.ping.unicast.hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] #在Elasticsearch7.0版本已被移除,配置错误 #写入候选主节点的设备地址,来开启服务时就可以被选为主节点 #默认主机列表只有127.0.0.1和IPV6的本机回环地址 #上面是书写格式,discover意思为发现,zen是判定集群成员的协议,unicast是单播的意思,ES5.0版本之后只支持单播的方式来进行集群间的通信,hosts为主机 #总结下来就是:使用zen协议通过单播方式去发现集群成员主机,在此建议将所有成员的节点名称都写进来,这样就不用仅靠集群名称cluster.name来判别集群关系了 discovery.zen.minimum_master_nodes: 2 #在Elasticsearch7.0版本已被移除,配置无效 #为了避免脑裂,集群的最少节点数量为,集群的总节点数量除以2加一 discovery.zen.fd.ping_timeout: 120s #在Elasticsearch7.0版本已被移除,配置无效 #探测超时时间,默认是3秒,我们这里填120秒是为了防止网络不好的时候ES集群发生脑裂现象 discovery.zen.fd.ping_retries: 6 #在Elasticsearch7.0版本已被移除,配置无效 #探测次数,如果每次探测90秒,连续探测超过六次,则认为节点该节点已脱离集群,默认为3次 discovery.zen.fd.ping_interval: 15s #在Elasticsearch7.0版本已被移除,配置无效 #节点每隔15秒向master发送一次心跳,证明自己和master还存活,默认为1秒太频繁, discovery.seed_hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] #Elasticsearch7新增参数,写入候选主节点的设备地址,来开启服务时就可以被选为主节点,由discovery.zen.ping.unicast.hosts:参数改变而来 cluster.initial_master_nodes: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] #Elasticsearch7新增参数,写入候选主节点的设备地址,来开启服务时就可以被选为主节点 cluster.fault_detection.leader_check.interval: 15s #Elasticsearch7新增参数,设置每个节点在选中的主节点的检查之间等待的时间。默认为1秒 discovery.cluster_formation_warning_timeout: 30s #Elasticsearch7新增参数,启动后30秒内,如果集群未形成,那么将会记录一条警告信息,警告信息未master not fount开始,默认为10秒 cluster.join.timeout: 30s #Elasticsearch7新增参数,节点发送请求加入集群后,在认为请求失败后,再次发送请求的等待时间,默认为60秒 cluster.publish.timeout: 90s #Elasticsearch7新增参数,设置主节点等待每个集群状态完全更新后发布到所有节点的时间,默认为30秒 cluster.routing.allocation.cluster_concurrent_rebalance: 32 #集群内同时启动的数据任务个数,默认是2个 cluster.routing.allocation.node_concurrent_recoveries: 32 #添加或删除节点及负载均衡时并发恢复的线程个数,默认4个 cluster.routing.allocation.node_initial_primaries_recoveries: 32 #初始化数据恢复时,并发恢复线程的个数,默认4个

    注意:关于Elasticsearch7更多配置参数请看官网:https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-discovery-settings.html 关于Elasticsearch6到Elasticsearch7的区别请看:https://www.dockerc.com/elasticsearch7-coordination/

    elasticsearch7.x的jvm.options配置文件详解

    grep -Ev "#|^$" /usr/local/elasticsearch-7.0.0/config/jvm.options | head -5 #堆内存配置 -Xms16g #Xms表示ES堆内存初始大小 -Xmx16g #Xmx表示ES堆内存的最大可用空间 #GC配置 -XX:+UseConcMarkSweepGC #使用CMS内存收集 -XX:CMSInitiatingOccupancyFraction=75 #使用CMS作为垃圾回收使用,75%后开始CMS收集 -XX:+UseCMSInitiatingOccupancyOnly #使用手动定义初始化开始CMS收集

    ES-Master:10.150.55.94的elasticsearch.yml配置文件

    cluster.name: ES-Cluster node.name: ES-master-10.150.55.94 node.master: true node.data: false path.data: /data/ES-Cluster/master/ES-master-10.150.55.94/data1,/data/ES-Cluster/master/ES-master-10.150.55.94/data2 path.logs: /data/ES-Cluster/master/ES-master-10.150.55.94/logs bootstrap.memory_lock: true network.host: 10.150.55.94 network.tcp.no_delay: true network.tcp.keep_alive: true network.tcp.reuse_address: true network.tcp.send_buffer_size: 128mb network.tcp.receive_buffer_size: 128mb transport.tcp.port: 9301 transport.tcp.compress: true http.max_content_length: 200mb http.cors.enabled: true http.cors.allow-origin: "*" http.port: 9201 discovery.seed_hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.initial_master_nodes: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.fault_detection.leader_check.interval: 15s discovery.cluster_formation_warning_timeout: 30s cluster.join.timeout: 120s cluster.publish.timeout: 90s cluster.routing.allocation.cluster_concurrent_rebalance: 32 cluster.routing.allocation.node_concurrent_recoveries: 32 cluster.routing.allocation.node_initial_primaries_recoveries: 32

    ES-Master:10.150.55.94的jvm.options配置文件

    -Xms16g -Xmx16g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly

    创建数据存储路径

    挂载存储: showmount -e 10.150.30.246 Export list for 10.150.30.246: /data 10.150.* mkdir /data mount -t nfs 10.150.30.246:/data /data 创建我们在elasticsearch.yml中指定的数据存储目录 mkdir -pv /data/ES-Cluster/master/ES-master-10.150.55.94/{data1,data2,logs} mkdir: created directory ‘/data/ES-Cluster’ mkdir: created directory ‘/data/ES-Cluster/master’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.55.94’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.55.94/data1’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.55.94/data2’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.55.94/logs’ 设置挂载权限为普通用户可读可写,不然启动ES时会报错无权限 chmod -Rf 777 /data

    注:设置完成后,暂时不要启动ES,先配置其它节点

    ES-Master:10.150.55.95

    以下四步均和10.150.55.94保持一致创建elasticsearch用户组修改文件描述符数量修改max_map_count值添加内存分配字段

    修改elasticsearch.yml文件

    vim /usr/local/elasticsearch-7.0.0/config/elasticsearch.yml cluster.name: ES-Cluster node.name: ES-master-10.150.55.95 #修改节点名称,在集群中具有唯一性 node.master: true node.data: false path.data: /data/ES-Cluster/master/ES-master-10.150.55.95/data1,/data/ES-Cluster/master/ES-master-10.150.55.95/data2 #修改数据存储目录位置,我这里为了好区分,用了IP地址来分别各个设备的数据节点 path.logs: /data/ES-Cluster/master/ES-master-10.150.55.95/logs #修改日志存储位置 bootstrap.memory_lock: true network.host: 10.150.55.95 #修改ES绑定地址 network.tcp.no_delay: true network.tcp.keep_alive: true network.tcp.reuse_address: true network.tcp.send_buffer_size: 128mb network.tcp.receive_buffer_size: 128mb transport.tcp.port: 9301 transport.tcp.compress: true http.max_content_length: 200mb http.cors.enabled: true http.cors.allow-origin: "*" http.port: 9201 discovery.seed_hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.initial_master_nodes: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.fault_detection.leader_check.interval: 15s discovery.cluster_formation_warning_timeout: 30s cluster.join.timeout: 120s cluster.publish.timeout: 90s cluster.routing.allocation.cluster_concurrent_rebalance: 32 cluster.routing.allocation.node_concurrent_recoveries: 32 cluster.routing.allocation.node_initial_primaries_recoveries: 32

    修改jvm.options文件

    grep -Ev "#|^$" /usr/local/elasticsearch-7.0.0/config/jvm.options | head -5 -Xms16g -Xmx16g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly

    创建数据存储路径

    挂载存储: showmount -e 10.150.30.246 Export list for 10.150.30.246: /data 10.150.* mkdir /data mount -t nfs 10.150.30.246:/data /data 创建我们在elasticsearch.yml中指定的数据存储目录 mkdir -pv /data/ES-Cluster/master/ES-master-10.150.55.95/{data1,data2,logs} mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.55.95’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.55.95/data1’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.55.95/data2’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.55.95/logs 设置挂载权限为普通用户可读可写,不然启动ES时会报错无权限 chmod -Rf 777 /data

    ES-Master:10.150.30.246

    以下四步均和10.150.55.94保持一致创建elasticsearch用户组修改文件描述符数量修改max_map_count值添加内存分配字段

    修改elasticsearch.yml文件

    vim /usr/local/elasticsearch-7.0.0/config/elasticsearch.yml cluster.name: ES-Cluster node.name: ES-master-10.150.30.246 node.master: true node.data: false path.data: /data/ES-Cluster/master/ES-master-10.150.30.246/data1,/data/ES-Cluster/master/ES-master-10.150.30.246/data2 path.logs: /data/ES-Cluster/master/ES-master-10.150.30.246/logs bootstrap.memory_lock: true network.host: 10.150.30.246 network.tcp.no_delay: true network.tcp.keep_alive: true network.tcp.reuse_address: true network.tcp.send_buffer_size: 128mb network.tcp.receive_buffer_size: 128mb transport.tcp.port: 9301 transport.tcp.compress: true http.max_content_length: 200mb http.cors.enabled: true http.cors.allow-origin: "*" http.port: 9201 discovery.seed_hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.initial_master_nodes: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.fault_detection.leader_check.interval: 15s discovery.cluster_formation_warning_timeout: 30s cluster.join.timeout: 120s cluster.publish.timeout: 90s cluster.routing.allocation.cluster_concurrent_rebalance: 32 cluster.routing.allocation.node_concurrent_recoveries: 32 cluster.routing.allocation.node_initial_primaries_recoveries: 32

    修改jvm.options文件

    grep -Ev "#|^$" /usr/local/elasticsearch-7.0.0/config/jvm.options | head -5 -Xms8g -Xmx8g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly

    创建数据存储路径

    挂载存储: 因为我们本机就是存储服务器,所以不需要挂载存储 创建我们在elasticsearch.yml中指定的数据存储目录 mkdir -pv /data/ES-Cluster/master/ES-master-10.150.30.246/{data1,data2,logs} mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.30.246’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.30.246/data1’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.30.246/data2’ mkdir: created directory ‘/data/ES-Cluster/master/ES-master-10.150.30.246/logs’ 设置挂载权限为普通用户可读可写,不然启动ES时会报错无权限 chmod -Rf 777 /data

    ES-Data:10.150.55.88

    以下四步均和10.150.55.94保持一致创建elasticsearch用户组修改文件描述符数量修改max_map_count值添加内存分配字段

    修改elasticsearch.yml文件

    vim /usr/local/elasticsearch-7.0.0/config/elasticsearch.yml cluster.name: ES-Cluster node.name: ES-Data-10.150.55.88 #修改节点名称,在集群中具有唯一性 node.master: false #因为是数据存储节点,所以我们不给他成为master的机会 node.data: true #开启数据存储 path.data: /data/ES-Cluster/data/ES-data-10.150.55.88/data1,/data/ES-Cluster/data/ES-data-10.150.55.88/data2 #修改数据存储目录位置,我这里为了好区分,用了IP地址来分别各个设备的数据节点 path.logs: /data/ES-Cluster/data/ES-data-10.150.55.88/logs #修改日志存储位置 bootstrap.memory_lock: true network.host: 10.150.55.88 #修改ES绑定地址 network.tcp.no_delay: true network.tcp.keep_alive: true network.tcp.reuse_address: true network.tcp.send_buffer_size: 128mb network.tcp.receive_buffer_size: 128mb transport.tcp.port: 9301 transport.tcp.compress: true http.max_content_length: 200mb http.cors.enabled: true http.cors.allow-origin: "*" http.port: 9201 discovery.seed_hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.initial_master_nodes: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.fault_detection.leader_check.interval: 15s discovery.cluster_formation_warning_timeout: 30s cluster.join.timeout: 120s cluster.publish.timeout: 90s cluster.routing.allocation.cluster_concurrent_rebalance: 32 cluster.routing.allocation.node_concurrent_recoveries: 32 cluster.routing.allocation.node_initial_primaries_recoveries: 32

    修改jvm.options

    grep -Ev "#|^$" /usr/local/elasticsearch-7.0.0/config/jvm.options | head -5 -Xms8g -Xmx8g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly

    创建数据存储路径

    挂载存储: showmount -e 10.150.30.246 Export list for 10.150.30.246: /data 10.150.* mkdir /data mount -t nfs 10.150.30.246:/data /data 创建我们在elasticsearch.yml中指定的数据存储目录 mkdir -pv /data/ES-Cluster/data/ES-data-10.150.55.88/{data1,data2,logs} mkdir: created directory ‘/data/ES-Cluster/data’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.55.88’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.55.88/data1’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.55.88/data2’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.55.88/logs’ 设置挂载权限为普通用户可读可写,不然启动ES时会报错无权限 chmod -Rf 777 /data

    ES-Data:10.150.55.89

    以下四步均和10.150.55.94保持一致创建elasticsearch用户组修改文件描述符数量修改max_map_count值添加内存分配字段

    修改elasticsearch.yml文件

    vim /usr/local/elasticsearch-7.0.0/config/elasticsearch.yml cluster.name: ES-Cluster node.name: ES-Data-10.150.55.89 node.master: false node.data: true path.data: /data/ES-Cluster/data/ES-data-10.150.55.89/data1,/data/ES-Cluster/data/ES-data-10.150.55.89/data2 path.logs: /data/ES-Cluster/data/ES-data-10.150.55.89/logs bootstrap.memory_lock: true network.host: 10.150.55.89 network.tcp.no_delay: true network.tcp.keep_alive: true network.tcp.reuse_address: true network.tcp.send_buffer_size: 128mb network.tcp.receive_buffer_size: 128mb transport.tcp.port: 9301 transport.tcp.compress: true http.max_content_length: 200mb http.cors.enabled: true http.cors.allow-origin: "*" http.port: 9201 discovery.seed_hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.initial_master_nodes: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.fault_detection.leader_check.interval: 15s discovery.cluster_formation_warning_timeout: 30s cluster.join.timeout: 120s cluster.publish.timeout: 90s cluster.routing.allocation.cluster_concurrent_rebalance: 32 cluster.routing.allocation.node_concurrent_recoveries: 32 cluster.routing.allocation.node_initial_primaries_recoveries: 32

    修改jvm.options文件

    grep -Ev "#|^$" /usr/local/elasticsearch-7.0.0/config/jvm.options | head -5 -Xms8g -Xmx8g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly

    创建数据存储路径

    showmount -e 10.150.30.246 Export list for 10.150.30.246: /data 10.150.* mkdir /data mount -t nfs 10.150.30.246:/data /data mkdir -pv /data/ES-Cluster/data/ES-data-10.150.55.89/{data1,data2,logs} mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.55.89’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.55.89/data1’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.55.89/data2’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.55.89/logs’ 设置挂载权限为普通用户可读可写,不然启动ES时会报错无权限 chmod -Rf 777 /data

    ES-Data:10.150.30.241

    以下四步均和10.150.55.94保持一致创建elasticsearch用户组修改文件描述符数量修改max_map_count值添加内存分配字段

    修改elasticsearch.yml文件

    vim /usr/local/elasticsearch-7.0.0/config/elasticsearch.yml cluster.name: ES-Cluster node.name: ES-Data-10.150.30.241 node.master: false node.data: true path.data: /data/ES-Cluster/data/ES-data-10.150.30.241/data1,/data/ES-Cluster/data/ES-data-10.150.30.241/data2 path.logs: /data/ES-Cluster/data/ES-data-10.150.30.241/logs bootstrap.memory_lock: true network.host: 10.150.30.241 network.tcp.no_delay: true network.tcp.keep_alive: true network.tcp.reuse_address: true network.tcp.send_buffer_size: 128mb network.tcp.receive_buffer_size: 128mb transport.tcp.port: 9301 transport.tcp.compress: true http.max_content_length: 200mb http.cors.enabled: true http.cors.allow-origin: "*" http.port: 9201 discovery.seed_hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.initial_master_nodes: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.fault_detection.leader_check.interval: 15s discovery.cluster_formation_warning_timeout: 30s cluster.join.timeout: 120s cluster.publish.timeout: 90s cluster.routing.allocation.cluster_concurrent_rebalance: 32 cluster.routing.allocation.node_concurrent_recoveries: 32 cluster.routing.allocation.node_initial_primaries_recoveries: 32

    修改jvm.options文件

    grep -Ev "#|^$" /usr/local/elasticsearch-7.0.0/config/jvm.options | head -5 -Xms8g -Xmx8g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly

    创建数据存储路径

    showmount -e 10.150.30.246 Export list for 10.150.30.246: /data 10.150.* mkdir /data mount -t nfs 10.150.30.246:/data /data mkdir -pv /data/ES-Cluster/data/ES-data-10.150.30.241/{data1,data2,logs} mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.30.241’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.30.241/data1’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.30.241/data2’ mkdir: created directory ‘/data/ES-Cluster/data/ES-data-10.150.30.241/logs’ 设置挂载权限为普通用户可读可写,不然启动ES时会报错无权限 chmod -Rf 777 /data

    ES-Client:10.150.30.242

    以下四步均和10.150.55.94保持一致创建elasticsearch用户组修改文件描述符数量修改max_map_count值添加内存分配字段

    修改elasticsearch.yml文件

    vim /usr/local/elasticsearch-7.0.0/config/elasticsearch.yml cluster.name: ES-Cluster node.name: ES-Client-10.150.30.242 #修改节点名称,在集群中具有唯一性 node.master: false #因为该节点的功能为client,所以不给予成为master的机会 node.data: false #该节点为client,不存储数据 path.data: /data/ES-Cluster/client/ES-client-10.150.30.242/data1,/data/ES-Cluster/client/ES-client-10.150.30.242/data2 path.logs: /data/ES-Cluster/client/ES-client-10.150.30.242/logs bootstrap.memory_lock: true network.host: 10.150.30.242 network.tcp.no_delay: true network.tcp.keep_alive: true network.tcp.reuse_address: true network.tcp.send_buffer_size: 128mb network.tcp.receive_buffer_size: 128mb transport.tcp.port: 9301 transport.tcp.compress: true http.max_content_length: 200mb http.cors.enabled: true http.cors.allow-origin: "*" http.port: 9201 discovery.seed_hosts: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.initial_master_nodes: ["10.150.55.94:9301", "10.150.55.95:9301","10.150.30.246:9301"] cluster.fault_detection.leader_check.interval: 15s discovery.cluster_formation_warning_timeout: 30s cluster.join.timeout: 120s cluster.publish.timeout: 90s cluster.routing.allocation.cluster_concurrent_rebalance: 32 cluster.routing.allocation.node_concurrent_recoveries: 32 cluster.routing.allocation.node_initial_primaries_recoveries: 32

    修改jvm.options文件

    grep -Ev "#|^$" /usr/local/elasticsearch-7.0.0/config/jvm.options | head -5 -Xms8g -Xmx8g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly

    创建数据存储路径

    showmount -e 10.150.30.246 Export list for 10.150.30.246: /data 10.150.* mkdir /data mount -t nfs 10.150.30.246:/data /data mkdir -pv /data/ES-Cluster/client/ES-client-10.150.30.242/{data1,data2,logs} mkdir: created directory ‘/data/ES-Cluster/client’ mkdir: created directory ‘/data/ES-Cluster/client/ES-client-10.150.30.242’ mkdir: created directory ‘/data/ES-Cluster/client/ES-client-10.150.30.242/data1’ mkdir: created directory ‘/data/ES-Cluster/client/ES-client-10.150.30.242/data2’ mkdir: created directory ‘/data/ES-Cluster/client/ES-client-10.150.30.242/logs’ 设置挂载权限为普通用户可读可写,不然启动ES时会报错无权限 chmod -Rf 777 /data

    ES集群启动

    我们一次性启动6台ES节点,等待他们组成ES集群,

    六台节点启动命令一样 su elasticsearch /usr/local/elasticsearch-7.0.0/bin/elasticsearch 上面这条命令默认是前台启动,在生产中,我们一定要放到后台启动,使用nohup command &

    查看集群状态

    使用curl命令来查看ES集群的节点信息

    curl查看ES集群支持的选项

    curl -XGET 'http://10.150.55.94:9201/_cat' =^.^= /_cat/allocation /_cat/shards /_cat/shards/{index} /_cat/master /_cat/nodes /_cat/tasks /_cat/indices /_cat/indices/{index} /_cat/segments /_cat/segments/{index} /_cat/count /_cat/count/{index} /_cat/recovery /_cat/recovery/{index} /_cat/health /_cat/pending_tasks /_cat/aliases /_cat/aliases/{alias} /_cat/thread_pool /_cat/thread_pool/{thread_pools} /_cat/plugins /_cat/fielddata /_cat/fielddata/{fields} /_cat/nodeattrs /_cat/repositories /_cat/snapshots/{repository} /_cat/templates

    查看ES节点信息

    curl -XGET 'http://10.150.55.94:9201/_cat/nodes?v' #命令添加?v是人性化显示,下面带星号的是我们的master节点 ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name 10.150.55.89 6 73 0 0.00 0.04 0.05 di - ES-Data-10.150.55.89 10.150.55.95 3 62 0 0.03 0.14 0.12 mdi * ES-master-10.150.55.95 10.150.30.242 6 71 0 0.03 0.06 0.09 i - ES-Client-10.150.30.242 10.150.55.94 5 65 0 0.00 0.02 0.05 mi - ES-master-10.150.55.94 10.150.55.88 6 73 0 0.00 0.04 0.10 di - ES-Data-10.150.55.88 10.150.30.241 6 70 0 0.00 0.02 0.05 di - ES-Data-10.150.30.241 10.150.30.246 9 70 0 0.00 0.04 0.11 mi - ES-master-10.150.30.246

    查看master节点信息

    curl -XGET 'http://10.150.55.94:9201/_cat/master?v' id host ip node Z-***Q6UQsyXMm6JPt-mKQ 10.150.55.95 10.150.55.95 ES-master-10.150.55.95

    查看ES集群的健康状态

    curl -XGET 'http://10.150.55.94:9201/_cat/health?v' epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 1558031981 18:39:41 ES-Cluster green 6 3 0 0 0 0 0 0 - 100.0%

    创建文档,生成索引 语法如下:

    curl -H "Content-Type:application/json" -XPUT 'http://10.150.55.94:9201/index_name/type_name/1?pretty' -d ' { "name": "xuwl", "age": 18, "job": "Linux" }' 命令介绍: -H:指定内容类型 -X:指定http请求方式,这里为PUT上传方式 http://10.211.55.10:9201:指定一台es服务器对外的http端口 /index_name:文档的索引名称,必须小写 /type_name:文档的类型名称,必须小写 /1:文档的ID编号 ?pretty:人性化创建索引 -d:指定使用JSON方式来撰写上传文档 { "name": "xuwl", "age": 18, "job": "Linux" }':使用JSON格式来撰写上传文档内容

    创建索引为index_name文档

     curl -H "Content-Type:application/json" -XPOST 'http://10.150.55.94:9201/index_name/index_type/1?pretty' -d ' > { "name": "es-clutser", "type": "data_index" } ' { "_index" : "index_name", "_type" : "index_type", "_id" : "1", "_version" : 1, "result" : "created", "_shards" : { "total" : 2, "successful" : 1, "failed" : 0 }, "_seq_no" : 0, "_primary_term" : 1 }

    查看索引

    [root@sx-sj-movie-yqwd-5 ES-Cluster]# curl -XGET 'http://10.150.55.94:9201/_cat/indices?v' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open migu bOnIzhTsQmOPTR5nsaVmBw 1 1 1 0 7.8kb 3.9kb green open xuweiliang JK_qzwOfReCCgp-IpXhWsA 1 1 1 0 8.1kb 4kb green open index_name krb465ShROy_iAk2pOd4IA 1 1 1 0 8.3kb 4.1kb green open wd PibYiRMSQOWPMU2pF_IuFQ 1 1 1 0 8kb 4kb green open pudong U9-lk1piQ-m85Vb3lmZFDQ 1 1 1 0 7.9kb 3.9kb

    查看分片 可以看到我们分片存储的对应的地址,没有出现我们的master节点地址,也没有出现我们的client节点,仅仅只有我们所设置的数据存储节点, 额,下面还有10.150.55.95是我们的master节点的原因,是因为我把node.data设置为了true..... 我错了,大家要相信我,只要把node.data设置为false它肯定是不会存储数据的。。。

    [root@sx-sj-movie-yqwd-5 ES-Cluster]# curl -XGET 'http://10.150.55.94:9201/_cat/shards?v' index shard prirep state docs store ip node migu 0 p STARTED 1 3.9kb 10.150.55.95 ES-master-10.150.55.95 migu 0 r STARTED 1 3.9kb 10.150.55.89 ES-Data-10.150.55.89 index_name 0 p STARTED 1 4.1kb 10.150.55.95 ES-master-10.150.55.95 index_name 0 r STARTED 1 4.1kb 10.150.55.89 ES-Data-10.150.55.89 pudong 0 p STARTED 1 3.9kb 10.150.55.95 ES-master-10.150.55.95 pudong 0 r STARTED 1 3.9kb 10.150.55.89 ES-Data-10.150.55.89 wd 0 p STARTED 1 4kb 10.150.30.241 ES-Data-10.150.30.241 wd 0 r STARTED 1 4kb 10.150.55.88 ES-Data-10.150.55.88 xuweiliang 0 p STARTED 1 4kb 10.150.30.241 ES-Data-10.150.30.241 xuweiliang 0 r STARTED 1 4kb 10.150.55.88 ES-Data-10.150.55.88

     

    3、安装elasticsearch-head

    安装elasticsearch-head插件需要nodejs的支持,到nodejs官网现在最新nodejs,官网下载地址:https://nodejs.org/en/download/。

    安装好nodejs后,执行以下命令安装elasticsearch-head

    git clone git://github.com/mobz/elasticsearch-head.gitcd elasticsearch-headnpm installnpm run start

    在elasticsearch配置中添加http.cors.enabled:true您还必须设置http.cors.allow-origin,因为默认情况下不允许原点。 http.cors.allow-origin:"*"是有效值,但它被视为安全风险,因为您的群集可以从任何地方跨越原点。

    默认情况下,elasticsearch会在elasticsearch-head连接到的端口9200上公开http rest API。

    使用浏览器打开 http://localhost:9100/,输入连接地址http://127.0.0.1:9200/,见下图。

    还有一种比较简单的方式就是安装Chrome插件,插件地址https://chrome.google.com/webstore/detail/elasticsearch-head/ffmkiejjmecolpfloofpjologoblkegm/,安装完成后点击工具栏图标即可。

    最新回复(0)