安装环境 一个多节点集群中,正常情况下应该包括以下3种类型的节点
master节点
维护元数据,管理集群各个节点状态,建议3台机器做master
data节点
负责数据的查询和导入的,压力较大,需要比master和client更高配置的机器
client节点
作为任务分发用的,它里面也会存元数据,但是它不会对元数据做任何修改。另 外它还分担了data节点压力,把data节点一次查询结果在client节点做二次汇聚,如果集群中没有client节点,那么二次汇聚的任务将由datanode完成 安装 1.Elasticsearch安装之前是不需要配置ssh互信的,因为在Elasticsearch集群的各个节点需要分别执行命令,进行单独启动,程序包里面没有提供一个统一的启停命令,如果集群节点数比较多的话,需要整理一个启停脚本,并配置互信,要不然每个节点都启动一次,很麻烦。 2.Elasticsearch软件内部实现了分布式,不需要zookeeper软件,索引zookeeper软件也无需再部署 关闭防火墙及selinux
systemctl stop firewalld systemctl disable firewalld setenforce 0 vim /etc/selinux/config SELINUX=disabled3个节点均操作 设置atguigu1为ntp时钟源
yum install ntp* vim /etc/ntp.conf设置其他2个节点配置定时任务,定时把atguigu1主机的时间同步过来
[root@atguigu2 ~]# crontab -e */30 * * * * root /usr/sbin/ntpdate 192.168.15.46修改系统参数
vim /etc/sysctl.conf vm.max_map_count = 655360 sysctl -psysctl.conf 添加一行,并使之生效,其他2个节点也需要同样处理
Ulimit限制 如下2个文件,3个节点均需要修改
# vim /etc/security/limits.conf es hard nofile 65535 es soft nofile 65535 * soft memlock unlimited * hard memlock unlimited软件包下载 去相应的官网下载jdk1.8和elasticsearch-7.1.0,然后进行解压安装把解压好的jdk和es目录scp到其他主机上:
1.安装和配置JDK 2.三台机器做ssh互信 3.jvm.options文件修改 4.elasticsearch.yml
修改/opt/module/elasticsearch-7.1.0/config/jvm.options的ESheapsize大小,最大值和最小值保持一致,三个节点均修改:
# Xms represents the initial size of total heap space # Xmx represents the maximum size of total heap space --------------------------------------------------------- -Xms8g -Xmx8g修改/opt/module/elasticsearch-7.1.0/config/elasticsearch.yml(三个节点都要修改)
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: Bgelktest # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: atguigu1 # # Add custom attributes to the node: # node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /opt/module/elasticsearch-7.1.0/data # # Path to log files: # path.logs: /opt/module/elasticsearch-7.1.0/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: false # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 192.168.15.46 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when this node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.seed_hosts: ["atguigu1", "atguigu2"] # # Bootstrap the cluster using an initial set of master-eligible nodes: # cluster.initial_master_nodes: ["atguigu1", "atguigu2"] # # For more information, consult the discovery and cluster formation module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true参数解析 启停集群 1.因elasticsearch版本在6.0之后不允许用root用户启动,故必须创建新的用户
1.1 创建用户组 groupadd es 1.2 创建用户 useradd es -g es -p elasticsearch 1.3 更改ELK所属用户 chown -R es:es elasticsearch-7.1.0 1.4 启动集群(三个节点都要启动) #su - es $cd /opt/module/elasticsearch-7.1.0/ $./bin/elasticsearch -d #后台启动es关闭集群 ES没有提供一个命令去关闭集群,所以需要手工执行kill -9 pid 暴力杀掉进程
三个节点分布执行,如下命令: [es@atguigu3 ~]$ jps 24899 Elasticsearch 25028 Jps