第一步:配置主机名
新机器初始化,挂载好硬盘,配置好hostname,例如 hadoop9
hostname hadoop9 vim /etc/sysconfig/network ---- HOSTNAME=hadoop9 ---- vi /etc/sysctl.conf ---- vm.swappiness = 10 ---- sysctl -p 第二步准备:安装java
mkdir /usr/java cd /usr/java rz #上传jdk包 jdk-8u201-linux-x64.tar.gz tar xzf jdk-8u201-linux-x64.tar.gz vi /etc/profile --- export JAVA_HOME=/usr/java/jdk1.8.0_201 export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export PATH=$PATH:$JAVA_HOME/bin ---- source /etc/profile第三步准备:
echo 0 > /proc/sys/vm/swappiness echo never > /sys/kernel/mm/transparent_hugepage/enabled echo never > /sys/kernel/mm/transparent_hugepage/defrag echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local第四步: 确保master namenode节点上存在合适的parcel包
ll /opt/cloudera/parcel-repo/ CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel.sha manifest.json 第五步:设置hosts, 添加已有和新加的主机
vim /etc/hosts
上传cloudera-manager-el6-cm5.13.0_x86_64.tar.gz文件
yum install lrzsz #如果没有rzsz cd /opt rz然后 上传cloudera-manager-el6-cm5.13.0_x86_64.tar.gz到/opt
cd /opt mkdir /opt/cloudera-manager tar xzf cloudera-manager-el6-cm5.13.0_x86_64.tar.gz -C /opt/cloudera-manager useradd --system --home=/opt/cloudera-manager/cm-5.13.0/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager 然后设置config.ini
vim /opt/cloudera-manager/cm-5.13.0/etc/cloudera-scm-agent/config.ini设置server_host=masternamenode主机(hadoop1)
/opt/cloudera-manager/cm-5.13.0/etc/init.d/cloudera-scm-agent start tail -f /opt/cloudera-manager/cm-5.13.0/log/cloudera-scm-agent/cloudera-scm-agent.log注意:如果遇到启动失败:
/opt/cloudera-manager/cm-5.13.0/etc/init.d/cloudera-scm-agent status cloudera-scm-agent (pid 25066) is running... #解决办法:检查hostname设置和/etc/sysconfig/network是否一致。再检查/etc/hosts文件,不能给127.0.0.1 设置主机名
下面登陆cm 管理页面:
主机-向群集添加新主机:
然后点“继续” 按钮:
打开主机列表
然后点“继续”。进入下一个页面,要等待几分钟。。。然后点“继续”
在等待时,可以查看运行日志,在当前主机上运行命令:
tail -f /opt/cloudera-manager/cm-5.13.0/log/cloudera-scm-agent/cloudera-scm-agent.log
下一步打开检查结果页面,继续点击“继续”。
下一步主机模板页面,保持默认,然后点击”继续“。
最后点击“完成”
添加hive server2时报错:
+ ERROR: HIVE_AUX_JARS_PATH is configured in Cloudera Manager as /etc/hive/auxlib.解决:手动在每个新主机上创建目录 mkdir -p /etc/hive/auxlib