CDH部署的机器是三台阿里云机器,配置都是2cpu,8G内存,40G硬盘
安装需要的软件包:
1.CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel(CDH软件包)
2.CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1
3.cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz
4.manifest.json
5.mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz(jdk包)
6.mysql-connector-java-5.1.47.jar
云主机的防火墙一般都是关闭的,阿里云有web界面的防火墙配置,可以自行修改端口访问限制;
如果是公司内部的服务器,一般把防火墙关闭再部署,部署完之后,通过CDH页面提供的端口,再将防火墙开启,再将这些端口设置通过.
#如果不确认是否关闭,自己手动关一下,3台都需要 [root@hadoop001 cdh5.16.1] systemctl stop firewalld [root@hadoop001 cdh5.16.1] systemctl disable firewalld [root@hadoop001 cdh5.16.1] iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination [root@hadoop001 cdh5.16.1] iptables -F云主机的时区跟时钟同步一般是不需要做同步的,都帮你做好了
非云主机是需要的,可以按照上面的步骤来进行时区跟时钟同步.
**我们在生产上,我们需要把 /usr/sbin/ntpdate hadoop001 时钟同步的这个命令加到crontab中,一小时同步一次
[root@hadoop002 opt]# crontab -l 00 * * * * /usr/sbin/ntpdate hadoop001MySQL离线部署的版本是5.7.11
#1.解压及创建目录 [root@hadoop001 local] tar -xzvf mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz -C /usr/local [root@hadoop001 local] mv mysql-5.7.11-linux-glibc2.5-x86_64 mysql [root@hadoop001 local] mkdir mysql/arch mysql/data mysql/tmp #2.修改my.cnf配置 [root@hadoop001 local] vi /etc/my.cnf #以下配置全部替换 [client] port = 3306 socket = /usr/local/mysql/data/mysql.sock default-character-set=utf8mb4 [mysqld] port = 3306 socket = /usr/local/mysql/data/mysql.sock skip-slave-start skip-external-locking key_buffer_size = 256M sort_buffer_size = 2M read_buffer_size = 2M read_rnd_buffer_size = 4M query_cache_size= 32M max_allowed_packet = 16M myisam_sort_buffer_size=128M tmp_table_size=32M table_open_cache = 512 thread_cache_size = 8 wait_timeout = 86400 interactive_timeout = 86400 max_connections = 600 # Try number of CPU's*2 for thread_concurrency #thread_concurrency = 32 #isolation level and default engine default-storage-engine = INNODB transaction-isolation = READ-COMMITTED server-id = 1739 basedir = /usr/local/mysql datadir = /usr/local/mysql/data pid-file = /usr/local/mysql/data/hostname.pid #open performance schema log-warnings sysdate-is-now binlog_format = ROW log_bin_trust_function_creators=1 log-error = /usr/local/mysql/data/hostname.err log-bin = /usr/local/mysql/arch/mysql-bin expire_logs_days = 7 innodb_write_io_threads=16 relay-log = /usr/local/mysql/relay_log/relay-log relay-log-index = /usr/local/mysql/relay_log/relay-log.index relay_log_info_file= /usr/local/mysql/relay_log/relay-log.info log_slave_updates=1 gtid_mode=OFF enforce_gtid_consistency=OFF # slave slave-parallel-type=LOGICAL_CLOCK slave-parallel-workers=4 master_info_repository=TABLE relay_log_info_repository=TABLE relay_log_recovery=ON #other logs #general_log =1 #general_log_file = /usr/local/mysql/data/general_log.err #slow_query_log=1 #slow_query_log_file=/usr/local/mysql/data/slow_log.err #for replication slave sync_binlog = 500 #for innodb options innodb_data_home_dir = /usr/local/mysql/data/ innodb_data_file_path = ibdata1:1G;ibdata2:1G:autoextend innodb_log_group_home_dir = /usr/local/mysql/arch innodb_log_files_in_group = 4 innodb_log_file_size = 1G innodb_log_buffer_size = 200M #根据生产需要，调整pool size innodb_buffer_pool_size = 2G #innodb_additional_mem_pool_size = 50M #deprecated in 5.6 tmpdir = /usr/local/mysql/tmp innodb_lock_wait_timeout = 1000 #innodb_thread_concurrency = 0 innodb_flush_log_at_trx_commit = 2 innodb_locks_unsafe_for_binlog=1 #innodb io features: add for mysql5.5.8 performance_schema innodb_read_io_threads=4 innodb-write-io-threads=4 innodb-io-capacity=200 #purge threads change default(0) to 1 for purge innodb_purge_threads=1 innodb_use_native_aio=on #case-sensitive file names and separate tablespace innodb_file_per_table = 1 lower_case_table_names=1 [mysqldump] quick max_allowed_packet = 128M [mysql] no-auto-rehash default-character-set=utf8mb4 [mysqlhotcopy] interactive-timeout [myisamchk] key_buffer_size = 256M sort_buffer_size = 256M read_buffer = 2M write_buffer = 2M #3.创建用户组及用户 [root@hadoop001 local] groupadd -g 101 dba [root@hadoop001 local] useradd -u 514 -g dba -G root -d /usr/local/mysql mysqladmin #4.copy 环境变量配置文件至mysqladmin用户的home目录中,为了以下步骤配置个人环境变量 [root@hadoop001 local] cp /etc/skel/.* /usr/local/mysql #5.配置环境变量 [root@hadoop001 local] vim mysql/.bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs export MYSQL_BASE=/usr/local/mysql export PATH=${MYSQL_BASE}/bin:$PATH unset USERNAME #stty erase ^H set umask to 022 umask 022 PS1=`uname -n`":"'$USER'":"'$PWD'":>"; export PS1 ## end #6.赋权限和用户组，切换用户mysqladmin，安装 #配置文件权限 [root@hadoop001 local] chown mysqladmin:dba /etc/my.cnf [root@hadoop001 local] chmod 640 /etc/my.cnf #mysql目录权限 [root@hadoop001 local] chown -R mysqladmin:dba /usr/local/mysql [root@hadoop001 local] chmod -R 755 /usr/local/mysql #7.配置服务及开机自启动 [root@hadoop001 local] cd /usr/local/mysql #将服务文件拷贝到init.d下，并重命名为mysql [root@hadoop001 mysql] cp support-files/mysql.server /etc/rc.d/init.d/mysql #赋予可执行权限 [root@hadoop001 mysql] chmod +x /etc/rc.d/init.d/mysql #删除服务 [root@hadoop001 mysql] chkconfig --del mysql #添加服务 [root@hadoop001 mysql] chkconfig --add mysql [root@hadoop001 mysql] chkconfig --level 345 mysql on #centos7 开机自启动 #新建systemctl配置文件 vi /usr/lib/systemd/system/mysql.service [Unit] Description=MySQL Server Documentation=man:mysqld(8) Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html After=network.target After=syslog.target [Install] WantedBy=multi-user.target [Service] User=mysql Group=mysql ExecStart=/usr/local/mysql --defaults-file=/etc/my.cnf LimitNOFILE = 5000 #Restart=on-failure #RestartPreventExitStatus=1 #PrivateTmp=false #编辑/etc/init.d/mysql vi /etc/init.d/mysql，找到start模块，添加--user=mysqladmin 到 $bindir/mysqld_safe #启动命令 service mysql start 8.安装libaio及安装mysql的初始db [root@hadoop001 mysql] yum -y install libaio [root@hadoop001 mysql] sudo su - mysqladmin hadoop001:mysqladmin:/usr/local/mysql:> bin/mysqld \ --defaults-file=/etc/my.cnf \ --user=mysqladmin \ --basedir=/usr/local/mysql/ \ --datadir=/usr/local/mysql/data/ \ --initialize #在初始化时如果加上 –initial-insecure，则会创建空密码的 root@localhost 账号，否则会创建带密码的 root@localhost 账号，密码直接写在 log-error 日志文件中（在5.6版本中是放在 ~/.mysql_secret 文件里，更加隐蔽，不熟悉的话可能会无所适从） #9.查看临时密码 hadoop001:mysqladmin:/usr/local/mysql/data:>cat hostname.err |grep password 2017-07-22T02:15:29.439671Z 1 [Note] A temporary password is generated for root@localhost: kFCqrXeh2y(0 #10.启动 /usr/local/mysql/bin/mysqld_safe --defaults-file=/etc/my.cnf & #11.登录及修改用户密码 hadoop001:mysqladmin:/usr/local/mysql/data:>mysql -uroot -p'kFCqrXeh2y(0' mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.11-log Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> alter user root@localhost identified by '123456'; Query OK, 0 rows affected (0.05 sec) mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' ; Query OK, 0 rows affected, 1 warning (0.02 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec) mysql> exit; Byehttp://hadoop001:7180/ 账号密码:admin/admin
假如本地parcel离线源配置正确，则"下载"阶段瞬间完成，其余阶段视节点数与内部⽹网络情况决定。
