安装CDH出现file optclouderaparcels.floodCDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel...does not exist

    xiaoxiao2022-07-02  142

    使用Cloudera Manager采取parcel方式安装CDH的时候,部分agent节点出现异常:

    Src file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel does not exist

    查看对应agent的日志文件查看异常信息:

    [root@cdh-01 parcel-cache]# tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log

    发现日志里有大量的如下输出:  

    20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader INFO Fetching torrent: http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.torrent [20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader INFO Starting download of: http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader INFO Failed adding torrent: file:///opt/cloudera/parcel-cache/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.torrent Already present torrent: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader INFO Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=2087665645 totalSize=2087665645 upload=322898691 state=seeding seed=['http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel'] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=1000000] [20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader INFO Completed download of http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel code=200 state=downloaded [20/May/2019 21:32:17 +0000] 8949 Thread-13 parcel_cache WARNING No checksum in header, skipping verification [20/May/2019 21:32:17 +0000] 8949 Thread-13 parcel_cache INFO Unpacking /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel into /opt/cloudera/parcels [20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader ERROR Failed op: Src file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel does not exist

    也就是提示Src file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel不存在,但是parcel应该从server节点下载下来了才对,为什么提示不存在?想不通,于是停止了该agent服务:

    [root@cdh-01 parcel-cache]# systemctl stop cloudera-scm-agent

    然后清理该agent节点上的parcel缓存(目录默认在 /opt/cloudera/):

    然后重启该agent服务,跟踪完整的日志输出,以确认出现问题的地方(不用重启server节点 ,agent节点重启后会主动向server节点通信,然后开始安装的流程):

    [root@cdh-01 parcel-cache]# systemctl start cloudera-scm-agent [root@cdh-01 parcel-cache]# tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log

    启动成功之后首先是Fetching torrent,然后开始下载:

    [20/May/2019 19:51:18 +0000] 8949 Thread-13 downloader INFO Fetching torrent: http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.torrent [20/May/2019 19:51:19 +0000] 8949 Thread-13 downloader INFO Starting download of: http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel ...... [20/May/2019 19:51:31 +0000] 8949 Thread-13 downloader INFO Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=94896207 totalSize=2087665645 upload=16947965 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=45455] [20/May/2019 19:51:35 +0000] 8949 Thread-13 downloader INFO Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=95420495 totalSize=2087665645 upload=17018436 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=45706] [20/May/2019 19:51:37 +0000] 8949 Thread-13 downloader INFO Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=101712030 totalSize=2087665645 upload=17018436 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=48720] [20/May/2019 19:51:39 +0000] 8949 Thread-13 downloader INFO Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=115343360 totalSize=2087665645 upload=18874368 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=55249] [20/May/2019 19:51:41 +0000] 8949 Thread-13 downloader INFO Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=116916224 totalSize=2087665645 upload=23068672 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=56003] ......

    下载成功之后,开始解压:

    [20/May/2019 20:07:23 +0000] 8949 Thread-13 parcel_cache INFO Unpacking /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel into /opt/cloudera/parcels

    这些流程都是正常进行的,下载也是成功了的,但是发现在解压一段时间之后就抛出了异常:  

    [20/May/2019 20:41:43 +0000] 8949 Thread-13 throttling_logger ERROR Error while unpacking /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel Traceback (most recent call last): File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/parcel_cache.py", line 194, in cb unpack_parcel(op.path, unpack_dir, self.path) File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/parcel_cache.py", line 220, in unpack_parcel raise Exception("Untar failed with return code: %d, with tar output:\n stdout: [%s],\n stderr: [%s]" % (return_code, output[0], output[1])) Exception: Untar failed with return code: 2, with tar output: stdout: [], stderr: [tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-release/kudu-tserver: Wrote only 2048 of 10240 bytes tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-release/kudu-master: Cannot open: No space left on device tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug: Cannot mkdir: No space left on device tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug: Cannot mkdir: No space left on device tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug/kudu-tserver: Cannot open: No such file or directory tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug: Cannot mkdir: No space left on device tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug/kudu-master: Cannot open: No such file or directory ......

    提示磁盘空间不足o(╯□╰)o。通过df命令查看,果然磁盘空间不够了,最后通过清理磁盘解决。

    注:由于parcel文件较大(将近两个G),agent从server下载下来后还需要解压,然后再激活,所以我们要保证充足的磁盘空间,特别是用虚拟机搭建集群的同学。

    最新回复(0)