Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: After the Ambari installation steps, if the user selected HIVE-HCAT-WEBHCAT services, then a jar file and 2 tar.gz files need to be copied to the volume mount directory, eg. /mnt/glusterfs/<VolName>/apps/webhcat. The files in questions are: /usr/lib/hadoop-mapreduce/hadoop-streaming-*.jar, /usr/share/HDP-webhcat/pig.tar.gz, and /usr/share/HDP-webhcat/hive.tar.gz. Failure to do the copy results in a failure when starting hcat. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
enable_vol.sh now copies the reqired hcat related file to the volume mount directory.
Tested and VERIFIED on Red Hat Storage Server 3.0 Update 4 with rhs-hadoop-install-2_44-1.el6rhs.noarch. # /usr/share/rhs-hadoop-install/enable_vol.sh -y --debug --yarn-master MASTER_NODE1 --hadoop-mgmt-node MASTER_NODE1 --rhs-node MASTER_NODE1 --user admin --pass admin --port 8080 --make-default HadoopVol1 *** *** enable_vol: version 2.44 *** DEBUG: unique nodes spanned by HadoopVol1: NODE2 MASTER_NODE1 DEBUG: passwordless ssh check for "NODE2" passed DEBUG: HadoopVol1 mount point is /mnt/glusterfs/HadoopVol1 DEBUG: Cluster name: TestCluster DEBUG: Default volume: gv0 *** Volume : HadoopVol1 (will become the DEFAULT volume) *** Current default vol: gv0 *** Cluster name : TestCluster *** Nodes : NODE2, MASTER_NODE1 *** Volume mount : /mnt/glusterfs/HadoopVol1 *** Ambari mgmt node : MASTER_NODE1 *** Yarn-master server : MASTER_NODE1 <<truncated>> --- copying webhcat related jar and tar files as needed... DEBUG: webhcat service node is MASTER_NODE1 DEBUG: on node MASTER_NODE1: copied jar and/or tar files with 0 warnings: cp /usr/lib/hadoop-mapreduce/hadoop-streaming-2.4.0.2.1.7.0-784.jar /mnt/glusterfs/HadoopVol1/apps/webhcat cp /usr/share/HDP-webhcat/pig.tar.gz /mnt/glusterfs/HadoopVol1/apps/webhcat cp /usr/share/HDP-webhcat/hive.tar.gz /mnt/glusterfs/HadoopVol1/apps/webhcat chmod -R 0755 /mnt/glusterfs/HadoopVol1/apps/webhcat && chown -R hcat:hadoop /mnt/glusterfs/HadoopVol1/apps/webhcat --- copied webhcat related jar and tar files as needed <<truncated>> HadoopVol1 enabled for hadoop workloads with no errors Required tar and jar files are properly copied to the Gluster Hadoop volume. >> VERIFIED
Sorry for moving this BZ to VERIFIED, I overlook some case where it doesn't work. I'll check it and send more details. >> ASSIGNED
It seems like it doesn't work correctly for scenario where MASTER server is not glusterfs node. In that case, HadoopVolume is not mounted in the time when it try to copy webhcat required files, so the directory /mnt/glusterfs/HadoopVol2/apps/webhcat is not present (it is mounted after this step). Related part from rhs-hadoop-install.log: [15:40:08] DEBUG: webhcat service node is MASTER [15:40:09] DEBUG: /mnt/glusterfs/HadoopVol1/apps/webhcat target dir missing on MASTER, cannot copy post-processing jar and tar files [15:40:09] --- setting up the yarn-master: MASTER... [15:40:10] DEBUG: setup_yarn on MASTER:\nHadoopVol1 glusterfs-fuse mounted on MASTER (yarn-master) HadoopVol1 is setup on MASTER (yarn-master)
This should be fixed in version 2.45
Tested and VERIFIED on Red Hat Storage Server 3.0 Update 4 with rhs-hadoop-install-2_47-1.el6rhs.noarch. Required tar and jar files are properly copied to the Gluster Hadoop volume in all three supported configurations (master and management on separated servers, master and management on the same server and master and management on storage node). >> VERIFIED
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2015-0761.html