Bug 1162367
| Summary: | If HCAT / WEBHCAT service is enabled certain jar and tar files need to be copied to the volume mount | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Jeff Vance <jvance> |
| Component: | rhs-hadoop-install | Assignee: | Jeff Vance <jvance> |
| Status: | CLOSED ERRATA | QA Contact: | Daniel Horák <dahorak> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rhgs-3.0 | CC: | bchilds, dahorak, eboyd, matt, mkudlej, nlevinki, rcyriac |
| Target Milestone: | --- | Keywords: | ZStream |
| Target Release: | RHGS 3.0.4 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | 2.45-1 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-03-31 10:18:24 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Jeff Vance
2014-11-10 23:07:32 UTC
enable_vol.sh now copies the reqired hcat related file to the volume mount directory. Tested and VERIFIED on Red Hat Storage Server 3.0 Update 4 with
rhs-hadoop-install-2_44-1.el6rhs.noarch.
# /usr/share/rhs-hadoop-install/enable_vol.sh -y --debug --yarn-master MASTER_NODE1 --hadoop-mgmt-node MASTER_NODE1 --rhs-node MASTER_NODE1 --user admin --pass admin --port 8080 --make-default HadoopVol1
***
*** enable_vol: version 2.44
***
DEBUG: unique nodes spanned by HadoopVol1: NODE2
MASTER_NODE1
DEBUG: passwordless ssh check for "NODE2" passed
DEBUG: HadoopVol1 mount point is /mnt/glusterfs/HadoopVol1
DEBUG: Cluster name: TestCluster
DEBUG: Default volume: gv0
*** Volume : HadoopVol1 (will become the DEFAULT volume)
*** Current default vol: gv0
*** Cluster name : TestCluster
*** Nodes : NODE2, MASTER_NODE1
*** Volume mount : /mnt/glusterfs/HadoopVol1
*** Ambari mgmt node : MASTER_NODE1
*** Yarn-master server : MASTER_NODE1
<<truncated>>
--- copying webhcat related jar and tar files as needed...
DEBUG: webhcat service node is MASTER_NODE1
DEBUG: on node MASTER_NODE1: copied jar and/or tar files with 0 warnings: cp /usr/lib/hadoop-mapreduce/hadoop-streaming-2.4.0.2.1.7.0-784.jar /mnt/glusterfs/HadoopVol1/apps/webhcat
cp /usr/share/HDP-webhcat/pig.tar.gz /mnt/glusterfs/HadoopVol1/apps/webhcat
cp /usr/share/HDP-webhcat/hive.tar.gz /mnt/glusterfs/HadoopVol1/apps/webhcat
chmod -R 0755 /mnt/glusterfs/HadoopVol1/apps/webhcat && chown -R hcat:hadoop /mnt/glusterfs/HadoopVol1/apps/webhcat
--- copied webhcat related jar and tar files as needed
<<truncated>>
HadoopVol1 enabled for hadoop workloads with no errors
Required tar and jar files are properly copied to the Gluster Hadoop volume.
>> VERIFIED
Sorry for moving this BZ to VERIFIED, I overlook some case where it doesn't work.
I'll check it and send more details.
>> ASSIGNED
It seems like it doesn't work correctly for scenario where MASTER server is not glusterfs node. In that case, HadoopVolume is not mounted in the time when it try to copy webhcat required files, so the directory /mnt/glusterfs/HadoopVol2/apps/webhcat is not present (it is mounted after this step). Related part from rhs-hadoop-install.log: [15:40:08] DEBUG: webhcat service node is MASTER [15:40:09] DEBUG: /mnt/glusterfs/HadoopVol1/apps/webhcat target dir missing on MASTER, cannot copy post-processing jar and tar files [15:40:09] --- setting up the yarn-master: MASTER... [15:40:10] DEBUG: setup_yarn on MASTER:\nHadoopVol1 glusterfs-fuse mounted on MASTER (yarn-master) HadoopVol1 is setup on MASTER (yarn-master) This should be fixed in version 2.45 Tested and VERIFIED on Red Hat Storage Server 3.0 Update 4 with
rhs-hadoop-install-2_47-1.el6rhs.noarch.
Required tar and jar files are properly copied to the Gluster Hadoop volume in all three supported configurations (master and management on separated servers, master and management on the same server and master and management on storage node).
>> VERIFIED
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2015-0761.html |