Bug 1166687
| Summary: | disabling default volume leaves it defined as default | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Martin Bukatovic <mbukatov> |
| Component: | rhs-hadoop-install | Assignee: | Jeff Vance <jvance> |
| Status: | CLOSED ERRATA | QA Contact: | Ondřej Komárek <okomarek> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rhgs-3.0 | CC: | bchilds, eboyd, esammons, matt, mkudlej, nlevinki, okomarek, rcyriac |
| Target Milestone: | --- | Keywords: | ZStream |
| Target Release: | RHGS 3.0.4 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | 2.40 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-03-31 10:18:34 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
1) there was a bug in bin/find_default_vol.sh which returned all vols in the fs.glusterfs.volumes prop except for the last vol. It has been fixed to return only rhe first vol in the list of vols. 2) with the correct default vol returned, disable_vol.sh prompts the user with a message that indicates the target vol is the default vol. 3) BZ 1151219 addresses the fixed "delete" action of bin/ambari_config_update.sh. 4) when a vol is disabled it is no longer accessible to Hadoop jobs (this is the point of disable_vol). 5) the design supports disabling the vol even if it is the default vol. The "are you sure" prompt indicates the target vol is the default vol, and the summary action display now indicates if the target vol is the default vol. Tested with: =========== disable_vol: version 2.44 rhs-hadoop-2.3.3-3.el6rhs.noarch hadoop-2.4.0.2.1.7.0-784.el6.x86_64 glusterfs-3.6.0.42.1-1.el6rhs.x86_64 I tried to run disable_vol.sh script on cluster with 2 volumes enabled for hadoop jobs. Prompt informed me, that I am trying to disable default volume and what volume will be new default. Disabling was successful and I was not able to use disabled volume after that. Other volume was marked as default correctly and I was able to run some fs commands and hadoop jobs on it. This issue has been solved -> VERIFIED. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2015-0761.html |
Description of problem ====================== Script disable_vol.sh of rhs-hadoop-install allows user to disable the default volume without marking the volume non default (you end up with default volume, which doesn't exist). This breaks hadoop fs access completely (it doesn't matter that you specify correct name of another working volume in the filepath). Version-Release number of selected component (if applicable) ============================================================ rhs-hadoop-install-2_29-1.el6rhs.noarch rhs-hadoop-2.3.3-3.el6rhs.noarch How reproducible ================ 100 % Steps to Reproduce ================== On cluster with several hadoop enabled RHS volumes: 1. Disable hadoop-enablement on the default volume: ~~~ # ./disable_vol.sh --hadoop-mgmt-node mgmt.example.com --yarn-master master.example.com --rhs-node node01.example.com BrownFieldVolume ~~~ 2. Try to access files on the other volume: ~~~ $ hadoop fs -ls glusterfs://NotBrownFieldVolume/ ~~~ Actual results ============== The script removes default volume without any complaint: ~~~ # ./disable_vol.sh --hadoop-mgmt-node mgmt.example.com --yarn-master master.example.com --rhs-node node01.example.com BrownFieldVolume *** *** disable_vol: version 2.29 *** *** Volume : BrownFieldVolume *** Default volume : BrownFieldVolume,gv0 *** Nodes : 10.16.66.77, 10.16.64.42, 10.16.67.178, 10.16.67.194 *** Ambari mgmt node : mgmt.example.com *** Yarn-master server: master.example.com BrownFieldVolume will be removed from the core-site config file and thus will not be available for any hadoop workloads. Continue? [y|N] y BrownFieldVolume disabled for hadoop workloads ~~~ Which breaks all further hcfs operations: ~~~ $ hadoop fs -ls glusterfs://NotBrownFieldVolume/ 14/11/20 12:42:28 INFO glusterfs.GlusterVolume: Initializing gluster volume.. 14/11/20 12:42:28 INFO glusterfs.GlusterFileSystem: Configuring GlusterFS 14/11/20 12:42:28 INFO glusterfs.GlusterFileSystem: Initializing GlusterFS, CRC disabled. 14/11/20 12:42:28 INFO glusterfs.GlusterFileSystem: GIT INFO={git.commit.id.abbrev=5962b38, git.commit.user. email=bchilds.rdu2.redhat.com, git.commit.message.full=[update RPM spec file/changelog] - 2.3.3 , git.commit.id=5962b3870d16b3a2309fb61b3787d66cf286844a, git.commit.message.short=[update RPM spec file/ changelog] - 2.3.3, git.commit.user.name=Brad Childs, git.build.user.name=Unknown, git.commit.id.describe=2.3. 11.0-9-g5962b38, git.build.user.email=Unknown, git.branch=2.3.3, git.commit.time=30.10.2014 @ 10:12:26 EDT, git.build.time=30.10.2014 @ 10:21:08 EDT} 14/11/20 12:42:28 INFO glusterfs.GlusterFileSystem: GIT_TAG=2.3.11.0 14/11/20 12:42:28 INFO glusterfs.GlusterFileSystem: Configuring GlusterFS 14/11/20 12:42:28 INFO glusterfs.GlusterVolume: Initializing gluster volume.. 14/11/20 12:42:28 ERROR glusterfs.GlusterVolume: Invalid volume name: BrownFieldVolume, No such property: fs. glusterfs.fuse.BrownFieldVolume -ls: Fatal internal error java.lang.RuntimeException: java.lang.RuntimeException: Invalid volume name: BrownFieldVolume, No mount point available for the volume. at org.apache.hadoop.fs.glusterfs.GlusterVolume.setConf(GlusterVolume.java:188) at org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:92) at org.apache.hadoop.fs.glusterfs.GlusterVolume.initialize(GlusterVolume.java:88) at org.apache.hadoop.fs.FilterFileSystem.initialize(FilterFileSystem.java:90) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2397) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2431) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2413) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:325) at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:224) at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:207) at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190) at org.apache.hadoop.fs.shell.Command.run(Command.java:154) at org.apache.hadoop.fs.FsShell.run(FsShell.java:255) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hadoop.fs.FsShell.main(FsShell.java:308) Caused by: java.lang.RuntimeException: Invalid volume name: BrownFieldVolume, No mount point available for the volume. at org.apache.hadoop.fs.glusterfs.GlusterVolume.setConf(GlusterVolume.java:126) ... 18 more ~~~ Expected results ================ There are multiple possible approaches: i) The disable operation would also remove the volume name from property which defines the volume. Showing warning that you are going to remove default volume would hurt. ii) To disallow disable operation on default volume (the default volume is crucial for hadoop to work) so that to remove default volume, one would need to make it not default first. Additional info =============== See related doc BZ 1166671