Bug 909446

Summary: No data from dfsadmin -report
Product: [Community] GlusterFS Reporter: Matthew Farrellee <matt>
Component: gluster-hadoopAssignee: Bradley Childs <bchilds>
Status: CLOSED EOL QA Contact: hcfs-gluster-bugs
Severity: low Docs Contact:
Priority: low    
Version: mainlineCC: bugs, chrisw, eboyd, poelstra, rhs-bugs, vbellur
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-10-22 15:46:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 947153    

Description Matthew Farrellee 2013-02-08 19:55:54 UTC 
Description of problem:

Command-line tool, dfsadmin -report, returns no data.

Report data is available from NameNode web interface.


Version-Release number of selected component (if applicable):

# rpm -qa gluster\* org.apache.hadoop\*
glusterfs-3.4.0qa6-1.el6rhs.x86_64
glusterfs-fuse-3.4.0qa6-1.el6rhs.x86_64
org.apache.hadoop.fs.glusterfs-glusterfs-0.20.2_0.2-1.noarch
glusterfs-server-3.4.0qa6-1.el6rhs.x86_64

http://www.eng.lsu.edu/mirrors/apache/hadoop/common/stable/hadoop-1.0.4-bin.tar.gz


How reproducible:

100%


Steps to Reproduce:
1. Install and setup via https://access.redhat.com/knowledge/articles/264053

  
Actual results:

[root@head hadoop-1.0.4]# ./bin/hadoop dfsadmin -report
Initializing GlusterFS


Expected results:

Either access denied, e.g. report: org.apache.hadoop.security.AccessControlException: Access denied for user test. Superuser privilege is required

or a report, e.g.

Configured Capacity: 211378749436 (196.86 GB)
Present Capacity: 184489046016 (171.82 GB)
DFS Remaining: 171123924992 (159.37 GB)
DFS Used: 13365121024 (12.45 GB)
DFS Used%: 7.24%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available: 4 (4 total, 0 dead)
...
Comment 3 Scott Haines 2013-02-13 22:32:43 UTC 
Per Feb-13 bug triage meeting, reassigning to swatt.
Comment 4 Steve Watt 2013-03-05 16:37:13 UTC 
Assigning to Jay
Comment 5 Jay Vyas 2013-03-31 16:33:41 UTC 
Are we sure this is a bug in gluster-fs hadoop plugin?  

GlusterFS Hadoop plugin *does not* replace the hadoop org.apache.hadoop.DistributedFileSystem, but rather, implements a FileSystem class of its own (GlusterFileSystem).

Meanwhile - the DFS admin is supposed to be specific to DistributedFileSystem instances - this is enforced statically in the code, i.e. . 

https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/tools/DFSAdmin.java

  /**
  * An abstract class for the execution of a file system command
  */
  abstract private static class DFSAdminCommand extends Command {
    final DistributedFileSystem dfs;
    /** Constructor */
    public DFSAdminCommand(FileSystem fs) {
      super(fs.getConf());
      if (!(fs instanceof DistributedFileSystem)) {
        throw new IllegalArgumentException("FileSystem " + fs.getUri() + 
            " is not a distributed file system");
      }
      this.dfs = (DistributedFileSystem)fs;
    }
  }
Comment 6 Jay Vyas 2013-03-31 17:02:28 UTC 
Actually, at further glance - the report() function does nothing when run against a non "DistributedFileSystem".  Where we define DistributedFileSystem as a file system which is an instance of class 

org.apache.hadoop.hdfs.DistributedFileSystem;


SUGGESTION: 

This should be filed as a JIRA to the apache hadoop folks (i.e. to remove the hardcoded dependency on g.apache.hadoop.hdfs.DistributedFileSystem) ?  Currently - the DFSADmin class does seem to be pluggable.
Comment 7 Jay Vyas 2013-04-29 21:57:17 UTC 
Marking as low since this may not be a "real" bug.  We have to determine the correct behaviour for hadoop dfs calls when running on gluster.
Comment 8 Matthew Farrellee 2013-05-21 13:44:43 UTC 
Related - https://issues.apache.org/jira/browse/HDFS-4837

A service could exist for HCFS Gluster that implements some NameNode APIs used by various tools, such as dfsadmin or even the web ui.
Comment 9 Scott Haines 2013-11-19 19:25:13 UTC 
Per 11/13 bug triage meeting, re-assigning to bchilds.
Comment 10 Kaleb KEITHLEY 2015-10-22 15:46:38 UTC 
because of the large number of bugs filed against mainline version\ is ambiguous and about to be removed as a choice.

If you believe this is still a bug, please change the status back to NEW and choose the appropriate, applicable version for it.