Bug 960626

Summary: Single brick volume leads to exception in xattr parsing code
Product: [Community] GlusterFS Reporter: Matthew Farrellee <matt>
Component: gluster-hadoopAssignee: Bradley Childs <bchilds>
Status: CLOSED EOL QA Contact: hcfs-gluster-bugs
Severity: medium Docs Contact:
Priority: medium    
Version: mainlineCC: bchilds, bugs, chrisw, dahorak, eboyd, matt, mkudlej, rhs-bugs, vbellur
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 2.4.0-2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-10-22 15:46:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Matthew Farrellee 2013-05-07 14:22:00 UTC
Description of problem:

A single brick volume leads to exception in xattr parsing code


Version-Release number of selected component (if applicable):

glusterfs-0.0.1-20130506-0941.jar (and likely since inception)


How reproducible:

100%


Steps to Reproduce:
1. setup a volume with a single brick, e.g.
# gluster volume info
Volume Name: HadoopVol
Type: Distribute
Volume ID: 5a45e34d-7133-463b-9def-9fda05cb67bf
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: rhel64.chadlapwin7:/mnt/brick1

2. run hadoop jar /usr/lib/hadoop/hadoop-examples.jar pi 10 100000

  
Actual results:

Number of Maps  = 10
Samples per Map = 100000
13/05/07 09:15:59 INFO glusterfs.GlusterFileSystem: Initializing GlusterFS
13/05/07 09:15:59 INFO fs.FileSystem: Gluster Output Buffering size configured to 0 bytes.
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
13/05/07 09:15:59 INFO mapred.FileInputFormat: Total input paths to process : 10
13/05/07 09:15:59 INFO mapred.JobClient: Cleaning up the staging area glusterfs://rhel64.chadlapwin7:9000/user/root/.staging/job_201305070828_0003
java.lang.NullPointerException
	at org.apache.hadoop.fs.glusterfs.GlusterFSXattr.execGetFattr(GlusterFSXattr.java:163)
	at org.apache.hadoop.fs.glusterfs.GlusterFSXattr.getPathInfo(GlusterFSXattr.java:76)
	at org.apache.hadoop.fs.glusterfs.GlusterFileSystem.getFileBlockLocations(GlusterFileSystem.java:563)
	at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:231)
...


Expected results:

A successful run


Additional info:

# getfattr -m . -n trusted.glusterfs.pathinfo /mnt/glusterfs/user
getfattr: Removing leading '/' from absolute path names
# file: mnt/glusterfs/user
trusted.glusterfs.pathinfo="<POSIX(/mnt/brick1):rhel64.chadlapwin7:/mnt/brick1/user>"

The plugin's attr parsing code fails if the first directive is POSIX.

From src/main/java/org/apache/hadoop/fs/glusterfs/GlusterFSXattr.java -

        String enclosingXl=null;
...
        Pattern pattern=Pattern.compile("<(.*?)[:\\(](.*?)>");
        Matcher matcher=pattern.matcher(cmdOut);

        Pattern p_px=Pattern.compile(".*?:(.*)");
        Matcher m_px;
        String gibberish_path;

        s=null;
        while (matcher.find()){
            xlator=matcher.group(1);
            if(xlator.equalsIgnoreCase("posix")){
                if(enclosingXl.equalsIgnoreCase("replicate"))

A volume with two bricks will report DISTRIBUTE before POSIX,

# getfattr -m . -n trusted.glusterfs.pathinfo /mnt/glusterfs/user
getfattr: Removing leading '/' from absolute path names
# file: mnt/glusterfs/user
trusted.glusterfs.pathinfo="((<DISTRIBUTE:HadoopVol-dht> <POSIX(/mnt/brick1):rhel64.chadlapwin7:/mnt/brick1/user>) (HadoopVol-dht-layout (HadoopVol-client-0 0 2147483646) (HadoopVol-client-1 2147483647 4294967295)))"

which results in an initialized enclosingXl.

Comment 4 Bradley Childs 2015-05-26 18:13:24 UTC
Changed xattr handling in latest glusterfs-hadoop code to resolve issue.

Comment 5 Kaleb KEITHLEY 2015-10-22 15:46:38 UTC
because of the large number of bugs filed against mainline version\ is ambiguous and about to be removed as a choice.

If you believe this is still a bug, please change the status back to NEW and choose the appropriate, applicable version for it.