Bug 1066514 - Input/output error because of quota
Summary: Input/output error because of quota
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: rhs-hadoop
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Bradley Childs
QA Contact: BigData QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-18 14:53 UTC by Martin Kudlej
Modified: 2014-08-14 15:41 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-08-14 15:41:39 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 994950 0 medium CLOSED quota: Input/output error while reaching to fill already set quota limit 2021-02-22 00:41:40 UTC

Internal Links: 994950

Description Martin Kudlej 2014-02-18 14:53:31 UTC
Description of problem:

$ ls -l 
-rw-r--r-- 1 test1 hadoop  512 Feb 18 08:35 test512
-rw-r--r-- 1 test1 hadoop  509 Feb 18 08:39 test512.2

$ hadoop fs -copyFromLocal test512.2
14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Initializing gluster volume..
14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: Configuring GlusterFS
14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: Initializing GlusterFS,  CRC disabled.
14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: GIT INFO={git.commit.id.abbrev=7b04317, git.commit.user.email=jayunit100, git.commit.message.full=Merge pull request #80 from jayunit100/2.1.6_release_fix_sudoers

include the sudoers file in the srpm, git.commit.id=7b04317ff5c13af8de192626fb40c4a0a5c37000, git.commit.message.short=Merge pull request #80 from jayunit100/2.1.6_release_fix_sudoers, git.commit.user.name=jay vyas, git.build.user.name=Unknown, git.commit.id.describe=2.1.6, git.build.user.email=Unknown, git.branch=master, git.commit.time=07.02.2014 @ 12:06:31 EST, git.build.time=07.02.2014 @ 13:58:44 EST}
14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: GIT_TAG=2.1.6
14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: Configuring GlusterFS
14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Initializing gluster volume..
14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Root of Gluster file system is /mnt/glusterfs
14/02/18 10:30:52 INFO glusterfs.GlusterVolume: mapreduce/superuser daemon : yarn
14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Working directory is : glusterfs:/user/test1
14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Write buffer size : 131072
copyFromLocal: Input/output error

quota stats before copy:
                  Path                   Hard-limit Soft-limit   Used  Available
--------------------------------------------------------------------------------
/user/test1                                1.0KB       80%    512Bytes 512Bytes

and after:
/user/test1                                1.0KB       80%       1.0KB  0Bytes

but entire file has not been copied:
$ ls -l
total 1
-rw-r--r-- 1 test1 hadoop 512 Feb 18 09:32 test512
-rw-r--r-- 1 test1 hadoop   3 Feb 18 10:30 test512.2 <-----------------------

Is this second issue new bug or it is feature because of default block size?
Question also is, if file can be copied partially or operation should be 

Version-Release number of selected component (if applicable):
glusterfs-3.4.0.44rhs-1.el6rhs.x86_64
glusterfs-fuse-3.4.0.44rhs-1.el6rhs.x86_64
glusterfs-geo-replication-3.4.0.44rhs-1.el6rhs.x86_64
glusterfs-hadoop-distribution-glusterfs-hadoop-setup_common-0.2-19.noarch
glusterfs-hadoop-distribution-glusterfs-hadoop-setup_gluster-0.2-13.noarch
glusterfs-hadoop-distribution-glusterfs-hadoop-setup_hadoop-0.1-21.noarch
glusterfs-libs-3.4.0.44rhs-1.el6rhs.x86_64
glusterfs-server-3.4.0.44rhs-1.el6rhs.x86_64
hadoop-2.2.0.2.0.6.0-101.el6.x86_64
hadoop-client-2.2.0.2.0.6.0-101.el6.x86_64
hadoop-hdfs-2.2.0.2.0.6.0-101.el6.x86_64
hadoop-libhdfs-2.2.0.2.0.6.0-101.el6.x86_64
hadoop-lzo-0.5.0-1.x86_64
hadoop-lzo-native-0.5.0-1.x86_64
hadoop-mapreduce-2.2.0.2.0.6.0-101.el6.x86_64
hadoop-mapreduce-historyserver-2.2.0.2.0.6.0-101.el6.x86_64
hadoop-yarn-2.2.0.2.0.6.0-101.el6.x86_64
hadoop-yarn-nodemanager-2.2.0.2.0.6.0-101.el6.x86_64
hadoop-yarn-resourcemanager-2.2.0.2.0.6.0-101.el6.x86_64
python-rhsm-1.8.17-1.el6_4.x86_64
redhat-storage-logos-60.0.17-1.el6rhs.noarch
rhs-hadoop-2.1.6-2.noarch
rhs-hadoop-install-0_65-2.el6rhs.noarch


How reproducible:
100%

Actual results:
There is Input/output error. 

Expected results:
Result of exceeding quota should be at least meaningful error message. If this is functionality bug, then the file should not been copied at all because there is no space. I'll check it on hdfs.
See also bug 994950

Additional info:

Comment 1 Martin Kudlej 2014-02-18 14:59:35 UTC
If I do the same scenario with bigger files(32MB and 32MB) it raises exception:
14/02/18 10:57:52 INFO glusterfs.GlusterVolume: Write buffer size : 131072
Exception in thread "main" org.apache.hadoop.fs.FSError: java.io.IOException: Disk quota exceeded
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:238)
        at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
        at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:59)
        at java.io.DataOutputStream.write(DataOutputStream.java:90)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:80)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:52)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:112)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:366)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:338)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
        at org.apache.hadoop.fs.glusterfs.GlusterVolume.rename(GlusterVolume.java:155)
        at org.apache.hadoop.fs.FilterFileSystem.rename(FilterFileSystem.java:210)
        at org.apache.hadoop.fs.FilterFileSystem.rename(FilterFileSystem.java:210)
        at org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.rename(CommandWithDestination.java:322)
        at org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:282)
        at org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:245)
        at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:188)
        at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:173)
        at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:306)
        at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:278)
        at org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:168)
        at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:260)
        at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:244)
        at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:145)
        at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:229)
        at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190)
        at org.apache.hadoop.fs.shell.Command.run(Command.java:154)
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:255)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:305)
Caused by: java.io.IOException: Disk quota exceeded
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:282)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:236)
        ... 30 more

I think there should be error message with proper return code and there should be not this exception.

Comment 2 Vivek Agarwal 2014-04-07 11:40:06 UTC
Per bug triage, between dev, PM and QA, moving these out of denali

Comment 3 Nagaprasad Sathyanarayana 2014-05-06 10:35:10 UTC
BZs not targeted for Denali.

Comment 4 Martin Kudlej 2014-05-26 11:14:07 UTC
I move this BZ back to rhs-hadoop component because I think it is related to rhs-hadoop. Also there is no comment why it was moved to gluster-quota.


Note You need to log in before you can comment on or make changes to this bug.