Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: $ ls -l -rw-r--r-- 1 test1 hadoop 512 Feb 18 08:35 test512 -rw-r--r-- 1 test1 hadoop 509 Feb 18 08:39 test512.2 $ hadoop fs -copyFromLocal test512.2 14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Initializing gluster volume.. 14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: Configuring GlusterFS 14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: Initializing GlusterFS, CRC disabled. 14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: GIT INFO={git.commit.id.abbrev=7b04317, git.commit.user.email=jayunit100, git.commit.message.full=Merge pull request #80 from jayunit100/2.1.6_release_fix_sudoers include the sudoers file in the srpm, git.commit.id=7b04317ff5c13af8de192626fb40c4a0a5c37000, git.commit.message.short=Merge pull request #80 from jayunit100/2.1.6_release_fix_sudoers, git.commit.user.name=jay vyas, git.build.user.name=Unknown, git.commit.id.describe=2.1.6, git.build.user.email=Unknown, git.branch=master, git.commit.time=07.02.2014 @ 12:06:31 EST, git.build.time=07.02.2014 @ 13:58:44 EST} 14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: GIT_TAG=2.1.6 14/02/18 10:30:52 INFO glusterfs.GlusterFileSystem: Configuring GlusterFS 14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Initializing gluster volume.. 14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Root of Gluster file system is /mnt/glusterfs 14/02/18 10:30:52 INFO glusterfs.GlusterVolume: mapreduce/superuser daemon : yarn 14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Working directory is : glusterfs:/user/test1 14/02/18 10:30:52 INFO glusterfs.GlusterVolume: Write buffer size : 131072 copyFromLocal: Input/output error quota stats before copy: Path Hard-limit Soft-limit Used Available -------------------------------------------------------------------------------- /user/test1 1.0KB 80% 512Bytes 512Bytes and after: /user/test1 1.0KB 80% 1.0KB 0Bytes but entire file has not been copied: $ ls -l total 1 -rw-r--r-- 1 test1 hadoop 512 Feb 18 09:32 test512 -rw-r--r-- 1 test1 hadoop 3 Feb 18 10:30 test512.2 <----------------------- Is this second issue new bug or it is feature because of default block size? Question also is, if file can be copied partially or operation should be Version-Release number of selected component (if applicable): glusterfs-3.4.0.44rhs-1.el6rhs.x86_64 glusterfs-fuse-3.4.0.44rhs-1.el6rhs.x86_64 glusterfs-geo-replication-3.4.0.44rhs-1.el6rhs.x86_64 glusterfs-hadoop-distribution-glusterfs-hadoop-setup_common-0.2-19.noarch glusterfs-hadoop-distribution-glusterfs-hadoop-setup_gluster-0.2-13.noarch glusterfs-hadoop-distribution-glusterfs-hadoop-setup_hadoop-0.1-21.noarch glusterfs-libs-3.4.0.44rhs-1.el6rhs.x86_64 glusterfs-server-3.4.0.44rhs-1.el6rhs.x86_64 hadoop-2.2.0.2.0.6.0-101.el6.x86_64 hadoop-client-2.2.0.2.0.6.0-101.el6.x86_64 hadoop-hdfs-2.2.0.2.0.6.0-101.el6.x86_64 hadoop-libhdfs-2.2.0.2.0.6.0-101.el6.x86_64 hadoop-lzo-0.5.0-1.x86_64 hadoop-lzo-native-0.5.0-1.x86_64 hadoop-mapreduce-2.2.0.2.0.6.0-101.el6.x86_64 hadoop-mapreduce-historyserver-2.2.0.2.0.6.0-101.el6.x86_64 hadoop-yarn-2.2.0.2.0.6.0-101.el6.x86_64 hadoop-yarn-nodemanager-2.2.0.2.0.6.0-101.el6.x86_64 hadoop-yarn-resourcemanager-2.2.0.2.0.6.0-101.el6.x86_64 python-rhsm-1.8.17-1.el6_4.x86_64 redhat-storage-logos-60.0.17-1.el6rhs.noarch rhs-hadoop-2.1.6-2.noarch rhs-hadoop-install-0_65-2.el6rhs.noarch How reproducible: 100% Actual results: There is Input/output error. Expected results: Result of exceeding quota should be at least meaningful error message. If this is functionality bug, then the file should not been copied at all because there is no space. I'll check it on hdfs. See also bug 994950 Additional info:
If I do the same scenario with bigger files(32MB and 32MB) it raises exception: 14/02/18 10:57:52 INFO glusterfs.GlusterVolume: Write buffer size : 131072 Exception in thread "main" org.apache.hadoop.fs.FSError: java.io.IOException: Disk quota exceeded at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:238) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:59) at java.io.DataOutputStream.write(DataOutputStream.java:90) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:80) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:52) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:112) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:366) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:338) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289) at org.apache.hadoop.fs.glusterfs.GlusterVolume.rename(GlusterVolume.java:155) at org.apache.hadoop.fs.FilterFileSystem.rename(FilterFileSystem.java:210) at org.apache.hadoop.fs.FilterFileSystem.rename(FilterFileSystem.java:210) at org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.rename(CommandWithDestination.java:322) at org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:282) at org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:245) at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:188) at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:173) at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:306) at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:278) at org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:168) at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:260) at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:244) at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:145) at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:229) at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190) at org.apache.hadoop.fs.shell.Command.run(Command.java:154) at org.apache.hadoop.fs.FsShell.run(FsShell.java:255) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hadoop.fs.FsShell.main(FsShell.java:305) Caused by: java.io.IOException: Disk quota exceeded at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:282) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:236) ... 30 more I think there should be error message with proper return code and there should be not this exception.
Per bug triage, between dev, PM and QA, moving these out of denali
BZs not targeted for Denali.
I move this BZ back to rhs-hadoop component because I think it is related to rhs-hadoop. Also there is no comment why it was moved to gluster-quota.