Bug 1005100 - [iso-uploader] engine-iso-uploader buggy - Message: [Errno 1] Operation not permitted
[iso-uploader] engine-iso-uploader buggy - Message: [Errno 1] Operation not p...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine-iso-uploader (Show other bugs)
3.3.0
Unspecified Unspecified
high Severity high
: ---
: 3.3.0
Assigned To: Sandro Bonazzola
Jiri Belka
integration
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-09-06 04:38 EDT by Jiri Belka
Modified: 2015-09-22 09 EDT (History)
9 users (show)

See Also:
Fixed In Version: rhevm-iso-uploader-3.3.0-1.el6ev
Doc Type: Bug Fix
Doc Text:
When uploading an image to an NFS destination, if an error occurred while renaming the used temporary file to the final destination, the tool displayed a misleading successful upload message. Now, when the error occurs, the tool exits with a status 3 upload error and displays an error message.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-01-21 11:31:41 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 19095 None None None Never
oVirt gerrit 19194 None None None Never

  None (edit)
Description Jiri Belka 2013-09-06 04:38:03 EDT
Description of problem:
uploading iso does not work

# engine-iso-uploader -v -i str01-iso-shared upload /usr/share/rhev-guest-tools-iso/RHEV-toolsSetup_3.3_5.iso                                  
Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort): 
DEBUG: API Vendor(None) API Version(3.3.0)
DEBUG: id=0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71 address=10.34.63.204 path=/home/iso/shared
Uploading, please wait...
DEBUG: local NFS mount point is /tmp/tmpTvEwe9
DEBUG: NFS mount command (/bin/mount -t nfs -o rw,sync,soft 10.34.63.204:/home/iso/shared /tmp/tmpTvEwe9)
DEBUG: /bin/mount -t nfs -o rw,sync,soft 10.34.63.204:/home/iso/shared /tmp/tmpTvEwe9
DEBUG: _cmds(['/bin/mount', '-t', 'nfs', '-o', 'rw,sync,soft', '10.34.63.204:/home/iso/shared', '/tmp/tmpTvEwe9'])
DEBUG: returncode(0)
DEBUG: STDOUT()
DEBUG: STDERR()
INFO: Start uploading /usr/share/rhev-guest-tools-iso/RHEV-toolsSetup_3.3_5.iso 
DEBUG: Size of /usr/share/rhev-guest-tools-iso/RHEV-toolsSetup_3.3_5.iso:       295915520 bytes 288980.0 1K-blocks      282.0 MB
DEBUG: Available space in /tmp/tmpTvEwe9/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111:      83272138752 bytes       8132044
8.0 1K-blocks   79414.0 MB
DEBUG: euid(0) egid(0)
DEBUG: euid(0) egid(0)
ERROR: Problem renaming /tmp/tmpTvEwe9/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111/.RHEV-toolsSetup_3.3_5.iso to /tmp/tmpT
vEwe9/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.3_5.iso.  Message: [Errno 1] Operation not permitted
INFO: /usr/share/rhev-guest-tools-iso/RHEV-toolsSetup_3.3_5.iso uploaded successfully
DEBUG: /bin/umount -t nfs -f  /tmp/tmpTvEwe9
DEBUG: /bin/umount -t nfs -f  /tmp/tmpTvEwe9
DEBUG: _cmds(['/bin/umount', '-t', 'nfs', '-f', '/tmp/tmpTvEwe9'])
DEBUG: returncode(0)
DEBUG: STDOUT()
DEBUG: STDERR()


Version-Release number of selected component (if applicable):
is13 / rhevm-iso-uploader-3.3.0-0.3.rc.el6ev.noarch

How reproducible:
100%

Steps to Reproduce:
1. try to upload iso file
2.
3.

Actual results:
not really uploaded

Expected results:
working?

Additional info:
Comment 1 Sandro Bonazzola 2013-09-10 03:08:44 EDT
Hi Jiri, can you check if you have selinux enabled and paste the output of:
 getenforce
 ls -lZ /home/iso/shared 
on host 10.34.63.204 ?

In the meanwhile I'll try to figure out why it says 
INFO: /usr/share/rhev-guest-tools-iso/RHEV-toolsSetup_3.3_5.iso uploaded successfully also when it fails.
Comment 2 Jiri Belka 2013-09-10 03:36:37 EDT
# ls -lZ /rhev/data-center/mnt/10.34.63.204:_home_iso_shared/
drwxr-xr-x. vdsm kvm system_u:object_r:nfs_t:s0       0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71
-rwxr-xr-x. vdsm kvm system_u:object_r:nfs_t:s0       __DIRECT_IO_TEST__

# getenforce 
Enforcing
Comment 3 Itamar Heim 2013-09-10 03:44:53 EDT
selinux does not support nfs / iso storage domains.
Comment 4 Sandro Bonazzola 2013-09-10 03:50:26 EDT
Jiri, I was meaning: can you ssh to 10.34.63.204 and run the commands on that host?
 getenforce
 ls -lZ /home/iso/shared 

The selinux context should be public_content_rw_t for all that tree.
can you try to set it and reproduce the issue?

If it's not correct, you can fix that context with:
 semanage fcontext -a -t public_content_rw_t /home/iso/shared(/.*)?
 restorecon -r /home/iso/shared
Comment 5 Sandro Bonazzola 2013-09-10 04:40:31 EDT
Tested on upstream / F19 it works fine with right contexts:

# engine-iso-uploader -v -i ISO upload /home/Fedora-19-x86_64-DVD.iso 
Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort): 
DEBUG: API Vendor(ovirt.org)	API Version(3.3.0)
DEBUG: id=809cbe8a-0ec7-4c03-a4d8-7994689a88ca address=dellserver.home path=/var/lib/exports/iso
Uploading, please wait...
DEBUG: local NFS mount point is /tmp/tmpg9Ana7
DEBUG: NFS mount command (/bin/mount -t nfs -o rw,sync,soft dellserver.home:/var/lib/exports/iso /tmp/tmpg9Ana7)
DEBUG: /bin/mount -t nfs -o rw,sync,soft dellserver.home:/var/lib/exports/iso /tmp/tmpg9Ana7
DEBUG: _cmds(['/bin/mount', '-t', 'nfs', '-o', 'rw,sync,soft', 'dellserver.home:/var/lib/exports/iso', '/tmp/tmpg9Ana7'])
DEBUG: returncode(0)
DEBUG: STDOUT()
DEBUG: STDERR()
INFO: Start uploading /home/Fedora-19-x86_64-DVD.iso 
DEBUG: Size of /home/Fedora-19-x86_64-DVD.iso:	4444913664 bytes	4340736.0 1K-blocks	4239.0 MB
DEBUG: Available space in /tmp/tmpg9Ana7/809cbe8a-0ec7-4c03-a4d8-7994689a88ca/images/11111111-1111-1111-1111-111111111111:	35995516928 bytes	35151872.0 1K-blocks	34328.0 MB
DEBUG: euid(0) egid(0)
DEBUG: euid(0) egid(0)
INFO: /home/Fedora-19-x86_64-DVD.iso uploaded successfully
DEBUG: /bin/umount -t nfs -f  /tmp/tmpg9Ana7
DEBUG: /bin/umount -t nfs -f  /tmp/tmpg9Ana7
DEBUG: _cmds(['/bin/umount', '-t', 'nfs', '-f', '/tmp/tmpg9Ana7'])
DEBUG: returncode(0)
DEBUG: STDOUT()
DEBUG: STDERR()


another thing to check: my NFS share is exported as:
/var/lib/exports/iso 0.0.0.0/0.0.0.0(rw)

can you check if your export has additional parameters?
Comment 6 Jiri Belka 2013-09-10 05:13:53 EDT
Ah, I got where is the problem. engine-iso-upload uploads to

/tmp/tmpG4K0L3/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111/.RHEV-toolsSetup_3.3_5.iso 

which starts with '.', it is hidden file, then it tries to rename the uploaded with to 'visible' filename.

The problem is we have 'append' file attribute on the directory /home/iso/shared/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images.

Is it really necessary to upload to a temporary file? What about to make cksum of iso file, upload it to real filename on iso domain and then recheck if it was correctly uploaded?

If this would not be accepted, then close the BZ as 'NOTABUG'.
Comment 7 Jiri Belka 2013-09-10 05:16:21 EDT
Still it seems DEBUG mode of successful upload does not show the renaming part of the work, so it seems DEBUG mode does not tell us whole story :)
Comment 8 Sandro Bonazzola 2013-09-10 05:28:42 EDT
(In reply to Jiri Belka from comment #6)
> Is it really necessary to upload to a temporary file? What about to make
> cksum of iso file, upload it to real filename on iso domain and then recheck
> if it was correctly uploaded?

suppose you've already a working iso image on destination and you're uploading an updated image with --force for overwriting it. Without the intermediate step you'll ends to having a corrupted iso if the upload fails. Rename is atomic so it can't lead to having a corrupt file.

(In reply to Jiri Belka from comment #7)
> Still it seems DEBUG mode of successful upload does not show the renaming
> part of the work, so it seems DEBUG mode does not tell us whole story :)

I'll add a debug message for tracing the renaming action.

I'll also add a note telling to check the directory permission for allowing file renaming if that operation fails.

Still to check why it says uploaded successfully also when it fails.

Reducing priority to high.
Comment 9 Sandro Bonazzola 2013-09-10 10:10:11 EDT
patch pushed to upstream master:

Uploading to a NFS destination, if an error
were encountered while renaming the used temporary
file to the final destination,
the application displayed: uploaded successfully.

Now the application exits with exit status 3
(upload error) and display an error message.
Added also a debug log for recording the rename
action in the verbose log.
Comment 10 Sandro Bonazzola 2013-09-12 10:09:48 EDT
patch merged upstream master, pushed on upstream 3.3 branch.
Comment 11 Sandro Bonazzola 2013-09-12 10:11:36 EDT
patch merged on upstream 3.3 branch.
Comment 13 Jiri Belka 2013-09-16 05:03:09 EDT
ok

DEBUG: Renaming /tmp/tmpIVqEjP/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111/.install54.iso to /tmp/tmpIVqEjP/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111/install54.iso
ERROR: Problem renaming /tmp/tmpIVqEjP/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111/.install54.iso to /tmp/tmpIVqEjP/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111/install54.iso. Message: [Errno 1] Operation not permitted
ERROR: Please ensure to have permissions for renaming files inside /tmp/tmpIVqEjP/0c78b4d6-ba00-4d3e-9f9f-65c7d5899d71/images/11111111-1111-1111-1111-111111111111
DEBUG: /bin/umount -t nfs -f  /tmp/tmpIVqEjP
DEBUG: /bin/umount -t nfs -f  /tmp/tmpIVqEjP
DEBUG: _cmds(['/bin/umount', '-t', 'nfs', '-f', '/tmp/tmpIVqEjP'])
DEBUG: returncode(0)
DEBUG: STDOUT()
DEBUG: STDERR()
# echo $?
3
Comment 14 errata-xmlrpc 2014-01-21 11:31:41 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0077.html

Note You need to log in before you can comment on or make changes to this bug.