This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1009484 - Image list not refreshed after uploading new iso
Image list not refreshed after uploading new iso
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.3.0
Unspecified Unspecified
unspecified Severity high
: ---
: 3.3.0
Assigned To: Sergey Gotliv
Elad
storage
: Regression, TestBlocker, Triaged
: 1022183 (view as bug list)
Depends On: 1018653
Blocks:
  Show dependency treegraph
 
Reported: 2013-09-18 10:23 EDT by Dušan Kajan
Modified: 2016-02-10 15:25 EST (History)
15 users (show)

See Also:
Fixed In Version: 3.3.0
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
amureini: Triaged+


Attachments (Terms of Use)
engine log (10.35 MB, text/x-log)
2013-09-18 10:23 EDT, Dušan Kajan
no flags Details
engine.log (200.45 KB, text/x-log)
2013-09-24 11:58 EDT, Dušan Kajan
no flags Details
vdsm log from host (800.12 KB, text/x-log)
2013-09-24 11:58 EDT, Dušan Kajan
no flags Details

  None (edit)
Description Dušan Kajan 2013-09-18 10:23:49 EDT
Created attachment 799401 [details]
engine log

Description of problem:
Image list of iso domain is not refreshed after uploading new iso via iso-uploader

Version-Release number of selected component (if applicable):
is15

How reproducible:
100%

Steps to Reproduce:
1. upload iso using iso-uploader

Actual results:
iso is uploaded successfully, but the image list in Images subtab is not refreshed

Expected results:
The list should contain uploaded iso

Additional info:
engine.log attached. The list is correct after reattaching the iso domain
Comment 1 Itamar Heim 2013-09-18 10:32:58 EDT
1. what was the iso-uploader command line? was it given credentials to access engine (which is supposed to have a call to refresh the iso content)?

2. did you wait 5 minutes for an engine refresh of the list of iso's?

3. did clicking the refresh iso domain button help?

4. what are the permissions on the iso domain folder for the various iso's?
Comment 2 Dušan Kajan 2013-09-19 04:42:23 EDT
1.command was rhevm-iso-uploader --iso-domain=ISO_DOMAIN upload /root/my.iso , I typed the credentials when I was asked

2. I wait for a few minutes (not sure if 5, will test it again), on of the domains I tried it on (I have tried two domains on different storage servers) was also attached to 3.2 setup and the iso was there visible and attachable to VM

3. the refresh button did not work

4. all of them have same permissions, -rw-r-----. 1 vdsm kvm (they worked without problem on 3.2 setup and after reattaching the iso domain)
Comment 3 Dušan Kajan 2013-09-19 05:00:20 EDT
I have waited for more than 5 minutes, the iso was still not shown neither in Images subtab of domain neither in the list of CDs when I tried to attach CD to VM
Comment 4 Nir Soffer 2013-09-23 07:57:00 EDT
Please do this:

1. Setup the environment for the test so that the only step left is to upload the image.
   - Verify that your SPM is up during the test
2. Shutdown engine and vdsm and delete old logs - we want short and relevant logs for the duration of the test
3. Start engine and vdsm and connect to engine with the browser
4. Start the upload command
   - Specify the upload command as is so we can see the iso 
     filename and iso domain name and find them in the logs. 
   - If you use enviroment variables please exapnd them in the report
5. After the upload, wait 10 minutes without touching the engine

If automatic refresh did not work:
6. Try to refresh, waiting 30 seconds between retries

If manual refresh did not work:
7. Try to close the browser window and connect again
8. Try to empty the browser cache
9. Try to restart the engine
10. Try to restart vdsm

Finally, attach engine, vdsm and iso-uploader logs.
Attach also the output of ls -lht in the iso domain images directory.
Comment 5 Nir Soffer 2013-09-23 08:20:53 EDT
Correction to comment 4: the iso list will not refresh automatically by design. The refresh button should work.

Why did you think that the refresh will be automatic? Is this documented somewhere?
Comment 6 Dušan Kajan 2013-09-24 11:57:03 EDT
it used to work on older setup, that when I clicked on the Images subtab, the list was refreshed, but even refresh button does not work now.

I am adding vdsm.log and engine.log according to your guide in next comment.

The problem was still there on is16

the ls - lht output was:
    # ls -lht
    celkom 597M
    -rw-r-----. 1 vdsm kvm  63M sep 24 17:24 my.iso
    -rw-r-----. 1 vdsm kvm 283M sep 18 16:03 RHEV-toolsSetup_3.3_5.iso
    -rw-r-----. 1 vdsm kvm 253M sep 12 15:37 RHEV-toolsSetup_3.2_13.iso

iso-uploader-log was 
2013-09-24 17:17:47::INFO::rhevm-iso-uploader::1076::root:: Start uploading /root/my.iso 
2013-09-24 17:17:51::INFO::rhevm-iso-uploader::1132::root:: /root/my.iso uploaded successfully
Comment 7 Dušan Kajan 2013-09-24 11:58:13 EDT
Created attachment 802332 [details]
engine.log
Comment 8 Dušan Kajan 2013-09-24 11:58:44 EDT
Created attachment 802333 [details]
vdsm log from host
Comment 9 Nir Soffer 2013-09-24 13:38:41 EDT
(In reply to Dušan Kajan from comment #6)
> it used to work on older setup, that when I clicked on the Images subtab,
> the list was refreshed, 

Can you check which was the last version that the automatic refresh did work?

> but even refresh button does not work now.

What did you have to do to get the list updated? close the browser and reconnect? restart the engine? restart vdsm? something else?
Comment 10 Dušan Kajan 2013-09-25 07:54:51 EDT
I have to reattach the domain (detach and attach and activate again)
Comment 11 Tal Nisan 2013-10-09 09:22:13 EDT
Uploaded a few ISO to the domain with the iso uploader, moved to the storage tab and clicked on the storage and images subtab and the already uploaded iso files, then without moving to another tab/subtab I uploaded another iso with the iso uploaded, clicked on the refresh icon on the right side of the subtab and the new iso file appeared correctly
Comment 13 Ayal Baron 2013-10-13 11:33:52 EDT
Dušan,

Can you verify whether this is working for you in an updated version?
Comment 14 Ayal Baron 2013-10-16 05:31:55 EDT
Sergey, please look at this, and add the reference to the similar bug you're working on.
Comment 15 Dušan Kajan 2013-10-22 06:52:39 EDT
I was not able to get it working. I uploaded two isos, wait for 15 minutes without leaving the subtab, upload another one, click refresh button (tried both subtab and tab refresh button), but neither of isos were in the list
Comment 16 Sergey Gotliv 2013-10-23 09:22:08 EDT
*** Bug 1022183 has been marked as a duplicate of this bug. ***
Comment 17 Sergey Gotliv 2013-10-24 05:40:06 EDT
*** Bug 1022183 has been marked as a duplicate of this bug. ***
Comment 19 Bill Sanford 2013-10-25 13:51:58 EDT
I just added en_windows_server_2008_with_sp2_x86_dvd_342333.iso to the ISO directory and did a chown 36:36 to the file. 

I can not see it in the refreshed ISO Storage list in RHEV-M.

I have hit the refresh button in the upper right corner of the "Images" list in the Storage tab in RHEV-M, I have brought the ISO storage in to Maintenance Mode and reactivated the ISO storage, and I have rebooted the RHEV-M server. There is nothing I can do to see this particular ISO which is a supported platform.
Comment 21 Sergey Gotliv 2013-10-25 14:28:22 EDT
Bill,

Did you read comment 18 and https://bugzilla.redhat.com/show_bug.cgi?id=1022183#c12? It explains that currently there is a problem in kernel related to refresh of NFS mounted directories.

Please, follow after that kernel bug which is the root cause of the problem:
https://bugzilla.redhat.com/show_bug.cgi?id=1018653

In order to workaround that problem, you can remove your ISO domain and then import it again, it should work because its actually unmount and then mount the iso directory again.

I hope it helps
Comment 22 Bill Sanford 2013-10-25 14:51:29 EDT
I just did a complete removal of the ISO storage and imported it again and I still do not see all the ISOs that are in the storage. 

In particular, the one in comment 19 is still not visible.
Comment 23 Sergey Gotliv 2013-11-05 07:51:22 EST
Please, retest it with kernel 2.6.32-428.el6.x86_64 or newer.
Comment 24 Scott Herold 2013-11-05 17:23:16 EST
I just ran into this today with RHEV 3.3.0-0.30-beta1 running on both RHEL 6.4 and 6.5 Beta.  Using kernel 2.6.32.422, I had to drill into the ISO export directory to where the actual file was being stored and run:

chmod +r filename.iso

As soon as the o+r attrib was set, the ISO showed up in my UI list.  If I remove the read attribute for Other (o-r), the ISO disappears.  

It seems the default permissions being set are mismatched to what is required for display.
Comment 25 Sergey Gotliv 2013-11-05 17:45:22 EST
Scott,

Can you, please, verify that default permissions were 640 and file belongs to the kvm group?

Thanks
Comment 26 Scott Herold 2013-11-05 18:15:59 EST
Just uploaded a fresh image without my playing around with it...

Default perms are 640 for user:vdsm and group:kvm

Same result, a simple chmod o+r filename.iso, and it shows up.
Comment 27 Scott Herold 2013-11-06 15:41:53 EST
I upgraded my hypervisor hosts with the is22 RHEV-Hypervisor build (20131106.0.el6).  Kernel version was updated to 2.6.32.430.  This seems to have solved the issue with the hosts mounting the NFS share with the wrong permissions.  After the upgrade, The proper 640 permissoins are set for vdsm:kvm on the /rhev/data-center mount point, and no longer nobody:nobody.
Comment 28 Elad 2013-11-14 07:21:17 EST
Images list refreshed after uploading new ISO to the ISO domain using rhevm-iso-uploader. 

I used the command:

rhevm-iso-uploader --nfs-server=orion.qa.lab.tlv.redhat.com:/export/shared_iso_domain/shared_iso_domain1/7233a711-98e8-4c3c-bcfa-44c4bcc4f6c6/images/11111111-1111-1111-1111-111111111111 upload 123.iso


Verified with is23, using kernel version 2.6.32-428.el6.x86_64 on hosts
Comment 29 Itamar Heim 2014-01-21 17:30:52 EST
Closing - RHEV 3.3 Released
Comment 30 Itamar Heim 2014-01-21 17:30:53 EST
Closing - RHEV 3.3 Released
Comment 31 Itamar Heim 2014-01-21 17:33:50 EST
Closing - RHEV 3.3 Released

Note You need to log in before you can comment on or make changes to this bug.