Hide Forgot
Description of problem: While running the script to rename the RHEVM with ovirt-engine-rename tool, following error observed when ISO domain exist on RHEVM system. ====== 2015-08-04 10:05:59 DEBUG otopi.ovirt_**FILTERED**_setup.**FILTERED**_common.database database.execute:214 Result: [{'connection': '<rhevm_hostname>:/_iso_domain', 'storage_name': 'ISOs'}] 2015-08-04 10:05:59 WARNING otopi.plugins.ovirt_**FILTERED**_rename.ovirt-**FILTERED**.database database._validation:138 Engine host hosting Storage Domains 2015-08-04 10:05:59 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND The following Storage Domains use the **FILTERED** host 2015-08-04 10:05:59 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND as an NFS server: 2015-08-04 10:05:59 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND 2015-08-04 10:05:59 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND ISOs 2015-08-04 10:05:59 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND 2015-08-04 10:05:59 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND Cannot rename the **FILTERED** host. Please backup relevant 2015-08-04 10:05:59 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND data if needed, remove all of these domains, and then 2015-08-04 10:05:59 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:215 DIALOG:SEND run this utility again. 2015-08-04 10:05:59 DEBUG otopi.context context._executeMethod:152 method exception Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/otopi/context.py", line 142, in _executeMethod method['method']() File "/usr/share/ovirt-**FILTERED**/setup/bin/../plugins/ovirt-**FILTERED**-rename/ovirt-**FILTERED**/database.py", line 153, in _validation raise RuntimeError(_('Cannot rename host hosting Storage Domains')) RuntimeError: Cannot rename host hosting Storage Domains ====== Version-Release number of selected component (if applicable): RHEVM 3.5.3 How reproducible: Always Steps to Reproduce: 1. Create local on host ISO domain 2. run ovirt-engine-rename utility 3. script fails Actual results: script fails with error: Cannot rename host hosting Storage Domains Expected results: Script should run and change the domain hostname as well. Additional info: I am not sure about the component to select here.
No idea what needs to change, nor what's the best way to do that (directly in db? api/sdk? etc.). Also no idea what effect, if at all, this might have on running VMs etc. Also not sure who to ask :-) Sandro?
"Cannot rename host hosting Storage Domains" has been there since the rename tool has been created (commit 50eada928cda82144c3801da5ba7494f6287d777) the referenced wiki http://www.ovirt.org/Changing_Engine_Hostname doesn't cover specifications about storage domains handling so I'm not sure about why the check has been added. Issue here is that we rename the host and it's hosting shared storage, all the VMs will probably be paused loosing storage connection. Adding allon to the loop.
(In reply to Sandro Bonazzola from comment #3) > "Cannot rename host hosting Storage Domains" has been there since the rename > tool has been created (commit 50eada928cda82144c3801da5ba7494f6287d777) > > the referenced wiki http://www.ovirt.org/Changing_Engine_Hostname doesn't > cover specifications about storage domains handling so I'm not sure about > why the check has been added. > > Issue here is that we rename the host and it's hosting shared storage, all > the VMs will probably be paused loosing storage connection. > > Adding allon to the loop. If I understand correctly, the issue here is that you're renaming the engine's host name and aren't able to access storage domains located ON THAT HOST afterwards, right? If this is indeed the case, the mount path to any NFS domain should also be update accordingly: UPDATE storage_server_connections SET connection = REPLACE(connection, 'old-host-name', 'new-host-name' WHERE vfs_type = 'nfs' AND connection LIKE CONCAT('old-host-name',':%')
Missing a bracket there, of course (copy-paste error, apologies): (In reply to Allon Mureinik from comment #4) > UPDATE storage_server_connections > SET connection = REPLACE(connection, 'old-host-name', 'new-host-name' > WHERE vfs_type = 'nfs' AND connection LIKE CONCAT('old-host-name',':%') UPDATE storage_server_connections SET connection = REPLACE(connection, 'old-host-name', 'new-host-name') WHERE vfs_type = 'nfs' AND connection LIKE CONCAT('old-host-name', ':%')
Thanks. What happens to running VMs? This tool is specifically for _renaming_, not renumbering. Can we safely assume that the hosts will simply keep the domains mounted (with same IP address etc) and will simply look up the new name when needed (e.g. on migration)? No restart of anything needed? I still think it will be wise to notify/query the user about this.
(In reply to Yedidyah Bar David from comment #6) > Can we safely > assume > that the hosts will simply keep the domains mounted (with same IP address > etc) > and will simply look up the new name when needed (e.g. on migration)? No > restart > of anything needed? The host (and thus - VM) will attempt to keep the same mount paths active until the engine explicitly tells them to do something else (e.g., by putting the domain to maintenance). Note that if the mount is done via the host NAME you might get a stale mount point on the host side. Having said that, a VM shouldn't really read from an ISO domain after it's done booting.
Can you please make sure to get docs on this for rename process? We should allow forcing maybe with flag in a case were the user doesn't have any VM depending on the iso domain.
Didi, can you update the oVirt wiki page about the rename tool[1] once fixed? [1] http://www.ovirt.org/Changing_Engine_Hostname Please also review Doc-Text.
(In reply to Allon Mureinik from comment #7) > (In reply to Yedidyah Bar David from comment #6) > > Can we safely > > assume > > that the hosts will simply keep the domains mounted (with same IP address > > etc) > > and will simply look up the new name when needed (e.g. on migration)? No > > restart > > of anything needed? > The host (and thus - VM) will attempt to keep the same mount paths active > until the engine explicitly tells them to do something else (e.g., by > putting the domain to maintenance). > Note that if the mount is done via the host NAME you might get a stale mount > point on the host side. To be on the safe side, I think we should check some/all of these: 1. Is the storage domain active? 2. Is there a VM with it attached? And if 'yes', query user, suggesting to stop VMs and/or put domain to maint. Allon - how do I check these? > > Having said that, a VM shouldn't really read from an ISO domain after it's > done booting. Users might use the iso domain also for other things, besides OSes. In particular we also supply (and expect people to use) the windows guest tools as an iso image. I agree that this one too is expected to be used mostly shortly after installation (or upgrades) but still. Thanks.
Amit - cam you answer this please? Thanks!
These are simple SQL queries for your question, tell me if you need anything further than that. > 1. Is the storage domain active? select CASE status WHEN 3 THEN 't' ELSE 'f' END from storage_domains where id = '%storageDomainID%' > 2. Is there a VM with it attached? SELECT exists( SELECT * FROM all_disks_for_vms WHERE storage_id = '%storageDomainID%' )
(In reply to Amit Aviram from comment #12) > These are simple SQL queries for your question, tell me if you need anything > further than that. > > > 1. Is the storage domain active? > select CASE status WHEN 3 THEN 't' ELSE 'f' END from storage_domains where > id = '%storageDomainID%' > > > 2. Is there a VM with it attached? > SELECT exists( > SELECT * FROM all_disks_for_vms > WHERE storage_id = '%storageDomainID%' > ) Just to be clear, all that is marked with "%something%" is a placeholder for the actual UUID you are looking for, and it is not the actual syntax to use (e.g. use: > select CASE status WHEN 3 THEN true ELSE false END from storage_domains where > id = '477afe45-c8f5-4985-81ba-b9e2603bb6d9' ) Also- for the 2nd query, you might want to get the VM names&id instead of just t/f. in that case you can use: SELECT distinct vm_names, vm_id FROM all_disks_for_vms WHERE storage_id = '%storageDomainID%' (Allon, thanks for highlighting this)
Any updates to this bug (it's marked as high priority) ?
(In reply to Yaniv Kaul from comment #14) > Any updates to this bug (it's marked as high priority) ? Working on it, hopefully will have something this week.
Amit, the above covers only disks, not iso images. How can I check if a vm has an iso image from a specific iso domain attached?
did 'pg_dump engine' before and after 'Change CD', and the only relevant change seems to me to be in vm_dynamic.current_cd .
Amit told me in private to check vm_static.iso_path. Amit - just a guess, judging the names - isn't it that 'vm_static.iso_path' is the image the vm is configured to boot from, whereas 'vm_dynamic.current_cd' is the currently attached cd?
Meanings of theses fields are not reflected so much from its names :) "vm_static.iso_path" describes the image that will be attached to the VM as a CD-ROM *when it will be activated* "vm_dynamic.current_cd" describes the images that was connected to the VM as a CD *When it was activated* For example- - I am attaching some image "disk.iso" which resides in my ISO domain to a turned off VM. now its "vm_static.iso_path" will have "disk.iso" as a value. - I am activating the VM. now "vm_dynamic.current_cd" will have "disk.iso" as a value. - I am editing the VM *while it is up*, and removing the CD-ROM. this will make change only when I'll restart the VM, vm_static.iso_path will contain nothing as a value, but "vm_dynamic.current_cd" will still have "disk.iso" as a value. Hope that explains the meaning better :)
Not sure how to continue working on this. Patch is currently pending review. There are two questions here: 1. What to check For vm status, patch currently allows only '0'. So paused will still emit a message. 2. whether to allow forcing a rename even if the checks fail Personally I do not like software that thinks it's smarter than me ... Is it absolutely mandatory? E.g. if a user forces a rename, waits until hosts/VMs are stuck, then forcefully reboots them, should there be any long-term problems, apart from normal problems that might happen when plugging out the power cord?
Talked with Amit about this. Summary: 1. Basically the checks are ok. 2. The tool will not allow forcing a rename if any checks fail. We'll change that if (when?) people complain about this with valid cases.
IIUC you should probably also make sure that the storage domains are moved to maintenance otherwise the old mount points will be used which now are pointing to a non existing host name
(In reply to Tal Nisan from comment #22) > IIUC you should probably also make sure that the storage domains are moved > to maintenance otherwise the old mount points will be used which now are > pointing to a non existing host name AFAIU I already do that, you are welcome to review the patch. Thanks!
(In reply to Tal Nisan from comment #22) > IIUC you should probably also make sure that the storage domains are moved > to maintenance otherwise the old mount points will be used which now are > pointing to a non existing host name That's also being taken care of. thanks!
Adapted the doc text to the current behavior. Andrew/QE - please search for "Verified+1" comments in [1] for an example flow. [1] https://gerrit.ovirt.org/50396
Andrew - you probably should also update the actual documentation. Setting needinfo on you as a reminder - you can open a doc bug if needed.
Verified with: rhevm-3.6.3.1-0.1.el6.noarch [ INFO ] Engine machine hosting Storage Domains The following Storage Domains use the engine machine as an NFS server, and are active: ISO_DOMAIN Needed action: They should be moved to Maintenance. Some storage domains hosted on the engine machine are in use. Please complete the above detailed actions. To abort, simply kill this utility with ^C. Press Enter to continue: [ INFO ] Engine machine hosting Storage Domains The following Storage Domains use the engine machine as an NFS server: ISO_DOMAIN They will be modified to use the new name.
Thanks, Didi, for bringing the fix to our attention. I've raised bug 1313217 to track the addition of a note to our existing ovirt-engine-rename content. Should the 'Doc Type' on this bug also be moved from 'Known Issue' to 'Bug Fix' now?
Done.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0376.html