Bug 1816004
| Summary: | Failed to create a new disk or copy template disk on iscsi of FCP storage domain - qemu-img: Protocol driver \'host_device\' does not support image creation | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [oVirt] vdsm | Reporter: | Avihai <aefrat> | ||||||||
| Component: | General | Assignee: | Benny Zlotnik <bzlotnik> | ||||||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Evelina Shames <eshames> | ||||||||
| Severity: | urgent | Docs Contact: | |||||||||
| Priority: | unspecified | ||||||||||
| Version: | 4.40.7 | CC: | bugs, bzlotnik, eshenitz, lsvaty, michal.skrivanek, nsoffer, sfishbai | ||||||||
| Target Milestone: | ovirt-4.4.0 | Keywords: | Automation, AutomationBlocker, Regression | ||||||||
| Target Release: | 4.40.9 | Flags: | pm-rhel:
ovirt-4.4+
pm-rhel: blocker? |
||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | vdsm-4.40.9 | Doc Type: | No Doc Update | ||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2020-05-20 20:03:36 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Bug Depends On: | 1816007 | ||||||||||
| Bug Blocks: | 1810143 | ||||||||||
| Attachments: |
|
||||||||||
LVM filters are configured on all hosts: [root@lynx24 vdsm]# vdsm-tool config-lvm-filter Analyzing host... LVM filter is already configured for Vdsm This bug report has Keywords: Regression or TestBlocker. Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP. We must resolve this in 4.4.0, basic functionality is broken. Created attachment 1672915 [details]
screenshot - when trying to import template from glance to iscsi SD
Created attachment 1672921 [details]
Logs
Yesterday we ran the new build(4.4.0-0.26) in our environments. The running failed to create a copy template disk on iscsi and FC.
After, I checked it manually via webadmin and trying to import template from glance to iscsi SD (screenshots attached).
From engine.log:
2020-03-23 16:03:50,181+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHostJobsVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-81) [7197f14a-1170-401d-a133-c7cd767bfc04]
START, GetHostJobsVDSCommand(HostName = host_mixed_2, GetHostJobsVDSCommandParameters:{hostId='fa9516de-7757-48e2-84fe-f99418546698', type='storage', jobIds='[93faaf0d-de20-47cf-be4a-d4397828551d]'}), log id: 4e
a5c056
2020-03-23 16:03:50,203+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHostJobsVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-81) [7197f14a-1170-401d-a133-c7cd767bfc04]
FINISH, GetHostJobsVDSCommand, return: {93faaf0d-de20-47cf-be4a-d4397828551d=HostJobInfo:{id='93faaf0d-de20-47cf-be4a-d4397828551d', type='storage', description='copy_data', status='failed', progress='0', error=
'VDSError:{code='GeneralException', message='General Exception: ('Command [\'/usr/bin/qemu-img\', \'convert\', \'-p\', \'-t\', \'none\', \'-T\', \'none\', \'-f\', \'qcow2\', \'/rhev/data-center/mnt/yellow-vdsb.q
a.lab.tlv.redhat.com:_Storage__NFS_storage__local__ge9__nfs__0/21b4df51-ab7c-4175-8203-66213643ad3a/images/5faee0e8-4565-4827-8364-485ec6981163/7e56ad89-5639-4da3-852f-0f19340ef7f8\', \'-O\', \'qcow2\', \'-o\',
\'compat=1.1\', \'/rhev/data-center/mnt/blockSD/94a5b646-b8e1-410d-9fd8-5d16d1d04f23/images/5faee0e8-4565-4827-8364-485ec6981163/7e56ad89-5639-4da3-852f-0f19340ef7f8\'] failed with rc=1 out=b\'\' err=bytearray(b
"qemu-img: Protocol driver \\\'host_device\\\' does not support image creation\\n")',)'}'}}, log id: 4ea5c056
2020-03-23 16:03:50,212+02 INFO [org.ovirt.engine.core.bll.StorageJobCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-81) [7197f14a-1170-401d-a133-c7cd767bfc04] Command CopyData id
: '103ee356-6513-46ec-989c-381a0cf994cd': job '93faaf0d-de20-47cf-be4a-d4397828551d' execution was completed with VDSM job status 'failed'
2020-03-23 16:03:50,221+02 INFO [org.ovirt.engine.core.bll.StorageJobCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-81) [7197f14a-1170-401d-a133-c7cd767bfc04] Command CopyData id
: '103ee356-6513-46ec-989c-381a0cf994cd': execution was completed, the command status is 'FAILED'
2020-03-23 16:03:51,231+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-92) [7197f14a-1170-401d-a133-c7cd767bfc04] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand' with failure.
2020-03-23 16:03:51,249+02 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-92) [7197f14a-1170-401d-a133-c7cd767bfc04] C
ommand 'CopyImageGroupVolumesData' id: 'e9ccdf13-b9a7-4aa4-b412-2255f169b7ce' child commands '[103ee356-6513-46ec-989c-381a0cf994cd]' executions were completed, status 'FAILED'
2020-03-23 16:03:52,258+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyImageGroupVolumesDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-58) [7197f14a-1170-401d-a133-
c7cd767bfc04] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.CopyImageGroupVolumesDataCommand' with failure.
Logs attached (vdsm + engine) .
Tell me if you need more information.
Targeting to 4.4.0 due to regression and blocker? flags isn't it the same thing as in https://bugzilla.redhat.com/show_bug.cgi?id=1820182 about measure? If so please move back to ON_QA and wait on the other bug (In reply to Michal Skrivanek from comment #11) > isn't it the same thing as in > https://bugzilla.redhat.com/show_bug.cgi?id=1820182 about measure? If so > please move back to ON_QA and wait on the other bug It is, these are two different bugs Verified on: vdsm-4.40.13-1.el8ev.x86_64 ovirt-engine-4.4.0-0.32.master.el8ev.noarch This bugzilla is included in oVirt 4.4.0 release, published on May 20th 2020. Since the problem described in this bug report should be resolved in oVirt 4.4.0 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report. |
Created attachment 1672546 [details] engine and vdsm logs with host/engine rpms Description of problem: Copy template disk to ISCSI or FCP storage domains fails. VDSM log: 2020-03-22 18:24:55,474+0200 ERROR (tasks/9) [root] Job '626b6b27-7a32-465d-ad8e-2e05d1ccdb2b' failed (jobs:223) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/jobs.py", line 159, in run self._run() File "/usr/lib/python3.6/site-packages/vdsm/storage/sdm/api/copy_data.py", line 86, in _run self._operation.run() File "/usr/lib/python3.6/site-packages/vdsm/storage/qemuimg.py", line 344, in run for data in self._operation.watch(): File "/usr/lib/python3.6/site-packages/vdsm/storage/operation.py", line 106, in watch self._finalize(b"", err) File "/usr/lib/python3.6/site-packages/vdsm/storage/operation.py", line 179, in _finalize raise cmdutils.Error(self._cmd, rc, out, err) vdsm.common.cmdutils.Error: Command ['/usr/bin/qemu-img', 'convert', '-p', '-t', 'none', '-T', 'none', '-f', 'qcow2', '/rhev/data-center/mnt/mantis-nfs-lif2.lab.eng.tlv2.redhat.com:_nas01_ge__6__nfs__3/c46e4bd3-af6d-48c8-8b4b-9719552657bf/images/9d35d3d7-84d8-4e89-a763-2ba265093a68/5014714d-1c52-4328-b7ec-15027f3c1119', '-O', 'qcow2', '-o', 'compat=1.1', '/rhev/data-center/mnt/blockSD/8c86ccb5-64a2-44cc-83c3-3051141fe808/images/9d35d3d7-84d8-4e89-a763-2ba265093a68/5014714d-1c52-4328-b7ec-15027f3c1119'] failed with rc=1 out=b'' err=bytearray(b"qemu-img: Protocol driver \'host_device\' does not support image creation\n") 2020-03-22 18:24:55,476+0200 INFO (tasks/9) [root] Job '626b6b27-7a32-465d-ad8e-2e05d1ccdb2b' will be deleted in 3600 seconds (jobs:251) Engine log: START, GetHostJobsVDSCommand(HostName = host_mixed_3, GetHostJobsVDSCommandParameters:{hostId='533aebaf-472a-495d-97e5-65d6a72e56a2', type='storage', jobIds='[626b6b27-7a32-465d-ad8e-2e05d1ccdb2b]'}), log id: 31cc87c6 2020-03-22 18:24:55,670+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHostJobsVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-88) [17627067-d718-49db-a5aa-21866467f236] FINISH, GetHostJobsVDSCommand, return: {626b6b27-7a32-465d-ad8e-2e05d1ccdb2b=HostJobInfo:{id='626b6b27-7a32-465d-ad8e-2e05d1ccdb2b', type='storage', description='copy_data', status='failed', progress='0', erro r='VDSError:{code='GeneralException', message='General Exception: ('Command [\'/usr/bin/qemu-img\', \'convert\', \'-p\', \'-t\', \'none\', \'-T\', \'none\', \'-f\', \'qcow2\', \'/rhev/data-center/mnt/mantis-nfs -lif2.lab.eng.tlv2.redhat.com:_nas01_ge__6__nfs__3/c46e4bd3-af6d-48c8-8b4b-9719552657bf/images/9d35d3d7-84d8-4e89-a763-2ba265093a68/5014714d-1c52-4328-b7ec-15027f3c1119\', \'-O\', \'qcow2\', \'-o\', \'compat=1. 1\', \'/rhev/data-center/mnt/blockSD/8c86ccb5-64a2-44cc-83c3-3051141fe808/images/9d35d3d7-84d8-4e89-a763-2ba265093a68/5014714d-1c52-4328-b7ec-15027f3c1119\'] failed with rc=1 out=b\'\' err=bytearray(b"qemu-img: Protocol driver \\\'host_device\\\' does not support image creation\\n")',)'}'}}, log id: 31cc87c6 2020-03-22 18:24:55,672+02 INFO [org.ovirt.engine.core.bll.StorageJobCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-88) [17627067-d718-49db-a5aa-21866467f236] Command CopyData i d: 'dbaeded3-6211-43da-9ee3-6aba85ee68e0': job '626b6b27-7a32-465d-ad8e-2e05d1ccdb2b' execution was completed with VDSM job status 'failed' 2020-03-22 18:24:55,674+02 INFO [org.ovirt.engine.core.bll.StorageJobCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-88) [17627067-d718-49db-a5aa-21866467f236] Command CopyData i d: 'dbaeded3-6211-43da-9ee3-6aba85ee68e0': execution was completed, the command status is 'FAILED' 2020-03-22 18:24:56,679+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-54) [17627067-d718-49db-a5aa-21866467f236] En ding command 'org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand' with failure. 2020-03-22 18:24:56,685+02 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-54) [17627067-d718-49db-a5aa-21866467f236] Command 'CopyImageGroupVolumesData' id: '80221db7-d130-41f5-80f9-0160394bb08a' child commands '[dbaeded3-6211-43da-9ee3-6aba85ee68e0]' executions were completed, status 'FAILED' 2020-03-22 18:24:57,688+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyImageGroupVolumesDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-19) [17627067-d718-49db-a5aa -21866467f236] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.CopyImageGroupVolumesDataCommand' with failure. 2020-03-22 18:24:57,694+02 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-19) [17627067-d718-49db-a5aa-21866467f236] Command 'CopyImageGroupWithData' id: 'd3e425dd-5ba5-4d84-bd30-d8d0ab66b1c2' child commands '[0420662b-04c2-4964-aff7-c362da6a5f47, 80221db7-d130-41f5-80f9-0160394bb08a]' executions were completed, status 'FAILE D' 2020-03-22 18:24:58,697+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyImageGroupWithDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-50) [17627067-d718-49db-a5aa-21866467f236] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.CopyImageGroupWithDataCommand' with failure. Version-Release number of selected component (if applicable): ovirt-engine-4.4.0-0.26.master.el8ev.noarch vdsm-4.40.7-1.el8ev.x86_64 How reproducible: 100% Steps to Reproduce:(done via ansible) 1.Clean setup after installation of RHEL8.2+RHV. 2.Create new DC+cluster + add hosts + create 9 SD's( 3XNFS, 3Xgluster, 3XISCSI, 3XFCP) 3. Import templates(RHEL8.2 image) from glance as a temelate to NFS SD 'nfs_0' 4. Copy template disk from 'nfs_0' to NFS('nfs_1','nfs_2') or GLuster SD's('gluster_0', 'gluster_1','gluster_2') works. 5. Copy template disk to ISCSI SD's('iscsi_0','iscsi_2','iscsi_2') or FCP SD's('fcp_0','fcp_1','fcp_2') Actual results: Copy template disk to ISCSI SD's('iscsi_0','iscsi_2','iscsi_2') or FCP SD's('fcp_0','fcp_1','fcp_2') fails. Expected results: Additional info: