Description of problem: In the REST API, the disk_attachment href contains "null" instead of "diskattachments". For example (from art log): 2018-11-01 06:04:02,332 - MainThread - diskattachments - DEBUG - GET request content is -- url:/ovirt-engine/api/vms/69739da6-21a5-4064-a478-a223c1763720/diskattachments 2018-11-01 06:04:02,352 - MainThread - diskattachments - DEBUG - Response code is valid: [200, 201] 2018-11-01 06:04:02,352 - MainThread - diskattachments - DEBUG - Response body for GET request is: <disk_attachments> <disk_attachment href="/ovirt-engine/api/vms/69739da6-21a5-4064-a478-a223c1763720/null/d75fc53b-6fd3-459f-9c9e-cf6d29beec04" id="d75fc53b-6fd3-459f-9c9e-cf6d29beec04"> <active>true</active> <bootable>true</bootable> <interface>virtio</interface> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/d75fc53b-6fd3-459f-9c9e-cf6d29beec04" id="d75fc53b-6fd3-459f-9c9e-cf6d29beec04"/> <vm href="/ovirt-engine/api/vms/69739da6-21a5-4064-a478-a223c1763720" id="69739da6-21a5-4064-a478-a223c1763720"/> </disk_attachment> <disk_attachment href="/ovirt-engine/api/vms/69739da6-21a5-4064-a478-a223c1763720/null/afcc5f9d-9bef-458f-8503-fff2c6399761" id="afcc5f9d-9bef-458f-8503-fff2c6399761"> <active>true</active> <bootable>false</bootable> <interface>virtio</interface> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/afcc5f9d-9bef-458f-8503-fff2c6399761" id="afcc5f9d-9bef-458f-8503-fff2c6399761"/> <vm href="/ovirt-engine/api/vms/69739da6-21a5-4064-a478-a223c1763720" id="69739da6-21a5-4064-a478-a223c1763720"/> </disk_attachment> </disk_attachments> Here the href is: href="/ovirt-engine/api/vms/69739da6-21a5-4064-a478-a223c1763720/null/d75fc53b-6fd3-459f-9c9e-cf6d29beec04" Therefore, operation such as remove/move disk failed with error 404 (Not Found) Version-Release number of selected component (if applicable): ovirt-engine-4.3.0-0.0.master.20181030100636.git6215452.el7.noarch How reproducible: Happens during automation execution. Didn't manage to reproduce it again using the same test cases. Steps to Reproduce: 1. 2. 3. Actual results: The href contained "null" and caused failures. Expected results: There should be "diskattachments" instead of "null" Additional info: From the engine log: 2018-11-01 06:04:01,654+02 WARN [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-81) [] Can't find relative path for class "org.ovirt.engine.api.resource.StorageDomainVmDiskAttachmentsResource", will return null * This warning appears a lot in the logs
Ondra, I think this is more of an infra issue, can you have a look please? Seems strange to me that it fails arbitrarily
This bug report has Keywords: Regression or TestBlocker. Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
Just from looking at the code, I see no reason for this error. So I would need to have a reproduction scenario, to go any further. Slight elaboration: the meaning of the log warning is that no @Path annotation exists on StorageDomainVmResource.getDiskAttachmentsResource(). But in fact there is @Path("diskattachments") annotation there. So I see no reason why this warning would be issued, and must have a scenario to reproduce the problem
(In reply to Ori Liel from comment #3) > Just from looking at the code, I see no reason for this error. So I would > need to have a reproduction scenario, to go any further. > > Slight elaboration: the meaning of the log warning is that no @Path > annotation exists on StorageDomainVmResource.getDiskAttachmentsResource(). > But in fact there is @Path("diskattachments") annotation there. So I see no > reason why this warning would be issued, and must have a scenario to > reproduce the problem This bug failed while running multiple automation test cases on the version mention in this bug description. The requests for disk_attachments via REST API returned the href contains the "null". I don't have a reproduction scenario since this bug works fine on 4.3.0-0.0.master.20181101091940.git61310aa.el7 Maybe it has been fixed between these versions. Currently, it works ok.
In that case, I suggest that we close the bug, until it resurfaces with a clearer reproduction scenario (hopefully it won't)
(In reply to Ori Liel from comment #5) > In that case, I suggest that we close the bug, until it resurfaces with a > clearer reproduction scenario (hopefully it won't) I agree with you. I'm Closing this bug for now.
Happened again in few automation TC: Version-Release number of selected component (if applicable): ovirt-engine-4.3.0-0.5.alpha1.el7.noarch 2018-11-24 17:23:06,640 - ThreadPoolExecutor-34_0 - diskattachments - DEBUG - Response body for GET request is: <disk_attachments> <disk_attachment href="/ovirt-engine/api/vms/c9a9229e-2209-41a0-ad73-b9ff60cdb0cc/null/8b67285e-bec5-4af9-b4ae-f363853d8332" id="8b67285e-bec5-4af9-b4ae-f363853d8332"> <active>true</active> <bootable>true</bootable> <interface>virtio</interface> <logical_name>/dev/vda</logical_name> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/8b67285e-bec5-4af9-b4ae-f363853d8332" id="8b67285e-bec5-4af9-b4ae-f363853d8332"/> <vm href="/ovirt-engine/api/vms/c9a9229e-2209-41a0-ad73-b9ff60cdb0cc" id="c9a9229e-2209-41a0-ad73-b9ff60cdb0cc"/> </disk_attachment> 2018-11-24 17:23:06,643 - ThreadPoolExecutor-34_0 - diskattachments - DEBUG - PUT request content is -- url:/ovirt-engine/api/vms/c9a9229e-2209-41a0-ad73-b9ff60cdb0cc/null/5137e5a0-082e-4083-895e-792dd2084113 body:<disk_attachment> <active>true</active> <disk/> </disk_attachment> 2018-11-24 17:23:06,660 - ThreadPoolExecutor-34_0 - api_utils - ERROR - Failed to update element NOT as expected: Status: 404 Reason: Not Found
Created attachment 1508767 [details] logs
Could you please provide reproducing steps? Ori already tried to investigate the issue, but failed to find out the cause, so we cannot proceed without reproducing steps
I am also seeing it on master: /ovirt-engine/api/v4/storagedomains/c0a6cc30-4626-4ca6-af47-c29d166d10ff/disks <disks><disk href="/ovirt-engine/api/v4/storagedomains/c0a6cc30-4626-4ca6-af47-c29d166d10ff/null/a025a26e-fe2b-4db6-9912-84390e31707f" id="a025a26e-fe2b-4db6-9912-84390e31707f"> I can show Ori if needed.
(In reply to Martin Perina from comment #9) > Could you please provide reproducing steps? Ori already tried to investigate > the issue, but failed to find out the cause, so we cannot proceed without > reproducing steps I think this will help understand where this issue exists - https://bugzilla.redhat.com/show_bug.cgi?id=1646375 Raising severity to urgent as our automation is broken
I can hypothesize by https://bugzilla.redhat.com/show_bug.cgi?id=1646375 (thanks Raz) that when a diskAttachment object has both vm-id and template-id set in it, the ambiguity results in a problem when creating the link. However I'm not sure what would lead to such a case; I've been playing around, creating vms from templates which have a disk attached and vica-versa and still everything works for me. I can write a patch which prevents even the potential ambiguity, but I have no way of verifying it because I still haven't found a reproduction scenario. I'll write the patch and let's try running the automation tests with it. But again, the best way would be to find an exact reproduction scenario
Removing need info as I believe it was answered
This happens under storage-domains, there it can be reproduced, see: https://bugzilla.redhat.com/show_bug.cgi?id=1660062
Tested using: ovirt-engine-4.3.0-0.8.rc2.el7.noarch The "null" still exists. For example, using REST API: https://<engine_url>/ovirt-engine/api/vms/<vm_id>/diskattachments returns: <disk_attachments> <disk_attachment href="/ovirt-engine/api/vms/03372811-6113-4a35-a1f5-7a6e74fc9c55/null/2f8e7667-7331-4494-a31d-f85dd54545d9" id="2f8e7667-7331-4494-a31d-f85dd54545d9"> <active>true</active> <bootable>true</bootable> <interface>virtio</interface> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/2f8e7667-7331-4494-a31d-f85dd54545d9" id="2f8e7667-7331-4494-a31d-f85dd54545d9"/> <vm href="/ovirt-engine/api/vms/03372811-6113-4a35-a1f5-7a6e74fc9c55" id="03372811-6113-4a35-a1f5-7a6e74fc9c55"/> </disk_attachment> </disk_attachments> Moving back to ASSIGNED
I really need a reproduction scenario. I just did the following on latest upstream build: In WebAdmin: 1) Create a VM 2) In create-VM dialogue: 2.1) Create a disk for this VM: (alias='Vm1_Disk1') 2.2) Attach a previously created disk (alias='disk_3') 3) GET http://localhost:8080/ovirt-engine/api/vms/9d9ec888-b81e-4879-a6ef-0bb45cc0ab7e/diskattachments And the result was: <disk_attachments> <disk_attachment href="/ovirt-engine/api/vms/9d9ec888-b81e-4879-a6ef-0bb45cc0ab7e/diskattachments/14c4ee82-5f6e-41b3-9da0-17e0cd469a96" id="14c4ee82-5f6e-41b3-9da0-17e0cd469a96"> <active>true</active> <bootable>true</bootable> <interface>virtio_scsi</interface> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/14c4ee82-5f6e-41b3-9da0-17e0cd469a96" id="14c4ee82-5f6e-41b3-9da0-17e0cd469a96"/> <vm href="/ovirt-engine/api/vms/9d9ec888-b81e-4879-a6ef-0bb45cc0ab7e" id="9d9ec888-b81e-4879-a6ef-0bb45cc0ab7e"/> </disk_attachment> <disk_attachment href="/ovirt-engine/api/vms/9d9ec888-b81e-4879-a6ef-0bb45cc0ab7e/diskattachments/14639765-ef36-4f89-a278-b69ccbef2607" id="14639765-ef36-4f89-a278-b69ccbef2607"> <active>true</active> <bootable>false</bootable> <interface>virtio</interface> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/14639765-ef36-4f89-a278-b69ccbef2607" id="14639765-ef36-4f89-a278-b69ccbef2607"/> <vm href="/ovirt-engine/api/vms/9d9ec888-b81e-4879-a6ef-0bb45cc0ab7e" id="9d9ec888-b81e-4879-a6ef-0bb45cc0ab7e"/> </disk_attachment> </disk_attachments> As you can see no 'null' in the href or anywhere. Can you please tell me what I need to change in the reproduction scenario, or once again close this bug. P.S - I get it that this happens, I'm not disputing that at all; it's just that I'm helpless without a reproduction scenario.
No failures/regressions related to this issue found in the last executions. Moving to VERIFIED for now.
This bugzilla is included in oVirt 4.3.1 release, published on February 28th 2019. Since the problem described in this bug report should be resolved in oVirt 4.3.1 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.
The issue resurfaced at 4.3.3.3 in Tier2 run TestCase18975 with the same scenario as original scenario(Create VM from template and start it, create some disks, attach disks). ovirt-engine-4.3.3.3-0.1.el7.noarch vdsm-4.30.13-1.el7ev.x86_64 Scenario with timestamps: 2019-04-13 01:30:25,430 INFO Test Setup 1: Creating VM vm_TestCase18975_1301302542 2019-04-13 01:30:25,612 INFO Test Setup 2: [class] Description: clone vm from a pre-defined template golden_template_default_rhel8.0 with {'vol_sparse': True, 'name': 'vm_TestCase18975_1301302542', 'clone': False, 'cluster': 'golden_env_mixed_1', 'storagedomain': 'fcp_2', 'display_type': 'spice', 'timeout': 900, 'virtio_scsi': True, 'vol_format': 'cow', 'os_type': 'RHEL6x64', 'type': 'desktop', 'wait': True} 2019-04-13 01:30:38,202 INFO Test Setup 3: Starting VM vm_TestCase18975_1301302542 2019-04-13 01:30:38,206 INFO Test Setup 4: [class] Start VM vm_TestCase18975_1301302542 with {'wait_for_ip': True, 'pause': False, 'use_cloud_init': False, 'timeout': 600, 'wait_for_status': 'up'} 2019-04-13 01:36:02,267 INFO Test Setup 5: Creating disks with filesystem and attach to VM vm_TestCase18975_1301302542 Engine log: 2019-04-13 01:30:35,968+03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [] EVENT_ID: USER_ADD_VM_FINISHED_SUCCESS(53), VM vm_T estCase18975_1301302542 creation has been completed. 2019-04-13 01:30:37,267+03 WARN [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-4) [] Can't find relative path for class "org.ovirt.engine.api.resource.StorageDomainVmDiskAttachmentsResource", wil l return null 2019-04-13 01:30:37,267+03 WARN [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-4) [] Can't find relative path for class "org.ovirt.engine.api.resource.StorageDomainVmDiskAttachmentsResource", wil l return null 2019-04-13 01:30:37,452+03 WARN [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-4) [] Can't find relative path for class "org.ovirt.engine.api.resource.StorageDomainVmDiskAttachmentsResource", wil l return null 2019-04-13 01:30:37,453+03 WARN [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-4) [] Can't find relative path for class "org.ovirt.engine.api.resource.StorageDomainVmDiskAttachmentsResource", will return null 2019-04-13 01:30:37,638+03 WARN [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-4) [] Can't find relative path for class "org.ovirt.engine.api.resource.StorageDomainVmDiskAttachmentsResource", will return null 2019-04-13 01:30:37,638+03 WARN [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-4) [] Can't find relative path for class "org.ovirt.engine.api.resource.StorageDomainVmDiskAttachmentsResource", will return null art(automarion log): 2019-04-13 01:30:37,642 - MainThread - diskattachments - DEBUG - PUT request content is -- url:/ovirt-engine/api/vms/af252c72-3e49-4bff-8c68-71618c78e15f/diskattachments/e5e80e98-ae23-444f-86cc-ca8c5c3400bb body:<disk_attachment> <bootable>true</bootable> <interface>virtio</interface> <disk> <alias>vm_TestCase18975_1301302542_Disk_0</alias> </disk> </disk_attachment> 2019-04-13 01:30:37,643 - MainThread - diskattachments - INFO - Using Correlation-Id: diskattachments_update_4efdc222-98ad-41c2 2019-04-13 01:30:38,197 - MainThread - core_api - DEBUG - Request PUT response time: 0.010 2019-04-13 01:30:38,197 - MainThread - diskattachments - DEBUG - Cleaning Correlation-Id: diskattachments_update_4efdc222-98ad-41c2 2019-04-13 01:30:38,198 - MainThread - diskattachments - DEBUG - Response code is valid: [200, 201] 2019-04-13 01:30:38,198 - MainThread - diskattachments - DEBUG - Response body for PUT request is: <disk_attachment href="/ovirt-engine/api/vms/af252c72-3e49-4bff-8c68-71618c78e15f/null/e5e80e98-ae23-444f-86cc-ca8c5c3400bb" id="e5e80e98-ae23-444f-86cc-ca8c5c3400bb"> <active>true</active> <bootable>true</bootable> <interface>virtio</interface> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/e5e80e98-ae23-444f-86cc-ca8c5c3400bb" id="e5e80e98-ae23-444f-86cc-ca8c5c3400bb"/> <vm href="/ovirt-engine/api/vms/af252c72-3e49-4bff-8c68-71618c78e15f" id="af252c72-3e49-4bff-8c68-71618c78e15f"/> </disk_attachment> This appears multiple times during the test: [~/Downloads]$ less art_test_runner.log | grep null <disk_attachment href="/ovirt-engine/api/vms/af252c72-3e49-4bff-8c68-71618c78e15f/null/e5e80e98-ae23-444f-86cc-ca8c5c3400bb" id="e5e80e98-ae23-444f-86cc-ca8c5c3400bb"> <disk_attachment href="/ovirt-engine/api/vms/af252c72-3e49-4bff-8c68-71618c78e15f/null/b0ae34af-422f-458e-9e40-974502d222e7" id="b0ae34af-422f-458e-9e40-974502d222e7"> <disk_attachment href="/ovirt-engine/api/vms/af252c72-3e49-4bff-8c68-71618c78e15f/null/18bd572f-688f-45e6-9c24-4c77c5a6d44f" id="18bd572f-688f-45e6-9c24-4c77c5a6d44f"> <disk_attachment href="/ovirt-engine/api/vms/af252c72-3e49-4bff-8c68-71618c78e15f/null/a07bbc0d-04da-48a0-9bc9-446fe02e0a4b" id="a07bbc0d-04da-48a0-9bc9-446fe02e0a4b"> <disk_attachment href="/ovirt-engine/api/vms/af252c72-3e49-4bff-8c68-71618c78e15f/null/15e6fb44-b5f1-4414-b5e3-cbf3386b3c18" id="15e6fb44-b5f1-4414-b5e3-cbf3386b3c18">
Created attachment 1555030 [details] 4.3.3.3_reproduction_logs
Issue also seen in 4.3.3.1 (ran at ~2019-03-30) so maybe this issue was not resolved after all. We missed it as it did not interfere with normal operations in automation and TC's passed.
Created attachment 1555031 [details] engine log with the issue at ovirt-engine 4.3.3.1 build
Moving to 4.3.4 until it gets clear if it's a new issue or verified improperly
Happened again in 4.3.3.8 The test case passed but failed to delete an image as there was null in the URL. <disk_attachment href="/ovirt-engine/api/vms/8695c5db-76e9-4f65-aa30-4517b73f84fe/null/df11ea87-dafe-42e0-b2ea-d716fa32428b" id="df11ea87-dafe-42e0-b2ea-d716fa32428b"> <active>true</active> <bootable>false</bootable> <interface>virtio</interface> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/df11ea87-dafe-42e0-b2ea-d716fa32428b" id="df11ea87-dafe-42e0-b2ea-d716fa32428b"/> <vm href="/ovirt-engine/api/vms/8695c5db-76e9-4f65-aa30-4517b73f84fe" id="8695c5db-76e9-4f65-aa30-4517b73f84fe"/> </disk_attachment> Version: ovirt-engine-4.3.3.6-0.1.el7.noarch * Attaching relevant logs
Created attachment 1564293 [details] 4.3.3.8 logs
The problem is no longer with getting (GET) disk-attachments. That was fixed successfully by https://gerrit.ovirt.org/#/c/97250/. This is an additional problem, with update (PUT) flow. After updating a disk-attachment, the href of the returned disk-attachment contains 'null'. Patch posted: https://gerrit.ovirt.org/#/c/99945/
Tested using: ovirt-engine-4.3.5.2-0.1.el7.noarch I executed PUT request via REST API to: https://<engine-url>/ovirt-engine/api/vms/96805272-cadb-4fc4-bef8-776558d7c004/diskattachments/33e108a6-0f93-4d48-86c0-a4004d577ec9 with: <disk_attachment> <disk> <name>mydisk</name> </disk> </disk_attachment> The response was: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <disk_attachment href="/ovirt-engine/api/vms/96805272-cadb-4fc4-bef8-776558d7c004/diskattachments/33e108a6-0f93-4d48-86c0-a4004d577ec9" id="33e108a6-0f93-4d48-86c0-a4004d577ec9"> <active>true</active> <bootable>true</bootable> <interface>virtio_scsi</interface> <pass_discard>false</pass_discard> <read_only>false</read_only> <uses_scsi_reservation>false</uses_scsi_reservation> <disk href="/ovirt-engine/api/disks/33e108a6-0f93-4d48-86c0-a4004d577ec9" id="33e108a6-0f93-4d48-86c0-a4004d577ec9"/> <vm href="/ovirt-engine/api/vms/96805272-cadb-4fc4-bef8-776558d7c004" id="96805272-cadb-4fc4-bef8-776558d7c004"/> </disk_attachment> Moving to VERIFIED
This bugzilla is included in oVirt 4.3.5 release, published on July 30th 2019. Since the problem described in this bug report should be resolved in oVirt 4.3.5 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.