Bug 2075435

Summary: Hybrid Backup - backup href has changed and causing backups to get stuck in finalizing stage
Product: [oVirt] ovirt-engine Reporter: Evelina Shames <eshames>
Component: BLL.StorageAssignee: Benny Zlotnik <bzlotnik>
Status: CLOSED CURRENTRELEASE QA Contact: Evelina Shames <eshames>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.5.0CC: aefrat, ahadas, bugs, bzlotnik
Target Milestone: ovirt-4.5.0-1Keywords: AutomationBlocker, Regression, TestBlocker
Target Release: 4.5.0.5Flags: sbonazzo: ovirt-4.5+
ahadas: blocker-
sbonazzo: exception+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-engine-4.5.0.5 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-03 06:46:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2070184    

Description Evelina Shames 2022-04-14 08:19:35 UTC
Description of problem:
As a result of fixing bug 2070184, href has changed and it causing backups to get stuck in finalizing stage:
/ovirt-engine/api/vms/05f2931e-65da-421a-a31d-0cd12fe7415f/snapshots/1adf1366-18db-4ed4-af62-850ccbf03472/null/43d626dd-8dcd-422d-aec5-a88b6364edf2

instead of:
/ovirt-engine/api/vms/05f2931e-65da-421a-a31d-0cd12fe7415f/backups/43d626dd-8dcd-422d-aec5-a88b6364edf2/disks

<backups>
<backup href="/ovirt-engine/api/vms/05f2931e-65da-421a-a31d-0cd12fe7415f/snapshots/1adf1366-18db-4ed4-af62-850ccbf03472/null/43d626dd-8dcd-422d-aec5-a88b6364edf2" id="43d626dd-8dcd-422d-aec5-a88b6364edf2">
<actions>
<link href="/ovirt-engine/api/vms/05f2931e-65da-421a-a31d-0cd12fe7415f/snapshots/1adf1366-18db-4ed4-af62-850ccbf03472/null/43d626dd-8dcd-422d-aec5-a88b6364edf2/finalize" rel="finalize"/>
</actions>
<link href="/ovirt-engine/api/vms/05f2931e-65da-421a-a31d-0cd12fe7415f/snapshots/1adf1366-18db-4ed4-af62-850ccbf03472/null/43d626dd-8dcd-422d-aec5-a88b6364edf2/disks" rel="disks"/>
<creation_date>2022-04-13T17:23:44.084+03:00</creation_date>
<modification_date>2022-04-13T17:23:53.900+03:00</modification_date>
<phase>ready</phase>
<to_checkpoint_id>148232f7-cb16-4afb-9efb-60b0d762d844</to_checkpoint_id>
<snapshot href="/ovirt-engine/api/vms/05f2931e-65da-421a-a31d-0cd12fe7415f/snapshots/1adf1366-18db-4ed4-af62-850ccbf03472" id="1adf1366-18db-4ed4-af62-850ccbf03472"/>
<vm href="/ovirt-engine/api/vms/05f2931e-65da-421a-a31d-0cd12fe7415f" id="05f2931e-65da-421a-a31d-0cd12fe7415f"/>
</backup>
</backups>


Version-Release number of selected component (if applicable):
ovirt-engine-4.5.0.2-0.7.el8ev

How reproducible:
Not sure yet, happened on 2 different clean envs

Steps to Reproduce:
currently unknown

Actual results:
the new hfer leads to nothing, backup gets stuck in finalizing stage

Expected results:
the href should be /ovirt-engine/api/vms/<vm-id>/backups/<backup-id>/disks

Additional info:

Comment 1 RHEL Program Management 2022-04-14 08:27:43 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 2 Benny Zlotnik 2022-04-14 10:52:30 UTC
Because the suggestedParent parent

Comment 3 Benny Zlotnik 2022-04-14 10:56:56 UTC
(In reply to Benny Zlotnik from comment #2)
> Because the suggestedParent parent

Accidentally sent before completing the comment...

Because the suggestedParent in[1] was missing, LinkHelper tried to assign a parent using the endpoints methods, and there were two matches getVm() and getSnapshot(), so if getVm() was the first match, the links would be rendered correctly, and will be malformed otherwise. The methods are cached, so the order would remain the same until ovirt engine is restarted, which is why it happened consistently only on some environments but not on others and why restarting ovirt engine resolved it.



[1] https://github.com/oVirt/ovirt-engine/blob/e2536dc1a80593a0feb843925fdfd131eeb1b4bc/backend/manager/modules/restapi/jaxrs/src/main/java/org/ovirt/engine/api/restapi/resource/BackendVmBackupsResource.java#L61

Comment 4 Arik 2022-04-14 12:56:14 UTC
Summarizing here what we discussed elsewhere:
1. It didn't affect the SDK and therefore OST and an internal deployment of Veeam were not affected by this
2. Acked it as a blocker before it affected QE automation

Comment 5 Arik 2022-04-14 12:57:13 UTC
2. Acked it as a blocker *because* it affected QE automation

Comment 6 Evelina Shames 2022-04-25 12:24:49 UTC
Seen again in one of our automation runs for rhv-4.5.0-7, Benny investigated it and was able to reproduce it.
Moving back to 'Assigned'.

Comment 7 RHEL Program Management 2022-04-25 12:24:57 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 8 Arik 2022-04-25 12:28:16 UTC
It doesn't seem to affect backup vendors as we've seen that the Veeam backup application works and our tests pass
It affects clients that use the API directly, like QE automation, but with the last attempt to change that, it happens much less
Therefore, moving to 4.5.0-1 and setting blocker-

Comment 9 Evelina Shames 2022-05-02 07:26:30 UTC
As there is no reproducer for this issue, and it wasn't seen in our automation + manual tests for rhv-4.5.0-8, it looks like the issue was solved.
Moving to 'Verified'.