Bug 1091030

Summary: Failed to create disk as no links exists
Product: Red Hat Enterprise Virtualization Manager Reporter: Kevin Alon Goldblatt <kgoldbla>
Component: vdsmAssignee: Federico Simoncelli <fsimonce>
Status: CLOSED ERRATA QA Contact: Elad <ebenahar>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.4.0CC: acanan, adahms, amureini, bazulay, fsimonce, gklein, iheim, kgoldbla, lpeer, scohen, tnisan, yeylon
Target Milestone: ---   
Target Release: 3.4.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: storage
Fixed In Version: av9 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-09 13:30:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1102906    
Attachments:
Description Flags
vdsm and engine logs
none
full vdsm log as requested none

Description Kevin Alon Goldblatt 2014-04-24 16:34:42 UTC 
Description of problem:
Failed to add a disk to a virtual machine as the link does not exist on the host

Version-Release number of selected component (if applicable):
rhevm-3.4.0-0.12.beta2.el6ev.noarch
vdsm-4.14.6-0.1.beta3.el6ev.x86_64


How reproducible:
reproduced this 3 times in succession

Steps to Reproduce:
1. Via the Rhevm manager - Virtual Machines - Disks - Add new disk
2. Size 50g, preallocated, nfs storage domain - ok
3. Failes to create the disk


Actual results:
The disk is not created. The link to the mount point on the host does not exist:


Expected results:

The disk should be created

Additional info:
From engine.log---------
2014-04-24 18:21:49,112 INFO  [org.ovirt.engine.core.bll.CommandAsyncTask] (DefaultQuartzScheduler_Worker-13) CommandAsyncTask::EndActionIfNecessar
y: All tasks of command 6a6ce499-9b58-40cb-a0fc-d75a1e5bff77 has ended -> executing endAction
2014-04-24 18:21:49,112 INFO  [org.ovirt.engine.core.bll.CommandAsyncTask] (DefaultQuartzScheduler_Worker-13) CommandAsyncTask::endAction: Ending a
ction for 1 tasks (command ID: 6a6ce499-9b58-40cb-a0fc-d75a1e5bff77): calling endAction .
2014-04-24 18:21:49,112 INFO  [org.ovirt.engine.core.bll.CommandAsyncTask] (org.ovirt.thread.pool-4-thread-23) CommandAsyncTask::EndCommandAction [
within thread] context: Attempting to endAction AddDisk, executionIndex: 0
2014-04-24 18:21:49,122 ERROR [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-4-thread-23) [1a306cc3] Ending command with failure
: org.ovirt.engine.core.bll.AddDiskCommand
2014-04-24 18:21:49,127 ERROR [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-4-thread-23) [2f3f15e6] Ending command with failure: org.ovirt.engine.core.bll.AddImageFromScratchCommand
2014-04-24 18:21:49,343 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-4-thread-23) Correlation ID: 1a306cc3, Job ID: e1a04e0f-ac43-45cc-a01d-64d010c03626, Call Stack: null, Custom Event ID: -1, Message: Operation Add-Disk failed to complete.
Comment 1 Kevin Alon Goldblatt 2014-04-27 07:42:46 UTC 
Created attachment 890176 [details]
vdsm and engine logs
Comment 2 Allon Mureinik 2014-04-27 14:48:26 UTC 
Kevin, the correlation ID in the snippet you provided is missing from the VDSM log.
Can you please provide the full log?
Comment 3 Allon Mureinik 2014-04-27 14:50:25 UTC 
Fede, we're getting too many bugs about missing links (e.g. bug 1069772).
Can you please take a look?
Comment 4 Allon Mureinik 2014-04-27 15:01:40 UTC 
(In reply to Allon Mureinik from comment #3)
> Fede, we're getting too many bugs about missing links (e.g. bug 1069772).
More appropriately, this seems like a dup of bug 1086210 ?
Comment 5 Kevin Alon Goldblatt 2014-04-27 16:15:06 UTC 
Created attachment 890241 [details]
full vdsm log as requested

Added the full vdsm log that was required
Comment 6 Federico Simoncelli 2014-05-02 22:09:01 UTC 
It is very important to know if this is a regression or not. Can you please reproduce this on 3.3? Please contact me so that we can verify this together. Thanks.
Comment 7 Federico Simoncelli 2014-05-07 12:58:15 UTC 
Probably a duplicate of bug 1086210
Comment 8 Allon Mureinik 2014-05-07 13:11:28 UTC 
(In reply to Federico Simoncelli from comment #7)
> Probably a duplicate of bug 1086210

Moving to MODIFIED based on this statement.
Comment 9 Kevin Alon Goldblatt 2014-05-14 16:01:27 UTC 
I was not able to reproduce this defect in 3.3. This does not mean that the defect doesn't exist in 3.3 as I do not have the specific instance that this problem occurs.
Comment 10 Elad 2014-05-18 12:30:31 UTC 
Verified according to steps from https://bugzilla.redhat.com/show_bug.cgi?id=1086210 :

2 NFS domains in DC:
1) Created a VM with a disk located on the masted domain, started it
2) Blocked connectivity form SPM to master domain, waited for reconstruct to take place
3) Once the other domain took master, destroyed the VM
4) Resumed connectivity to the first domain
5) Started the VM


The VM was started normally, the link to the mount of the storage domain re-appeared under /rhev/data-ceter/SPUUID. Tested 3 times


Thread-220::INFO::2014-05-18 15:22:27,387::sp::1113::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_elad_6/4b73f56c-a54a-4f81-b9b2-010cc1b5904e to /rhev/data-center/4aa2760a-c779-4b5c-93aa-8aafd334aeb1/4b73f56c-a54a-4f81-b9b2-010cc1b5904e



As this scenario didn't cause the issue of missing links to reproduce, I'm moving this bug to VERIFIED. 
 
Verified using av9.1
Comment 12 errata-xmlrpc 2014-06-09 13:30:22 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0504.html