Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1130038

Summary: prepareImage api call fails with [Errno 2] No such file or directory
Product: [Retired] oVirt Reporter: Jiri Moskovcak <jmoskovc>
Component: vdsmAssignee: Federico Simoncelli <fsimonce>
Status: CLOSED NOTABUG QA Contact: Gil Klein <gklein>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 3.5CC: acathrow, amureini, bazulay, bugs, dfediuck, ecohen, fsimonce, gklein, iheim, jmoskovc, mavital, mgoldboi, sbonazzo, yeylon
Target Milestone: ---   
Target Release: 3.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: sla
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1129261 Environment:
Last Closed: 2014-08-25 15:30:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1067162, 1073943    
Attachments:
Description Flags
logs and test none

Description Jiri Moskovcak 2014-08-14 08:25:53 UTC
+++ This bug was initially created as a clone of Bug #1129261 +++

Description of problem:
when hosted engine broker tries to connect it's storage using vdsm it fail with: [Errno 2] No such file or directory

Version-Release number of selected component (if applicable):
vdsm-4.16.1-6.gita4a4614.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1. install hosted-engine
2. reboot the machine
3. check if the hosted engine storage is up'n'running
4. /rhev/data-center/mnt/blockSD/3498465f-eb04-49f3-81d2-f7713b392e5a/ha_agent/hosted-engine.metadata points to volume which is not activated

Actual results:
ha-agent doesn't run

Expected results:
storage connected, ha-agent running

Additional info:
vdsm log attached

--- Additional comment from Jiri Moskovcak on 2014-08-12 08:28:11 EDT ---

after additional testing, it seems like it actually activates the lvm device, but fails to create some symlink and returns non-zero error code which makes the he-broker think that it fails even though the storage is actually available.

--- Additional comment from Dan Kenigsberg on 2014-08-12 10:21:50 EDT ---

Thread-1205::ERROR::2014-08-12 14:13:03,605::blockSD::1054::Storage.StorageDomain::(linkBCImage) Failed to create path to image directory: /rhev/data-center/84cfc3d9-03cf-4ec5-9f29-a18ba1f6f02f/3498465f-eb04-49f3-81d2-f7713b392e5a/images/04f6b652-5e14-40d8-973c-ca384ffc0c50
Thread-1205::ERROR::2014-08-12 14:13:03,605::task::866::Storage.TaskManager.Task::(_setError) Task=`0ffda0cc-8afc-46bb-8b80-49e0a4291a68`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 3213, in prepareImage
    runImgPath = dom.linkBCImage(imgPath, imgUUID)
  File "/usr/share/vdsm/storage/blockSD.py", line 1047, in linkBCImage
    os.symlink(imgPath, dst)
OSError: [Errno 2] No such file or directory

--- Additional comment from Federico Simoncelli on 2014-08-13 11:52:06 EDT ---

The request fails because you're not connected to a pool. To prepare an image when you're not connected to a pool (general HE use case) you should specify a blank pool uuid (00000000-0000-0000-0000-000000000000). E.g.:

prepareImage(sdUUID='3498465f-eb04-49f3-81d2-f7713b392e5a', spUUID='00000000-0000-0000-0000-000000000000', imgUUID='e3449f52-b227-4666-a4d5-9fdc56f000ce', leafUUID='04f6b652-5e14-40d8-973c-ca384ffc0c50')

--- Additional comment from Jiri Moskovcak on 2014-08-14 03:33:25 EDT ---

(In reply to Federico Simoncelli from comment #3)
> The request fails because you're not connected to a pool. To prepare an
> image when you're not connected to a pool (general HE use case) you should
> specify a blank pool uuid (00000000-0000-0000-0000-000000000000). E.g.:
> 
> prepareImage(sdUUID='3498465f-eb04-49f3-81d2-f7713b392e5a',
> spUUID='00000000-0000-0000-0000-000000000000',
> imgUUID='e3449f52-b227-4666-a4d5-9fdc56f000ce',
> leafUUID='04f6b652-5e14-40d8-973c-ca384ffc0c50')

- thanks for the hint, going to try that, btw, I'm pretty sure it used to work back in May when we start using vdsm api for storage creation.

--- Additional comment from Jiri Moskovcak on 2014-08-14 03:47:59 EDT ---

btw, does the same apply for getVolumePath ?

--- Additional comment from Jiri Moskovcak on 2014-08-14 04:06:09 EDT ---

This is what I get if I use blank pool:

Thread-199::ERROR::2014-08-14 10:04:35,846::task::866::Storage.TaskManager.Task::(_setError) Task=`95689457-eec0-405c-8d0c-3f81e2556121`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 3237, in prepareImage
    return {'path': leafPath, 'info': leafInfo,
UnboundLocalError: local variable 'leafInfo' referenced before assignment

Comment 1 Jiri Moskovcak 2014-08-21 07:45:16 UTC
Created attachment 929080 [details]
logs and test

Comment 2 Federico Simoncelli 2014-08-25 09:54:56 UTC
Please report the output of:

 # lvs -o +tags

Comment 3 Federico Simoncelli 2014-08-25 10:15:45 UTC
(In reply to Federico Simoncelli from comment #2)
> Please report the output of:
> 
>  # lvs -o +tags

No need for that I just saw that you inverted imgUUID and volUUID:

conn = vdscli.connect()
print "connecting metadata"
response = conn.prepareImage(
       sp_uuid,
       sd_uuid,
       metadata_volume_UUID,
       metadata_image_UUID
)

print response
print "connecting lockspace"
reposnse = conn.prepareImage(
       sp_uuid,
       sd_uuid,
       lockspace_volume_UUID,
       lockspace_image_UUID
)

Should be:

response = conn.prepareImage(
       sp_uuid,
       sd_uuid,
       metadata_image_UUID,
       metadata_volume_UUID
)

reposnse = conn.prepareImage(
       sp_uuid,
       sd_uuid,
       lockspace_image_UUID,
       lockspace_volume_UUID
)

Comment 4 Federico Simoncelli 2014-08-25 15:30:52 UTC
Closing as the error was caused from the API misuse.

To improve the error message we'll use bug 1133617.