Bug 863265 - Payload cannot be set via the API
Payload cannot be set via the API
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm (Show other bugs)
6.3
Unspecified Unspecified
urgent Severity unspecified
: rc
: 6.3
Assigned To: Dan Kenigsberg
Tareq Alayan
virt
: ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-04 17:39 EDT by Joe Vlcek
Modified: 2012-12-04 14:12 EST (History)
19 users (show)

See Also:
Fixed In Version: vdsm-4.9.6-38.0
Doc Type: Bug Fix
Doc Text:
Previously, wiki documentation wrongly instructed readers to use the vm_payload tag instead of the payload tag, which made it impossible to set payloads via the API by using the documentation. File injection was also absent. This made it impossible to setting payloads through the REST API. Documentation has been updated to instruct readers to use the payload tag, and vdsm has been updated so that file injection is added to vm (this was included in VDSM 4.9.1-12). A vm_payload is now found in the database when the payload is set through the REST API.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-12-04 14:12:17 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Joe Vlcek 2012-10-04 17:39:10 EDT
Description of problem:
Setting vm_payload data via API to an existing VM
or when creating a VM does not show up in engine
database.


Version-Release number of selected component (if applicable):


How reproducible:

Create a VM with vm_payload
or
"PUT" vm_payload to an existing VM

Steps to Reproduce:
1. Create a VM with vm_payload:
e.g.:
% curl -X POST -H "Accept: application/xml" \
-H "Content-Type: application/xml" \
-u [USER:PASS] --cacert [CERT] \ 
-d "<vm><cluster><name>Default</name></cluster><template><name>[TEMPLATE]</name></template><vm_payloads><vm_payload type='cdrom/'><file name='my.txt/'><content>some stuff</content></file></vm_payload></vm_payloads></vm>" \
https://[RHEVM Host]/api/vms

Seems to work but

2. Check for the existence of the vm_payload

Check in the xml returned does not show any vm_payload.
and
Check the db on the RHEVm engine does not contain
the vm_payload.

Enter the database via
$ psql [dbname] [password]
the dbname may be rhev or rhevm
select * from vm_device;


  
Actual results:

No vm_payload found

Expected results:

A vm_payload should be found in the db

Additional info:

I feel strongly that this feature needs a way to
display the vm_payload for a VM without having to
access the db on the engine.
Comment 1 Shahar Havivi 2012-10-07 05:44:05 EDT
its look like the problem is in the syntax...
we remove the vm_prefix from the api, so now we use <payloads> and <payload> instead of <vm_payloads> and <vm_payload>, unfortunately we didn't update the ovrit wiki page: 
http://wiki.ovirt.org/wiki/Features/VMPayload#API_Design
now its updated with the right syntax,

also tested with the following curl command:

curl -X PUT -H "Accept: application/xml" -H "Content-Type: application/xml" -u user:pass -d "<vm> <payloads> <payload type='cdrom'> <file name='test.txt'> <content>my content</content> </file> </payload> </payloads> </vm>" http://shaharh:8080/api/vms/82f8c82a-2f36-4a32-a120-d18cb93df5e0

Joe please check if that is working for you as well.
Comment 2 Joe Vlcek 2012-10-08 10:22:53 EDT
(In reply to comment #1)
> its look like the problem is in the syntax...
> we remove the vm_prefix from the api, so now we use <payloads> and <payload>
> instead of <vm_payloads> and <vm_payload>, unfortunately we didn't update
> the ovrit wiki page: 
> http://wiki.ovirt.org/wiki/Features/VMPayload#API_Design
> now its updated with the right syntax,
> 
> also tested with the following curl command:
> 
> curl -X PUT -H "Accept: application/xml" -H "Content-Type: application/xml"
> -u user:pass -d "<vm> <payloads> <payload type='cdrom'> <file
> name='test.txt'> <content>my content</content> </file> </payload>
> </payloads> </vm>"
> http://shaharh:8080/api/vms/82f8c82a-2f36-4a32-a120-d18cb93df5e0
> 
> Joe please check if that is working for you as well.

Thank you Shahar!

That correct syntax seems to work correctly.

The output from the "curl -X PUT..." does show the payload and
a query of the VM also does. So that is good progress!

However the VM fails to start with the error.

I had created the VM using the UI, then issued the curl command
you provided from the command line. Then I tried to use the UI
to start the VM the following errors were listed in Events:

Failed to run VM joev_vm_pl2 (User: xyz@dom)
Failed to run VM joev_vm_pl2 on Host XYZ
VM joev_vm_pl2 is down. Wxit message: [Errno 2] No such file or directory: '/var/run/vdsm/payload/<vm ID>.<some other ID>.img'.


So it seems like we are making progress now that we have the correct
syntax but it still seems the vm_payload injection is not working
correctly.
Comment 3 Joe Vlcek 2012-10-08 10:28:18 EDT
Sorry for the typo; The Events errors should read: "Exit message:" instead
of "Wxit message:"

i.e.:

Failed to run VM joev_vm_pl2 (User: xyz@dom)
Failed to run VM joev_vm_pl2 on Host XYZ
VM joev_vm_pl2 is down. Exit message: [Errno 2] No such file or directory: '/var/run/vdsm/payload/<vm ID>.<some other ID>.img'.
Comment 4 Joe Vlcek 2012-10-08 11:18:54 EDT
I just thought of something else: When we had been using the wrong
syntax why didn't we get an error from the PUT? Shouldn't incorrect
input produce an error?
Comment 5 Shahar Havivi 2012-10-08 11:32:33 EDT
(In reply to comment #4)
> I just thought of something else: When we had been using the wrong
> syntax why didn't we get an error from the PUT? Shouldn't incorrect
> input produce an error?

its look like it does related to that patch:
http://gerrit.ovirt.org/#/c/8092/
the path shouldn't be empty, it should contain the vmid which is missing here:
'/var/run/vdsm/payload/<vm ID>.<some other ID>.img'.

I will check it and modify you Joe.
Comment 6 Michael Pasternak 2012-10-09 02:59:25 EDT
(In reply to comment #4)
> I just thought of something else: When we had been using the wrong
> syntax why didn't we get an error from the PUT? Shouldn't incorrect
> input produce an error?

no you shouldn't, - this is by spec, non-existent xml elements just
get ignored.
Comment 7 Shahar Havivi 2012-10-09 06:04:07 EDT
Hi,
I just tested a VM with the following payload
     <payloads>
         <payload type='cdrom'>
             <file name='my.txt'>
                <content>some content</content>
             </file>
         </payload>
     </payloads>

It works fine (upstream), running with no problem and I mounted the /dev/cdrom and saw the file my.txt with its content.

Joe, please try to create a new VM and test if its work for you.
Comment 8 Joe Vlcek 2012-10-09 09:17:51 EDT
Using the latest bits in RHEVm3.1 (not oVirt)

I am seeing the same error. The image I am launching is RHEL6.

I created a new VM using the UI.
Started it fine.
I stopped it .
Then using curl added a vm_payload.
After which the VM will not start with the same error I reported earlier.



The RHEVm version from the UI "About" box is:
Red Hat Enterprise Virtualization Manager Version: '3.1.0-20.el6ev'


My understanding is that this is the version QA has for testing.
Comment 9 Michal Skrivanek 2012-10-09 09:21:34 EDT
can you please follow comment #7, i.e. use "payload">
Comment 10 Joe Vlcek 2012-10-09 09:44:14 EDT
(In reply to comment #9)
> can you please follow comment #7, i.e. use "payload">

Yes Michal that is what I did use to set the payload.

I tried to describe that in my comment 2 but perhaps it wasn't fully clear.

Here is an example of it:
% curl -X PUT -H "Accept: application/xml" -u [UN:PW] --cacert [cert] -H "Content-Type: application/xml" -d "<vm> <payloads> <payload type='cdrom'> <file name='test.txt'> <content>my content</content> </file> </payload> </payloads> </vm>" https://[RHEVm Host]/api/vms/[UUID]

This does seem to work in that after I issue it the returned XML does
show the payload and future GET to: https://[RHEVm Host]/api/vms/[UUID] 
do too. However the instance fails to start with the error I reported
in comments 2 and 3
Comment 11 Shahar Havivi 2012-10-09 10:48:26 EDT
The problem was old VDSM version, 
after running in Joes environment and updating to latest version,
was: vdsm-4.9.6-36.0.el6_3.x86_64
and now: vdsm-4.9.6-37.0.el6_3.x86_64

Joe please verify.
Comment 12 Joe Vlcek 2012-10-09 15:18:19 EDT
Shahar,

Thanks for looking into this but something is still not right.

I am still seeing the same error under the VM's Events.

I also tried creating a new VM from scratch.

I see the vdsm update to version 4.9.6-37.0.el6_3

# yum list 2>&1 | grep -i vdsm
vdsm.x86_64                     4.9.6-37.0.el6_3   @ovirt20                     
vdsm-bootstrap.noarch           4.9.6-37.0.el6_3   @ovirt20                     
vdsm-cli.noarch                 4.9.6-37.0.el6_3   @ovirt20                     
vdsm-python.x86_64              4.9.6-37.0.el6_3   @ovirt20                     
vdsm-debug-plugin.noarch        4.9.6-37.0.el6_3   ovirt20                      
vdsm-gluster.noarch             4.9.6-37.0.el6_3   ovirt20                      
vdsm-hook-faqemu.noarch         4.9.6-37.0.el6_3   ovirt20                      
vdsm-hook-vhostmd.noarch        4.9.6-37.0.el6_3   ovirt20                      
vdsm-reg.noarch                 4.9.6-37.0.el6_3   ovirt20    


Here is part of the tailing of the vdsm.log file with the traceback:

% tail -f /var/log/vdsm/vdsm.log
...
MainProcess|Thread-11952::ERROR::2012-10-09 20:43:09,593::supervdsmServer::66::SuperVdsm.ServerCallback::(wrapper) Error in mkIsoFs
Traceback (most recent call last):
  File "/usr/share/vdsm/supervdsmServer.py", line 64, in wrapper
    return func(*args, **kwargs)
  File "/usr/share/vdsm/supervdsmServer.py", line 227, in mkIsoFs
    return mkimage.mkIsoFs(vmId, files)
  File "/usr/share/vdsm/mkimage.py", line 104, in mkIsoFs
    _commonCleanFs(dirname, isopath)
  File "/usr/share/vdsm/mkimage.py", line 54, in _commonCleanFs
    resolveGid(DISKIMAGE_GROUP))
OSError: [Errno 2] No such file or directory: '/var/run/vdsm/payload/f3ec6cd4-da70-49b8-8c8b-6185a7b4eed5.374196c7b0b9d960449b0b862c06753d.img'
Thread-11952::DEBUG::2012-10-09 20:43:09,594::vm::589::vm.Vm::(_startUnderlyingVm) vmId=`f3ec6cd4-da70-49b8-8c8b-6185a7b4eed5`::_ongoingCreations released
Thread-11952::ERROR::2012-10-09 20:43:09,594::vm::613::vm.Vm::(_startUnderlyingVm) vmId=`f3ec6cd4-da70-49b8-8c8b-6185a7b4eed5`::The vm start process failed
Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 579, in _startUnderlyingVm
    self._run()
  File "/usr/share/vdsm/libvirtvm.py", line 1318, in _run
    self.preparePaths(devices[vm.DISK_DEVICES])
  File "/usr/share/vdsm/vm.py", line 625, in preparePaths
    drive['path'] = self.cif.prepareVolumePath(drive, self.id)
  File "/usr/share/vdsm/clientIF.py", line 221, in prepareVolumePath
    volPath = supervdsm.getProxy().mkIsoFs(vmId, files)
  File "/usr/share/vdsm/supervdsm.py", line 59, in __call__
    return callMethod()
  File "/usr/share/vdsm/supervdsm.py", line 57, in <lambda>
    callMethod = lambda : getattr(self._supervdsmProxy._svdsm, self._funcName)(*args, **kwargs)
  File "<string>", line 2, in mkIsoFs
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
    raise convert_to_error(kind, result)
OSError: [Errno 2] No such file or directory: '/var/run/vdsm/payload/f3ec6cd4-da70-49b8-8c8b-6185a7b4eed5.374196c7b0b9d960449b0b862c06753d.img'
Comment 13 Shahar Havivi 2012-10-10 05:25:11 EDT
I will check it on a QE environment, and if I cannot reproduce it we will see why it happened only at your environment.
Comment 14 Shahar Havivi 2012-10-10 07:45:00 EDT
Ok, I think I found the problem....
Joe, please check if you have the file /bin/mkisofs if not please install it via:
yum install mkisofs

I will send a vdsm fix with the appropriate error.
Comment 15 Dan Kenigsberg 2012-10-10 08:09:13 EDT
Requires: genisoimage  has already added upstream (thanks, Zhou)

http://gerrit.ovirt.org/7232

a downstream port is

https://gerrit.eng.lab.tlv.redhat.com/2529
Comment 16 Joe Vlcek 2012-10-10 10:36:36 EDT
(In reply to comment #14)
> Ok, I think I found the problem....
> Joe, please check if you have the file /bin/mkisofs if not please install it
> via:
> yum install mkisofs


After installing mkisofs I was able to:
- create a VM using the UI.
- inject the payload using API
- start the VM using UI
- access the payload via /dev/cdrom

So that is successful.

> I will send a vdsm fix with the appropriate error.

I am not sure what you mean by that ^^



The reason I marked this BZ "needs more information" is because
it seems there were three issues here. How will each of these
be tracked?

Issue 1.
   A documentation issue describing the correct syntax
   <vm_payload> to <payload>

   I see that has been fixed in the wiki here:
   http://wiki.ovirt.org/wiki/Features/VMPayload#API_Design

   Is there more documentation that needs to be updated?
   Is a "doc" BZ required?

Issue 2.
   vdsm must be at least version 4.9.6-37.0

   Is a BZ required to track the fixing of this packaging?

Issue 3.

   mkisofs must be install.

   Is a BZ required to track the fixing of this packaging?
Comment 17 Shahar Havivi 2012-10-10 10:41:07 EDT
issue 2:
    I don't think so.

issue 3:
    this bug is the fix with the patch that Dan sent at: https://gerrit.eng.lab.tlv.redhat.com/2529
Comment 18 Joe Vlcek 2012-10-10 11:18:22 EDT
Thank you Shahar for all the help! I greatly appreciate it.

Also:

If using the API to create the VM, instead of the UI, the VM will
not start. The event error is:
	
VM joev_pl_5 is down. Exit message: local variable 'isopath' referenced before assignment.

I believe this is because of:
http://gerrit.ovirt.org/#/c/8092/

Please confirm this is correct and advise how this should be tracked.
Comment 19 Shahar Havivi 2012-10-10 11:38:18 EDT
(In reply to comment #18)
> Thank you Shahar for all the help! I greatly appreciate it.
> 
> Also:
> 
> If using the API to create the VM, instead of the UI, the VM will
> not start. The event error is:
> 	
> VM joev_pl_5 is down. Exit message: local variable 'isopath' referenced
> before assignment.
> 
> I believe this is because of:
> http://gerrit.ovirt.org/#/c/8092/
> 
> Please confirm this is correct and advise how this should be tracked.
its not related to this bug...
Federico, do you know if that is related to your fix: http://gerrit.ovirt.org/#/c/8092/
Comment 20 Joe Vlcek 2012-10-10 11:55:54 EDT
Do we need a doc bug to ensure the correct syntax as described in comment #1
is reflected in all the proper docs?

It is already updated on the wiki:
http://wiki.ovirt.org/wiki/Features/VMPayload#API_Design

but... are there other docs that need to be updated? Is a BZ needed to
track that doc work?


sgrinber@redhat.com If you are the wrong person to ask please direct me
to the correct person.
Comment 21 Simon Grinberg 2012-10-10 12:18:24 EDT
(In reply to comment #20)
> but... are there other docs that need to be updated? Is a BZ needed to
> track that doc work?
> 
> 
> sgrinber@redhat.com If you are the wrong person to ask please direct me
> to the correct person.

I've set the requires_doc_text? request to get doc team attention, no need for another bug ATM, they will clone if needed. 

The effected guides should be the CLI guide and the developer guide.
Comment 22 Federico Simoncelli 2012-10-11 11:11:22 EDT
(In reply to comment #19)
> (In reply to comment #18)
> > Thank you Shahar for all the help! I greatly appreciate it.
> > 
> > Also:
> > 
> > If using the API to create the VM, instead of the UI, the VM will
> > not start. The event error is:
> > 	
> > VM joev_pl_5 is down. Exit message: local variable 'isopath' referenced
> > before assignment.
> > 
> > I believe this is because of:
> > http://gerrit.ovirt.org/#/c/8092/
> > 
> > Please confirm this is correct and advise how this should be tracked.
> 
> its not related to this bug...
> Federico, do you know if that is related to your fix:
> http://gerrit.ovirt.org/#/c/8092/

I don't know where you saw that error but it looks more a vdsm issue, related to:

23985ce with exceptions some variables could be uninitialised

Which could eventually be in some way mitigated by my engine fix but probably 23985ce is a better fix for this specific situation.
Comment 33 Oded Ramraz 2012-11-28 07:56:32 EST
Moving to verified per comment32
Comment 35 errata-xmlrpc 2012-12-04 14:12:17 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-1508.html

Note You need to log in before you can comment on or make changes to this bug.