Bug 970179

Description of problem:

Customer upgraded rhev hypervisors from:
[root@up-rhevh-2 ~]# cat /etc/redhat-release
Red Hat Enterprise Virtualization Hypervisor release 6.4 (20130318.0.bz918541.el6_4)
[root@up-rhevh-2 ~]# rpm -qa|grep vdsm
vdsm-cli-4.10.2-1.8.el6_4.bz918541.noarch
vdsm-4.10.2-1.8.el6_4.bz918541.x86_64
vdsm-hook-vhostmd-4.10.2-1.8.el6_4.bz918541.noarch
vdsm-python-4.10.2-1.8.el6_4.bz918541.x86_64
vdsm-xmlrpc-4.10.2-1.8.el6_4.bz918541.noarch
vdsm-reg-4.10.2-1.8.el6_4.bz918541.noarch
[root@up-rhevh-2 ~]# rpm -qa|grep libvirt
libvirt-lock-sanlock-0.10.2-18.el6.x86_64
libvirt-client-0.10.2-18.el6.x86_64
libvirt-python-0.10.2-18.el6.x86_64
libvirt-cim-0.6.1-4.el6.x86_64
libvirt-0.10.2-18.el6.x86_64

To the current level at :

[root@up-rhevh-01 ~]# cat /etc/redhat-release
Red Hat Enterprise Virtualization Hypervisor release 6.4 (20130501.0.el6_4)
[root@up-rhevh-01 ~]# rpm -qa|grep vdsm
vdsm-cli-4.10.2-1.13.el6ev.noarch
vdsm-4.10.2-1.13.el6ev.x86_64
vdsm-hook-vhostmd-4.10.2-1.13.el6ev.noarch
vdsm-xmlrpc-4.10.2-1.13.el6ev.noarch
vdsm-reg-4.10.2-1.13.el6ev.noarch
vdsm-python-4.10.2-1.13.el6ev.x86_64
[root@up-rhevh-01 ~]# rpm -qa|grep libvirt
libvirt-lock-sanlock-0.10.2-18.el6_4.4.x86_64
libvirt-client-0.10.2-18.el6_4.4.x86_64
libvirt-python-0.10.2-18.el6_4.4.x86_64
libvirt-cim-0.6.1-4.el6.x86_64
libvirt-0.10.2-18.el6_4.4.x86_64

The customer upgraded his rhev hypervisors to the levels posted in the case description and is unable to deploy virtual machines in a timely manner. We ruled out the Cloudforms 2.0 by using the following scripts with curl to the api to create vm's. 

#!/bin/bash

NUM_OF_VMS=8

for i in `seq 1 $NUM_OF_VMS`;
do
  ID=$(uuidgen)
  echo $ID
  curl -X POST -H "Accept: application/xml" -H "Content-Type: application/xml" \
   -u admin@internal:redhat \
   --cacert /etc/pki/ovirt-engine/ca.pem \
   -d "<vm id='$ID'><name>start-test-$i</name><cluster><name>Default</name></cluster><template><name>vm1-test-template</name></template></vm>" \
    https://rhevdm31.rhev.gsslab.rdu.redhat.com/api/vms &
done

Tt took 11 minutes before the VM's were "startable" from the admin portal . The template I used is probably the worse case test. I used the template for creating our database machines. It provisions 2 50GB disk with 1 core and 3GB memory.

I ran a second test with a template that is more "standard" in our environment. It has 1 50GB disk, 6GB of memory and 1 core. It only took 6 minutes. 

All of the virtual machine templates are using thin provisioned disk and after discussing this case with other rhev engineers the times reports are slow. 

Customer has also stated that he is seeing alot of events saying that vm's are not responding.



Customer is also seeing the following db connection requests that are failing but after talking to Derrick this seems to be resolved in the .50 release that I am currently trying to get the customer to upgrade to. 


2013-05-30 10:40:41,871 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-56) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDB
C Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: