(In reply to comment #0)
> 
> Steps to Reproduce:
> 1. Create a VM.  Mine has a qcow2 disk on an NFS server.
> 2. Power it on with the attached program.  ./setmemandpoweron VM1 4000000 4
> 3. Reboot the hypervisor.

Are you using the libvirt-guests init script to restore running guests across host reboots?

> 4. When the HV comes back up, the VM will be off (of course).

Apparently not.  At any rate, that shouldn't affect the real analysis.

> 5. Check the VM config
> 
> [root@hb05b15 ~]# virsh dumpxml VM1 | head -20
> <domain type='kvm'>
>   <name>VM1</name>
>   <uuid>d4385c25-0946-4fb0-bbb8-55092585e7b4</uuid>
>   <memory unit='KiB'>4000000</memory>
>   <currentMemory unit='KiB'>4000000</currentMemory>
>   <vcpu placement='static'>4</vcpu>

So this is the current offline state of the domain...

> 
> 6. Start the VM.
> 
> [root@hb05b15 src]# ./setmemandpoweron VM1 7000000 7
> Config change done.  hit enter to power on.

and here, you asked to change the offline config...

> 
> Before hitting enter to initiate the poweron, check the VM config in a
> different terminal.
> 
> [root@hb05b15 ~]# virsh dumpxml VM1 | head -20
> <domain type='kvm'>
>   <name>VM1</name>
>   <uuid>d4385c25-0946-4fb0-bbb8-55092585e7b4</uuid>
>   <memory unit='KiB'>7000000</memory>
>   <currentMemory unit='KiB'>7000000</currentMemory>
>   <vcpu placement='static'>7</vcpu>

...and those requests appear to take hold...

> After hitting enter (and the VM powers on).  Check the config again.
> 
> [root@hb05b15 ~]# virsh dumpxml VM1 | head -20
> <domain type='kvm' id='2'>
>   <name>VM1</name>
>   <uuid>d4385c25-0946-4fb0-bbb8-55092585e7b4</uuid>
>   <memory unit='KiB'>4000768</memory>
>   <currentMemory unit='KiB'>4000768</currentMemory>
>   <vcpu placement='static'>4</vcpu>

...except that the virDomainCreate() is reverting back to the earlier state.  To me, this implies that the VIR_DOMAIN_AFFECT_CONFIG flag of the API in question is affecting an in-memory copy, and as long as the guest is offline, further dumpXML calls read the in-memory copy.  But it is forgetting to properly flush it back to the persistent state, so that when the domain starts, it reads from the disk instead of in-memory settings, and starts with the wrong values.

Definitely a bug in our VIR_DOMAIN_AFFECT_CONFIG handling.

The guest is not configured to boot when the host boots.



I (unsuccessfully) tried the following to work around the problem. 

To change mem/vcpus:
- Load the XML directly from /etc/libvirt/qemu/VM1.xml
- Change the vcpus node "vcpu".  (and the two mem nodes.)
- Use virDomainDefineXML().

The behaviour is the same.  After the call to virDomainDefineXML(), virsh dumpxml shows the updated values, but they are reverted upon the call to virDomainCreate().

Any suggestions to work around this problem are appreciated.

(In reply to comment #3)
> The guest is not configured to boot when the host boots.
> 
> 
> 
> I (unsuccessfully) tried the following to work around the problem. 
> 
> To change mem/vcpus:
> - Load the XML directly from /etc/libvirt/qemu/VM1.xml

Don't ever directly edit files in /etc/libvirt.  It means you are going behind libvirt's back, and all bets are off.
http://wiki.libvirt.org/page/FAQ#Where_are_VM_config_files_stored.3F_How_do_I_edit_a_VM.27s_XML_config.3F

> Any suggestions to work around this problem are appreciated.

Using virDomainDefineXML() might work as a workaround, if you don't mind hand-editing the XML returned from virDomainGetXMLDesc(), and can't wait for the bug in virDomainSetMemoryFlags() and friends to be patched, although I haven't tried it.

pkgs:
libvirt-0.10.2-6.el6.x86_64
qemu-kvm-0.12.1.2-2.316.el6.x86_64
kernel-2.6.32-330.el6.x86_64

steps:
1. prepare a domain without autostart

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     libvirt_test_api               shut off

# service libvirt-guests stop
# chkconfig --level 345 libvirt-guests off
# virsh autostart libvirt_test_api --disable

2. using python script to start domain:

#!/usr/bin/env python
import sys
import time
import libvirt

guestname = sys.argv[1]
mem = int(sys.argv[2])
vcpu = int(sys.argv[3])

con = libvirt.open(None)
domobj = con.lookupByName(guestname)
domobj.setMemoryFlags(mem, libvirt.VIR_DOMAIN_AFFECT_CONFIG|libvirt.VIR_DOMAIN_MEM_MAXIMUM)
domobj.setMemoryFlags(mem, libvirt.VIR_DOMAIN_AFFECT_CONFIG|libvirt.VIR_DOMAIN_MEM_CURRENT)
domobj.setVcpusFlags(vcpu, libvirt.VIR_DOMAIN_AFFECT_CONFIG|libvirt.VIR_DOMAIN_MEM_MAXIMUM)
domobj.setVcpusFlags(vcpu, libvirt.VIR_DOMAIN_AFFECT_CONFIG|libvirt.VIR_DOMAIN_MEM_CURRENT)
str = domobj.XMLDesc(0)
print str
time.sleep(3)
domobj.create()

# python cpu_mem_set.py libvirt_test_api 4000000 4
...
  <memory unit='KiB'>4000768</memory>
  <currentMemory unit='KiB'>4001792</currentMemory>
  <vcpu placement='static'>4</vcpu>
...

# virsh list
 Id    Name                           State
----------------------------------------------------
 4     libvirt_test_api               running

3. reboot host
# reboot

4. retest with script
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     libvirt_test_api               shut off

# virsh dumpxml libvirt_test_api
...
  <memory unit='KiB'>4000000</memory>
  <currentMemory unit='KiB'>4000000</currentMemory>
  <vcpu placement='static'>4</vcpu>
...

# python cpu_mem_set.py libvirt_test_api 7000000 7
...
  <memory unit='KiB'>7000000</memory>
  <currentMemory unit='KiB'>7000000</currentMemory>
  <vcpu placement='static'>7</vcpu>
...

...
  <memory unit='KiB'>7000064</memory>
  <currentMemory unit='KiB'>7000064</currentMemory>
  <vcpu placement='static'>7</vcpu>
...

Can not reproduce it here.

Also tried with the c program attached on libvirt-0.9.10-21.el6_3.4.x86_64 and with exact steps as described(both with img on nfs and local), still fail to reproduce.

Hi Wayne,  I tested this again, with the most recent libvirt code.  Here is my result.

Step 1: Check the versions.

[root@dcdemo04 ~]# rpm -qa | egrep 'libvirt'
libvirt-client-0.9.10-21.el6_3.5.x86_64
libvirt-python-0.9.10-21.el6_3.5.x86_64
libvirt-0.9.10-21.el6_3.5.x86_64


Step 2: Start a VM.

[root@dcdemo04 ~]# setmemandpoweron VM1 2000000 2
Config change done.  hit enter to power on.

[root@dcdemo04 ~]# virsh dumpxml VM1 | egrep 'emory|vcpu'
  <memory unit='KiB'>2000896</memory>
  <currentMemory unit='KiB'>2000000</currentMemory>
  <vcpu placement='static'>2</vcpu>
[root@dcdemo04 ~]# ps -ef | grep kvm
root      2029     2  0 15:26 ?        00:00:00 [kvm-irqfd-clean]
root      5502     1 19 15:34 ?        00:00:08 /usr/libexec/qemu-kvm ... -m 1954 -smp 2,sockets=2,cores=1,threads=1 -name VM1 ...


Step 3: Reboot the hypervisor.


[root@dcdemo04 ~]# reboot

Broadcast message from root@dcdemo04
	(/dev/pts/0) at 15:36 ...

The system is going down for reboot NOW!
[root@dcdemo04 ~]# Connection to dcdemo04 closed by remote host.
Connection to dcdemo04 closed.


Step 4: When the hypervisor comes back up, try to power it on again with a different value for memory and vcpus.  The program will prompt you to check the config and then hit enter.


[root@dcdemo04 ~]# setmemandpoweron VM1 4000000 4
Config change done.  hit enter to power on.

# In another window, check the config.

[root@dcdemo04 ~]# virsh dumpxml VM1 | egrep 'emory|vcpu'
  <memory unit='KiB'>4000000</memory>
  <currentMemory unit='KiB'>4000000</currentMemory>
  <vcpu placement='static'>4</vcpu>

# In the original window, hit enter to actually trigger the api call.  Then check the VM.

[root@dcdemo04 ~]# virsh dumpxml VM1 | egrep 'emory|vcpu'
  <memory unit='KiB'>2000896</memory>
  <currentMemory unit='KiB'>2000000</currentMemory>
  <vcpu placement='static'>2</vcpu>
[root@dcdemo04 ~]# 


root      4454     1  3 15:46 ?        00:00:02 /usr/libexec/qemu-kvm ... -m 1954 -smp 2,sockets=2,cores=1,threads=1 -name VM1 ...



Seems that the bug is still there.  Can you retry and post the same output as mine?

I can also reproduce the bug with your python script.

Initial start.

[root@dcdemo04 ~]# ./libvirt_test_api.py VM1 3000000 3
  <memory unit='KiB'>3000000</memory>
  <currentMemory unit='KiB'>3000000</currentMemory>
  <vcpu placement='static'>3</vcpu>

Reboot.

[root@dcdemo04 ~]# ./libvirt_test_api.py VM1 5000000 5
  <memory unit='KiB'>5000000</memory>
  <currentMemory unit='KiB'>5000000</currentMemory>
  <vcpu placement='static'>5</vcpu>


[root@dcdemo04 ~]# virsh dumpxml VM1 | egrep 'emory|vcpu'
  <memory unit='KiB'>3000320</memory>
  <currentMemory unit='KiB'>3000000</currentMemory>
  <vcpu placement='static'>3</vcpu>

root      6663     1 17 16:11 ?        00:00:01 /usr/libexec/qemu-kvm ... -m 2930 -smp 3,sockets=3,cores=1,threads=1 ...


And then if I kill the VM, the mem/vcpu changes back.

[root@dcdemo04 ~]# kill 6663
[root@dcdemo04 ~]# virsh dumpxml VM1 | egrep 'emory|vcpu'
  <memory unit='KiB'>5000000</memory>
  <currentMemory unit='KiB'>5000000</currentMemory>
  <vcpu placement='static'>5</vcpu>

steps 1
# rpm -qa|grep libvirt
libvirt-client-0.9.10-21.el6_3.5.x86_64
libvirt-devel-0.9.10-21.el6_3.5.x86_64
libvirt-python-0.9.10-21.el6_3.5.x86_64
libvirt-debuginfo-0.9.10-21.el6_3.5.x86_64
libvirt-0.9.10-21.el6_3.5.x86_64

Step 2: Start a VM.
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     libvirt_test_api               shut off


# ./setmemandpoweron libvirt_test_api 2000000 2
Config change done.  hit enter to power on.

# virsh dumpxml libvirt_test_api | egrep 'emory|vcpu'
  <memory unit='KiB'>2000896</memory>
  <currentMemory unit='KiB'>2000896</currentMemory>
  <vcpu placement='static'>2</vcpu>

# ps aux|grep kvm
root      2199  0.0  0.0      0     0 ?        S    15:22   0:00 [kvm-irqfd-clean]
qemu      3914 64.9  0.1 2557568 251432 ?      Sl   15:30   0:07 /usr/libexec/qemu-kvm -S -M rhel6.4.0 -enable-kvm -m 1954 -smp 2,sockets=2,cores=1,threads=1 -name libvirt_test_api ...

Step 3: Reboot the hypervisor.

# reboot

Step 4: 

# ./setmemandpoweron libvirt_test_api 4000000 4
Config change done.  hit enter to power on.

# In another window, check the config.
# virsh dumpxml libvirt_test_api | egrep 'emory|vcpu'
  <memory unit='KiB'>4000000</memory>
  <currentMemory unit='KiB'>4000000</currentMemory>
  <vcpu placement='static'>4</vcpu>

# In the original window, hit enter to actually trigger the api call.  Then check the VM.

# virsh dumpxml libvirt_test_api | egrep 'emory|vcpu'
  <memory unit='KiB'>4000768</memory>
  <currentMemory unit='KiB'>4001792</currentMemory>
  <vcpu placement='static'>4</vcpu>

# ps aux|grep kvm
root      2056  0.0  0.0      0     0 ?        S    15:40   0:00 [kvm-irqfd-clean]
qemu      3645 66.8  0.5 4664060 668300 ?      Sl   15:44   0:20 /usr/libexec/qemu-kvm -S -M rhel6.4.0 -enable-kvm -m 3907 -smp 4,sockets=4,cores=1,threads=1 -name libvirt_test_api ...


still can't reproduce.

Hi Michael,

can you attach your domain xml, check dominfo and libvirt-guests service?

# virsh dumpxml libvirt_test_api > /tmp/domain.xml
then attach it here

# service libvirt-guests status
stopped, with no saved guests

# virsh dominfo libvirt_test_api|grep Autostart
Autostart:      disable

The libvirt-guests service was enabled and running on my hypervisor host.  But the domain wasn't set to autostart.

If I disable that service, then I cannot reproduce the problem anymore.

Also, to double check, I enabled the service again, rebooted the HV and was againa able to reproduce the problem.

For my requirements this isn't a bug any more. So I will close it.  But the behaviour is not right.

- The libvirt-guests service is enabled.
- A VM is _not_ configured to autostart.
- The HV host gracefully restarts.
- On shutdown, the guest's mem and vcpu config is "remembered".
- When the HV comes backup, the guest isn't automatically restarted.  Which is expected since it wasn't configured to autostart.
- When the guest is started, with a different mem/cpu config, the previous "remembered" config is actually used, not the user supplied one.  IMO, using the user supplied config is the right design.

Eric, it looks to me like Michael's program is modifying the persistent config, so that's what's subsequently being used to start the VM so this is behavior by design, is that your reading also?

(In reply to comment #10)
> The libvirt-guests service was enabled and running on my hypervisor host. 
> But the domain wasn't set to autostart.

The design of the libvirt-guests service is that it is _supposed_ to re-start any guest that was running when the host shut down, whether or not that guest was marked autostart.  And when it does re-start the guest, it uses the configuration that was in use by the guest at the time the host went down.

> 
> If I disable that service, then I cannot reproduce the problem anymore.

Therefore, what you are seeing is that the guest state saved by libvirt-guests is surviving a host reboot, even if it is different than the state you manually put into the config file.  The fact that the guest is resuming where it left off (including the state it was previously running with, and NOT the state in the persistent config file) is a feature.

> 
> Also, to double check, I enabled the service again, rebooted the HV and was
> againa able to reproduce the problem.

I think what you are seeing is by design, and not a bug, and that you missed the fact that using libvirt-guests will re-start a guest even if it is not marked as autostart.

"The design of the libvirt-guests service is that it is _supposed_ to re-start any guest that was running when the host shut down, whether or not that guest was marked autostart."

I see.  The problem that my system had is that the VM disk is on a shared filesystem that isn't available until after libvirt-guests runs.  So resuming the managed save fails.  Not really a problem with libvirt-guests since its probably assumed that the VM disk is available when libvirt-guests starts.  And the negative case handling for the case that the VM disk isn't available probably hasn't been defined yet.