Created attachment 512863 [details]
/var/log/livbirt/qemu/winxp-lk.log

Created attachment 512864 [details]
vdsm.log

Problematic targets in vdsm.log

e37055a7-8205-470c-a546-109ca2307c9c     winxp-lk               
be2b16df-6144-4191-8715-27cc2739f709     win7x32-lk

Didn't we fixed this already, what's the later virtio-win version?

(In reply to comment #6)
> Didn't we fixed this already, what's the later virtio-win version?

https://bugzilla.redhat.com/show_bug.cgi?id=702258

VERIFIED with virtio-win-prewhql-0.1.11 

http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/11/win/virtio-win-prewhql-0.1.zip

(In reply to comment #7)
> (In reply to comment #6)
> > Didn't we fixed this already, what's the later virtio-win version?
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=702258
> 
> VERIFIED with virtio-win-prewhql-0.1.11 
> 
> http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/11/win/virtio-win-prewhql-0.1.zip

Vadim,

According to your advice I did following:
I have a WinXP guest with ic114 tools installed (from wrapped installer), I did update Virtio serial driver using Device Manager (going to the "Have disk" option) with driver from your zip - WXp->x86->vioser.inf. Then I shutdown the guest and start it again.

I do not have any problems when connecting to the guest using Admin portal of RHEVM and shutdown/reboot works but I have problems when connecting to the guest using User portal of RHEVM.
When I start guest and connect to it using User portal during "Powering up" stage, Spice client window is opened and guest is displayed but I do not have mouse (I see cursor, but I am not able to click). Then I open the same guest using Admin portal and BSOD crash appears on guest with message DRIVER_IRQL_NOT_LESS_OR_EQUAL pointing to file wdf01000.sys.
When I start guest and connect to it using User portal during "UP" stage, Spice client window does not open because of spice vdagent timeout. During timeout (waiting for spice vdagent response) RHEVM Admin portal says that guest is "Reboot in Progress". Once I connect to the guest using Admin portal, I get the same BSOD as earlier described.
RHEV agent and Spice vdagent are running all the time I can connect and check.
This does not happen with virtio serial driver from ic114 tools.
Maybe I am not doing the update of driver properly? Could you help or check that?

(In reply to comment #8)
> (In reply to comment #7)
> > (In reply to comment #6)
> > > Didn't we fixed this already, what's the later virtio-win version?
> I do not have any problems when connecting to the guest using Admin portal of
> RHEVM and shutdown/reboot works but I have problems when connecting to the
> guest using User portal of RHEVM.
> When I start guest and connect to it using User portal during "Powering up"
> stage, Spice client window is opened and guest is displayed but I do not have
> mouse (I see cursor, but I am not able to click). Then I open the same guest
> using Admin portal and BSOD crash appears on guest with message
> DRIVER_IRQL_NOT_LESS_OR_EQUAL pointing to file wdf01000.sys.
Hi Marian,
Can you publish the crash dump file, in case you have it somewhere around?
> When I start guest and connect to it using User portal during "UP" stage, Spice
> client window does not open because of spice vdagent timeout. During timeout
> (waiting for spice vdagent response) RHEVM Admin portal says that guest is
> "Reboot in Progress". Once I connect to the guest using Admin portal, I get the
> same BSOD as earlier described.
> RHEV agent and Spice vdagent are running all the time I can connect and check.
> This does not happen with virtio serial driver from ic114 tools.
> Maybe I am not doing the update of driver properly? Could you help or check
> that?
The simplest way is to check "Driver Date" in vioserial driver "Properties"
dialog. I don't remember exactly, but for build 0.1.11 it must be around July 10.

Best regards,
Vadim.

(In reply to comment #7)
I run into the same symptoms on a WinXP guest with IDE (not VirtIO) disk.

(In reply to comment #10)
> (In reply to comment #7)
> I run into the same symptoms on a WinXP guest with IDE (not VirtIO) disk.

It seems to be a virtio serial problem, which is not related to the storage subsystem. 
Regards,
Vadim.

Hi Arkady,

There is a new bug, introduced with 
81e852174dbd581cd4678eca4127865b43aa14f2.
As we can see, we don't reset ComleteRequest
bask to FALSE.
Could you check and commit the patch, I have provided
below, right after windbg bug analysis?

Cheers,
Vadim.

0: kd> lmvm vioser

start    end        module name

f76ae000 f76b7100   vioser     (private pdb symbols)  z:\721355\wxp\x86\vioser.pdb

    Loaded symbol image file: vioser.sys

    Image path: \SystemRoot\system32\DRIVERS\vioser.sys

    Image name: vioser.sys

    Timestamp:        Sat Jul 09 01:55:35 2011 (4E181787)

    CheckSum:         0000F572

    ImageSize:        00009100

    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4

0: kd> !analyze -v

*******************************************************************************

*                                                                             *

*                        Bugcheck Analysis                                    *

*                                                                             *

*******************************************************************************



DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)

An attempt was made to access a pageable (or completely invalid) address at an

interrupt request level (IRQL) that is too high.  This is usually

caused by drivers using improper addresses.

If kernel debugger is available get stack backtrace.

Arguments:

Arg1: 0000001c, memory referenced

Arg2: 00000002, IRQL

Arg3: 00000001, value 0 = read operation, 1 = write operation

Arg4: f6eab931, address which referenced memory



Debugging Details:

------------------





WRITE_ADDRESS:  0000001c 



CURRENT_IRQL:  2



FAULTING_IP: 

wdf01000!imp_WdfRequestCompleteWithInformation+c2

f6eab931 89511c          mov     dword ptr [ecx+1Ch],edx



DEFAULT_BUCKET_ID:  INTEL_CPU_MICROCODE_ZERO



BUGCHECK_STR:  0xD1



PROCESS_NAME:  Idle



TRAP_FRAME:  80551340 -- (.trap 0xffffffff80551340)

ErrCode = 00000002

eax=862fdcf8 ebx=00000000 ecx=00000000 edx=00000045 esi=862fdcb0 edi=00000100

eip=f6eab931 esp=805513b4 ebp=805513bc iopl=0         nv up ei ng nz na po nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010282

wdf01000!imp_WdfRequestCompleteWithInformation+0xc2:

f6eab931 89511c          mov     dword ptr [ecx+1Ch],edx ds:0023:0000001c=????????

Resetting default scope



LAST_CONTROL_TRANSFER:  from f6eab931 to 80544758



STACK_TEXT:  

80551340 f6eab931 badb0d00 00000045 00000000 nt!KiTrap0E+0x238

805513bc f76aeb24 00000000 862fdcb0 00000000 wdf01000!imp_WdfRequestCompleteWithInformation+0xc2

805513fc f6ee5dad 00000002 79abd938 8055c0c0 vioser!VIOSerialInterruptDpc+0x1a6 [c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\vioserial\sys\isrdpc.c @ 104]

80551418 f6ee5df6 865398c8 00000000 ffdff980 wdf01000!FxInterrupt::DpcHandler+0x60

80551428 80545eef 86539914 865398c8 865398c8 wdf01000!FxInterrupt::_InterruptDpcThunk+0x13

80551450 80545dd4 00000000 0000000e 00000000 nt!KiRetireDpcList+0x61

80551454 00000000 0000000e 00000000 00000000 nt!KiIdleLoop+0x28





STACK_COMMAND:  kb



FOLLOWUP_IP: 

vioser!VIOSerialInterruptDpc+1a6 [c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\vioserial\sys\isrdpc.c @ 104]

f76aeb24 ff4508          inc     dword ptr [ebp+8]



FAULTING_SOURCE_CODE:  

   100:            WdfSpinLockRelease(port->InBufLock);

   101: 

   102:            if (ComleteRequest)

   103:            {

>  104:                WdfRequestCompleteWithInformation(request, STATUS_SUCCESS, information);

   105:            }

   106:         }

   107:     }

   108:     TraceEvents(TRACE_LEVEL_VERBOSE, DBG_DPC, "<-- %s\n", __FUNCTION__);

   109: }





SYMBOL_STACK_INDEX:  2



SYMBOL_NAME:  vioser!VIOSerialInterruptDpc+1a6



FOLLOWUP_NAME:  MachineOwner



MODULE_NAME: vioser



IMAGE_NAME:  vioser.sys



DEBUG_FLR_IMAGE_TIMESTAMP:  4e181787



FAILURE_BUCKET_ID:  0xD1_vioser!VIOSerialInterruptDpc+1a6



BUCKET_ID:  0xD1_vioser!VIOSerialInterruptDpc+1a6



Followup: MachineOwner

---------



0: kd> .frame 2

02 805513fc f6ee5dad vioser!VIOSerialInterruptDpc+0x1a6 [c:\cygwin\tmp\build\source\internal-kvm-guest-drivers-windows\vioserial\sys\isrdpc.c @ 104]

0: kd> dv

      Interrupt = 0x00000002

AssociatedObject = 0x79abd938

         Device = 0x79abd938

       pContext = 0x86542888

        request = 0x79d02348

 ComleteRequest = 0x01 ''

   systemBuffer = 0x85f9c000 "???"

         Length = 0x1000

              i = 2

    information = 0x45

0: kd> !wdfkd.wdfrequest 0x79d02348

   !IRP 0x00000000

    irp is NULL, the remaining results may not be correct(Reserved Requests may have a NULL IRP)...



   State:  Completed, Allocated by WDF for incoming IRP











diff --git a/vioserial/sys/IsrDpc.c b/vioserial/sys/IsrDpc.c

index 7d2bbaf..722ea78 100644

--- a/vioserial/sys/IsrDpc.c

+++ b/vioserial/sys/IsrDpc.c

@@ -92,8 +92,8 @@ VIOSerialInterruptDpc(

                  }

                  else

                  {

-                    request = NULL;

                     TraceEvents(TRACE_LEVEL_INFORMATION, DBG_DPC, "Request = %p was cancelled\n", request);

+                    request = NULL;

                  }

               }

            }

@@ -102,6 +102,7 @@ VIOSerialInterruptDpc(

            if (ComleteRequest)

            {

                WdfRequestCompleteWithInformation(request, STATUS_SUCCESS, information);

+               ComleteRequest = FALSE;

            }

         }

     }

Done!
Arkady

Marian, 
could you please check it again, with the vioserial driver from
the latest build:
http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/13/win/virtio-win-prewhql-0.1.zip

Thank you,
Vadim.

Tried on
http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/13/win/virtio-win-prewhql-0.1.zip
for this bug .

still hit some issue .


(In reply to comment #8)

> According to your advice I did following:
> I have a WinXP guest with ic114 tools installed (from wrapped installer), I did
> update Virtio serial driver using Device Manager (going to the "Have disk"
> option) with driver from your zip - WXp->x86->vioser.inf. Then I shutdown the
> guest and start it again.

steps of mine :
1.install a fresh winxp guest with ide disk& rtl8139 nic by using rhevm
2.then install ic116 tools (virtio driver& vdagent)by using 
RHEV-ToolsSetup_428.exe  
3.then update virtio serial driver to 0.1.13 one
3 .shutdown or reboot guest 

Actual Results:
Both Tried admin portal & User portal.
guest still hang ,can not reboot or shutdown .
I also tried above steps w/o step 2 ,guest could reboot or shutdown
successfully.


----
I had some question abt this bug:
1.What's the difference between this bug and Bug 702258 ?

2.What's the difference between installing driver via RHEV-ToolsSetup_428.exe
and installing driver in device manager ?

3.based on above ,seems this bug due to RHEV-ToolsSetup_428.exe .which
component
does it belongs to ?

Best Regards,
Mike

Hi, Mike!
1) This two bug separated, because of each cause different bsod screens
2/3) Try to install ic129 because rhev agent service changed to treat shutdown event
in windows.
Best, Arkady

(In reply to comment #18)
> Marian, 
> could you please check it again, with the vioserial driver from
> the latest build:
> http://download.devel.redhat.com/brewroot/packages/virtio-win-prewhql/0.1/13/win/virtio-win-prewhql-0.1.zip
> 
> Thank you,
> Vadim.

Hi Vadim,
I retested with 0.1-13 and seems the problem with BSOD reported here is solved - I am not able to reproduced it. Reboot/Shutdown in my setup works well. As well as connecting from User portal of Rhevm.
I would call this BSOD problem as solved. Thank you.
However as I described in my comment #8: "Spice client window is opened and guest is displayed but I do not have mouse (I see cursor, but I am not able to click)." This problem still occurs and I personally did not see if I used 0.1-11 virtio-serial driver version.
This mouse issue pls see in bug #725734. I do believe It's virtio-serial driver related.

> 
> steps of mine :
> 1.install a fresh winxp guest with ide disk& rtl8139 nic by using rhevm
> 2.then install ic116 tools (virtio driver& vdagent)by using 
> RHEV-ToolsSetup_428.exe  
> 3.then update virtio serial driver to 0.1.13 one
> 3 .shutdown or reboot guest 
> 
> Actual Results:
> Both Tried admin portal & User portal.
> guest still hang ,can not reboot or shutdown .
> I also tried above steps w/o step 2 ,guest could reboot or shutdown
> successfully.

I just wanna note that I believe the shutdown/reboot in step 3. will not work I would add more steps.

4. stop guest by force.
5. Boot guest
6. Shutdown or reboot - this should work, at least in my case does.

(In reply to comment #22)
> > 
> > steps of mine :
> > 1.install a fresh winxp guest with ide disk& rtl8139 nic by using rhevm
> > 2.then install ic116 tools (virtio driver& vdagent)by using 
> > RHEV-ToolsSetup_428.exe  
> > 3.then update virtio serial driver to 0.1.13 one
> > 3 .shutdown or reboot guest 
> > 
> > Actual Results:
> > Both Tried admin portal & User portal.
> > guest still hang ,can not reboot or shutdown .
> > I also tried above steps w/o step 2 ,guest could reboot or shutdown
> > successfully.
> 
> I just wanna note that I believe the shutdown/reboot in step 3. will not work I
> would add more steps.
> 
> 4. stop guest by force.
> 5. Boot guest
> 6. Shutdown or reboot - this should work, at least in my case does.

Hi ,Marian

Actual I did ,still can not shutdown ,I will referring to Arkady's advice to install rhev agent in ic129

I want to ask you 2 questions since I was not very familiar with RHEVM
1.which RHEV-Tools do you installed in the guest ,could u gave me the link？
2.As I know ,ic129 is the deployed in linux platform ,could the rhev-agent be used in RHEVM IC116 which  was deployed in windows 2k8 ?
3.May I based on your comment #21,move status to VERIFIED ?

Thanks,
Mike

(In reply to comment #22)
> > 
> > steps of mine :
> > 1.install a fresh winxp guest with ide disk& rtl8139 nic by using rhevm
> > 2.then install ic116 tools (virtio driver& vdagent)by using 
> > RHEV-ToolsSetup_428.exe  
> > 3.then update virtio serial driver to 0.1.13 one
> > 3 .shutdown or reboot guest 
> > 
> > Actual Results:
> > Both Tried admin portal & User portal.
> > guest still hang ,can not reboot or shutdown .
> > I also tried above steps w/o step 2 ,guest could reboot or shutdown
> > successfully.
> 
> I just wanna note that I believe the shutdown/reboot in step 3. will not work I
> would add more steps.
> 
> 4. stop guest by force.
> 5. Boot guest
> 6. Shutdown or reboot - this should work, at least in my case does.

Hi Marian,

Please take a look at the following comment
https://bugzilla.redhat.com/show_bug.cgi?id=702258#c19
Is it your case?

Best regards,
Vadim.

(In reply to comment #20)
> Hi, Mike!
> 1) This two bug separated, because of each cause different bsod screens
> 2/3) Try to install ic129 because rhev agent service changed to treat shutdown
> event
> in windows.

Hi, Arkady 

I tried on RHEV-toolsetup-560 in ic129 with the steps suggested by Marian in comment#22.,

Actual Results:
1.install RHEV virtio-serial & RHEV agent both ,then update virtio serial driver to 0.1.13
 guest still can *not* shutdown/poweroff if i installed RHEV-agent.

2.only install virtio-serial & w/o RHEV agent
 gusest works fine.


Does RHEV-agent is a must for this bug ? 
Based on above ,seems RHEV-agent in RHEV-toolsetup-560.exe (ic129) still has a bug ,which componenent does it belongs to ?
Do we had a bug to investigate it ?

Thanks,
Mike

Hi, Mike!

To analyse the problem I need the dmp file of the hung system. Please create it and upload for analysis. 
How to do that check either http://msdn.microsoft.com/en-us/library/ff545499.aspx
or download bung utility from http://www.osronline.com/article.cfm?article=153

TIA
Arkady

Forgot to mention Mike, if that not work you can use NMI interrupt, as described in http://support.microsoft.com/kb/927069

Best, Arkady

Last addition, I beleive :)
NMI switch in our case is qemu's nmi command.
Be aware that really last driver work in the system. For that force shutdown the VM by qemu if that hung. And after reboot check that vioserial driver running is really the last driver you installed. After that shutdown the system and if that hung create dmp

TIA
Arkady

more info about comment #29

I tried both ctrl+ Scroll  & (qemu)nmi when guest not hang ,it could generate dmp file successfully .

Mike

Hi, Mike!

Vadim proposed another option - .dump command from windbg.
For that you have to run host VM with WinDbg connected to target VM in the way
described in http://www.linux-kvm.org/page/WindowsGuestDrivers/GuestDebugging.
When Target VM hungs, Host VM will produce dump file with .dump command.

Before open new bug we have to find what cause of the problem ( maybe that old bug )

Best,Arkady

(In reply to comment #25)
> (In reply to comment #20)
> > Hi, Mike!
> > 1) This two bug separated, because of each cause different bsod screens
> > 2/3) Try to install ic129 because rhev agent service changed to treat shutdown
> > event
> > in windows.
> 
> Hi, Arkady 
> 
> I tried on RHEV-toolsetup-560 in ic129 with the steps suggested by Marian in
> comment#22.,
> 
> Actual Results:
> 1.install RHEV virtio-serial & RHEV agent both ,then update virtio serial
> driver to 0.1.13
>  guest still can *not* shutdown/poweroff if i installed RHEV-agent.
> 
> 2.only install virtio-serial & w/o RHEV agent
>  gusest works fine.
> 
> 
> Does RHEV-agent is a must for this bug ? 
> Based on above ,seems RHEV-agent in RHEV-toolsetup-560.exe (ic129) still has a
> bug ,which componenent does it belongs to ?
> Do we had a bug to investigate it ?
> 
> Thanks,
> Mike

Hi Mike,
Stupid question, How are you running the guest? Using RHEVM? If not, how are you able to start rhevm agent service?
Arkady,
Does the rhevm agent service need to be running to influence the shutdown behaviour?
Anyway I did not hit what Mike is describing, I gotta retest by myself.
Thanks.

(In reply to comment #34)
> (In reply to comment #25)
> > (In reply to comment #20)
> > > Hi, Mike!
> > > 1) This two bug separated, because of each cause different bsod screens
> > > 2/3) Try to install ic129 because rhev agent service changed to treat shutdown
> > > event
> > > in windows.
> > 
> > Hi, Arkady 
> > 
> > I tried on RHEV-toolsetup-560 in ic129 with the steps suggested by Marian in
> > comment#22.,
> > 
> > Actual Results:
> > 1.install RHEV virtio-serial & RHEV agent both ,then update virtio serial
> > driver to 0.1.13
> >  guest still can *not* shutdown/poweroff if i installed RHEV-agent.
> > 
> > 2.only install virtio-serial & w/o RHEV agent
> >  gusest works fine.
> > 
> > 
> > Does RHEV-agent is a must for this bug ? 
> > Based on above ,seems RHEV-agent in RHEV-toolsetup-560.exe (ic129) still has a
> > bug ,which componenent does it belongs to ?
> > Do we had a bug to investigate it ?
> > 
> > Thanks,
> > Mike
> 
> Hi Mike,
> Stupid question, How are you running the guest? Using RHEVM? If not, how are
> you able to start rhevm agent service?
> Arkady,
> Does the rhevm agent service need to be running to influence the shutdown
> behaviour?
> Anyway I did not hit what Mike is describing, I gotta retest by myself.
> Thanks.

Scratch that, I noticed that you mentioned that you were using Admin/User portal. Sorry.

Hi, Marian!
Sure, Rhev agent service have to run to start serial driver to work.
OTOH possible instead of service to start vioser-test.exe ( from the same directory where sys/inf files lacated ) and during async read to shutdown/reboot the VM.
 
Arkady

Hi, Mike!

That's additional question what happened it you try to shutdown the VM , with vioser-test.exe running. Just start it from dos box , but before stop the rhev agent service if its run, so you'll see that vioser-test.exe on start will open serial device ( no parameters needed ). You'll see the text:
Ruuning in non-blocking mode.
Open vioserial device \\?\GUID of the device

Now you can enter 'r' - for read
And the answer should be text on the new line - ReadTest.

After some period ( ~10 sec ) you have to see error 995, that mean that driver finish read operation with error after TO so app ( service when it run ) can continue to process input commands.

Try that instead of running service meanwhile.

TIA
Arkady

(In reply to comment #24)

> Hi Marian,
> 
> Please take a look at the following comment
> https://bugzilla.redhat.com/show_bug.cgi?id=702258#c19
> Is it your case?
> 
> Best regards,
> Vadim.

Hi Vadim,
Yes, It is.

I still fail to reproduce the bug with reboot/shutdown with virtio serial driver 0.1-13. 
I have clean install of Win7, then I installed whatever tools I had (ic128) and updated the virtio-serial. Then I did reboot, and I observed behaviour described in https://bugzilla.redhat.com/show_bug.cgi?id=702258#c19 (as vadim posted earlier). Once I closed the console, guest got rebooted.
And then rebooting/shutdown work fine in my case, only when doing shutdown, shutdown is successful but RHEV Manager still shows that guest is running (even no qemu process exists anymore), but I assume it's the bug of rhevm agent, Arkady mentioned earlier.

Only problems I can see are problems with mouse as I stated earlier and is described in separated bug #725734.

I am afraid I will not be any helpful anymore in this bug.

(In reply to comment #39)
> I still fail to reproduce the bug with reboot/shutdown with virtio serial
> driver 0.1-13. 
> I have clean install of Win7, then I installed whatever tools I had (ic128) and
> updated the virtio-serial. Then I did reboot, and I observed behaviour
> described in https://bugzilla.redhat.com/show_bug.cgi?id=702258#c19 (as vadim
> posted earlier). Once I closed the console, guest got rebooted.
> And then rebooting/shutdown work fine in my case, only when doing shutdown,
> shutdown is successful but RHEV Manager still shows that guest is running (even
> no qemu process exists anymore), but I assume it's the bug of rhevm agent,
> Arkady mentioned earlier.
> Only problems I can see are problems with mouse as I stated earlier and is
> described in separated bug #725734.
> I am afraid I will not be any helpful anymore in this bug.

Hi， Marian

Ｏnly install virtio serial driver 0.1-13 ,guest works fine .as your described in comment #39.
guest can ×not× poweroff/shutdown after　I install *rhev-agent* package which was bundled in RHEV-toolsetup-560.exe（steps：double click rhev-toolsetup-560.exe ,choose "rhev serial" and “rhev agent" both , then install them） .
After I uninstall *rhev-agent* ,guest works as normal ,it could be reboot or poweroff .
I was not very familiar with rhev-agent.I will post a screendump tomorrow for my steps.May I ask what's the usage of rhev-agent and what's the relation between rhev-agent& virtio-serial driver ?
whether have you installed rhev-agent during testing ?

Best Regards,
Mike

> Hi， Marian
> 
> Ｏnly install virtio serial driver 0.1-13 ,guest works fine .as your described
> in comment #39.
> guest can ×not× poweroff/shutdown after　I install *rhev-agent* package which
> was bundled in RHEV-toolsetup-560.exe（steps：double click rhev-toolsetup-560.exe
> ,choose "rhev serial" and “rhev agent" both , then install them） .
> After I uninstall *rhev-agent* ,guest works as normal ,it could be reboot or
> poweroff .
> I was not very familiar with rhev-agent.I will post a screendump tomorrow for
> my steps.May I ask what's the usage of rhev-agent and what's the relation
> between rhev-agent& virtio-serial driver ?
> whether have you installed rhev-agent during testing ?
> 
> Best Regards,
> Mike

Hi,
I've reproduced the shutdown hang as well, The reason why I was not able to reproduce earlier is that I tried it on Win7/64bit where I am not able to reproduce the shutdown hang at all and Shutdown/reboot works.
So I've tried Win7/32bit and It's reproducible.

Hi, Mike!

RHEV agent use serial for communication with host.
You have to install new driver after installing rhev-m agent. After reboot the system, just stop the RHEV agent service and run test app as I proposed.
If you observe the behaviour I mentioned ( returned error 995 ) that mean driver finished IO read op with time-out and allow the service to be shutdowned. Try shutdown/reboot the system without closing test app. If that work correctly.
After restart the service will start automatically and try to shutdown/reboot again. This time you check service and not driver

TIA, Arkady

(In reply to comment #42)
> Hi, Mike!
> 
> RHEV agent use serial for communication with host.
> You have to install new driver after installing rhev-m agent. After reboot the
> system, just stop the RHEV agent service and run test app as I proposed.
> If you observe the behaviour I mentioned ( returned error 995 ) that mean
> driver finished IO read op with time-out and allow the service to be
> shutdowned. Try shutdown/reboot the system without closing test app. If that
> work correctly.

yes ,It work correctly .guest could reboot succesfully.

> After restart the service will start automatically and try to shutdown/reboot
> again. This time you check service and not driver

open service.msc ,make rhev agent startup type to automatic and status to start 
then reboot guest 

failed to reboot ,guest will hang

FYI ,based on comment #41 ,seems it only happened in 32bit guest 

> 
> TIA, Arkady

Looks strange! 
I took the last RHEV-Agent msi ( ic129 ).
Installed only it because new driver already was installed on the guest and have no problem to shutdown the system.
I put screenshots of win7( 32bit ) with steps I did without last one when VM closed ( nothing to show :).

Can you try to do the same?

TIA
Arkady

Created attachment 515652 [details]
service installed from ic129

Created attachment 515653 [details]
Step 1: Picture after RHEV Agent service installed , vioserial already installed before

Created attachment 515654 [details]
Close service. Service release vioserial device, so test app can open device correctly

Created attachment 515655 [details]
Driver TimeOuted read IO correctly. Now app have to be close to release device

Created attachment 515658 [details]
Service restarted and opned vioserial device to work with it.

After service run I have no problem to shutdown/reboot the 32 bit Win7 VM.

Hi, Arkady

thanks for your detailed steps.

I tried some steps as yours with my xp guest(32 bit) ,still can reproduce the issue. 
As long as the rhevagent services is started before poweroff ,guest always hang 

Mike

Hi, Mike!
Can you try the same on Win7 ( 32 bit ) not on XP. That works for me without problems. If hang will happen there too please produce dmp file for me to check.
If that happened for XP only  please do dmp file from XP as a target, during hang.

TIA
Arkady

Created attachment 515693 [details]
RHEVAgent properties

Do you have the same service version/date?
TIA
Arkady

(In reply to comment #52)
> Created attachment 515693 [details]
> RHEVAgent properties
> 
> Do you have the same service version/date?
> TIA
> Arkady

Yes ,exactly this one .

Hi, Arkady 

I tried your steps ,that is install rhev-agent560.msi after install 0.1.13virtio-serial-drivers

Actual Results:
the rhev-agent service can *not* start while guest could shutdown.the step which you mentioned in comment #49.(screen dump will attached)

then I tried to install rhev-agent by using http://bob.eng.lab.tlv.redhat.com/builds/ic129/isoRoot/RHEV-ToolsSetup_560.exe

Actual Results:
the rhev-agent could start automatically ,with guest could *not* reboot/poweroff. (not 100% happen)

bcao--->afrenkel
From all my comments and Marian's comment #41 ,this might be a bug about virtio-agent for 32 bit windows guest.
Today is the embargo day for me to verify or re-assign this bug so I want to confirm with you that ,whether rhev-agent belongs to virtio-win component ? 
If no ,I prefer to mark this one as verified and re-open a new one to further debug this issue. 

Thanks,
Mike

Created attachment 515880 [details]
screen dump mentioned in comment #53

(In reply to comment #33)
> Hi, Mike!
> 
> Vadim proposed another option - .dump command from windbg.
> For that you have to run host VM with WinDbg connected to target VM in the way
> described in http://www.linux-kvm.org/page/WindowsGuestDrivers/GuestDebugging.
> When Target VM hungs, Host VM will produce dump file with .dump command.
> 
> Before open new bug we have to find what cause of the problem ( maybe that old
> bug )
> 
> Best,Arkady

Tried .I start another host to install windbg. but always failed to connect 
referring to screendump

Created attachment 515881 [details]
windbg failed to connect guest

(In reply to comment #55)
> (In reply to comment #33)
> > Hi, Mike!
> > 
> > Vadim proposed another option - .dump command from windbg.
> > For that you have to run host VM with WinDbg connected to target VM in the way
> > described in http://www.linux-kvm.org/page/WindowsGuestDrivers/GuestDebugging.
> > When Target VM hungs, Host VM will produce dump file with .dump command.
> > 
> > Before open new bug we have to find what cause of the problem ( maybe that old
> > bug )
> > 
> > Best,Arkady
> 
> Tried .I start another host to install windbg. but always failed to connect 
> referring to screendump

Hi Mike,
Try using com1 instead of 1394.
Best regards,
Vadim.

(In reply to comment #57)
> (In reply to comment #55)
> Hi Mike,
> Try using com1 instead of 1394.
> Best regards,
> Vadim.

hi, Vadim

seems it still failed .
referring to screendump .

Created attachment 515911 [details]
screendump for failed to connect to guest

(In reply to comment #60)
> Created attachment 515911 [details]
> screendump for failed to connect to guest

Hi Mike,
Please try WXP/W2K3 as a WinDbg host machine.
Cheers,
Vadim.

(In reply to comment #53)

> From all my comments and Marian's comment #41 ,this might be a bug about
> virtio-agent for 32 bit windows guest.

I need to correct myself, I've given it another try and
1. Installed ISO tools from latest ic build (ic134) on clean Win7/32bit and then updated the virtio-serial driver to 0.1-13
2. Installed ISO tools from latest ic build (ic134) on clean WinXP and then updated the virtio-serial driver to 0.1-13
3. Installed RHEV agent from ic129 using msi and virtio-serial driver 0.1-13 on another clean WinXP 

In all cases I could do reboot/shutdown successfully. So I assume when I reported problem with Win7/32bit The mistake had to be between my monitor and my chair.

Problems that I am having are:
With mouse as reported earlier and reported in separated bug (seen in combination with QXL driver)
The shutdown/reboot itself seems to be quite long even on clean installed VM with no additional apps - ~2mins.
Note: Before the VM is shutdowned Admin portal says that VM is paused what I did not see earlier, but it may be issue of RHEV agent, maybe expected behaviour and maybe I only did not notice earlier (?).

(In reply to comment #63)

> In all cases I could do reboot/shutdown successfully. So I assume when I
> reported problem with Win7/32bit The mistake had to be between my monitor and
> my chair.
> 

Ah, I've just remembered what the difference was between previous Windows7/32bit testing and the testing from today - Updates. 
When I reproduced the hang during shutdown my windows 7 VM was not fully updated, I installed it from DVD - Win7 SP1 without any additional windows updates.
On the other hand, today I installed my VM from template which is Win7 fully latest updated.

I've just retested. 
- Shutdown hang occurs on not fully updated VM of Win7/32bit SP1.
- Shutdown works fine on fully latest updated Win7/32bit.

At least in my case.

(In reply to comment #64)
> (In reply to comment #63)
> 
> > In all cases I could do reboot/shutdown successfully. So I assume when I
> > reported problem with Win7/32bit The mistake had to be between my monitor and
> > my chair.
> > 
> 
> Ah, I've just remembered what the difference was between previous
> Windows7/32bit testing and the testing from today - Updates. 
> When I reproduced the hang during shutdown my windows 7 VM was not fully
> updated, I installed it from DVD - Win7 SP1 without any additional windows
> updates.
> On the other hand, today I installed my VM from template which is Win7 fully
> latest updated.
> 
> I've just retested. 
> - Shutdown hang occurs on not fully updated VM of Win7/32bit SP1.
> - Shutdown works fine on fully latest updated Win7/32bit.
Hi,Marian

The image I use is installed via win7sp1 ISO.

thanks,
Mike

> 
> At least in my case.

(In reply to comment #65)
> (In reply to comment #64)
> > (In reply to comment #63)
> > 
> > > In all cases I could do reboot/shutdown successfully. So I assume when I
> > > reported problem with Win7/32bit The mistake had to be between my monitor and
> > > my chair.
> > > 
> > 
> > Ah, I've just remembered what the difference was between previous
> > Windows7/32bit testing and the testing from today - Updates. 
> > When I reproduced the hang during shutdown my windows 7 VM was not fully
> > updated, I installed it from DVD - Win7 SP1 without any additional windows
> > updates.
> > On the other hand, today I installed my VM from template which is Win7 fully
> > latest updated.
> > 
> > I've just retested. 
> > - Shutdown hang occurs on not fully updated VM of Win7/32bit SP1.
> > - Shutdown works fine on fully latest updated Win7/32bit.
> Hi,Marian
> 
> The image I use is installed via win7sp1 ISO.
> 
> thanks,
> Mike

What I wanted to say was: Retest on VM with latest Windows updates, because in my case VM installed from Win7SP1 ISO without any additional windows updates is not shutdowned/rebooted properly, but VM of Win7 with latest Windows Updates can be shutdowned/rebooted properly.

Can you update your VM to latest Windows Updates? (I am trying to find out the differences between your setup and ours so that we can get out of that loop).
Thanks.

> 
> > 
> > At least in my case.

Hi, Mike!

Do you you set correctly ( as site mentioned ) proposed the parameters in target VM ( Win7 ). You can do you it or through msconfig or command line ( bcdedit ).
As Vadim proposed , I too like to use xp as host VM to debug the target.

Best, Arkady

Tnx, Marian!

I'll take that for check
Arkady

From my testing .

win7SP1 w/o windows update .guest will hang during reboot/shutdown.
win7SP! w/ windows update .guest will *not* hang.
winxpsp3 (seems windows xp support in MS is expired ,I can not do windows upadate) ,guest will hand during reboot/shutdown .

Hi, Arkady

According to the dump provided by Marian ,Is "the guest hang" a same bug as the original one ?

Best Regards,
Mike

Hi, Mike!
Yet had no time to check dump but it's on my VM with WinDbg already.
ASAP I'll check it, yet have to treat some BSODs before

Best, Arkady

Sure, you can mark that as verify without problems.
I'm not sure you need to open a new one. I'll tell ( though e-mail ) you, if  you  need to open a new bz, when I'll check the dump and see where the problem : agent/driver/OS...
 
Best, Arkady

(In reply to comment #73)
> Sure, you can mark that as verify without problems.
> I'm not sure you need to open a new one. I'll tell ( though e-mail ) you, if 
> you  need to open a new bz, when I'll check the dump and see where the problem
> : agent/driver/OS...
> 
> Best, Arkady

thanks ,pls let me know if I need to report an new bug .

Based on comment #73 ,comment #66 ,this issue has been fixed .
move status to verified .

Arkady,
I would like to know the result as well and Anyway I do believe It should be tracked somewhere.

Hi ,all ,after some R&D work 

I find that win7sp1 w/o update 
-smp 1 ,guest will hang 
-smp 2 ,guest won't hang 

will attach mini dump & windbg debug info later

Created attachment 522090 [details]
windbg debug info

Created attachment 522091 [details]
memroy dump after hang

Re-tested on virtio-win-prewhql-0.1.16 ,it passed .
for comment #76 ,I report a new bug against qemu-kvm component (Bug 739470) to investigate.

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Do not document.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1542.html