| Summary: | Take too much time when running win-xp/win2k3 whql test | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Qunfang Zhang <qzhang> | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Component: | qemu-kvm | Assignee: | Vadim Rozenfeld <vrozenfe> | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Status: | CLOSED NOTABUG | QA Contact: | Virtualization Bugs <virt-bugs> | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Severity: | high | Docs Contact: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Priority: | high | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Version: | 6.1 | CC: | akong, bcao, dawu, ehabkost, juzhang, leiwang, mdeng, michen, mkenneth, mshao, rhod, virt-maint, vrozenfe | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Target Milestone: | rc | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Target Release: | --- | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hardware: | Unspecified | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| OS: | Unspecified | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Whiteboard: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Doc Text: | Story Points: | --- | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Clone Of: | Environment: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Last Closed: | 2011-09-07 14:15:16 UTC | Type: | --- | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Documentation: | --- | CRM: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Verified Versions: | Category: | --- | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Bug Depends On: | 580954 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Bug Blocks: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Attachments: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
File the bug in qemu-kvm component as the virtio balloon driver has no change from virtio-win-1.1.20 to virtio-win-prewhql-0.1-9. Vadim, right? And the test passed within a few hours in virtio-win-1.1.20. (In reply to comment #2) > File the bug in qemu-kvm component as the virtio balloon driver has no change > from virtio-win-1.1.20 to virtio-win-prewhql-0.1-9. > Vadim, right? > And the test passed within a few hours in virtio-win-1.1.20. Yes, the driver is the same. Execute 'Common Scenario Stress with IO' very slowly on Windows XP guest.It has already spent almost 20 hours so far but it doesn't finish yet.Would you please double check this issue ? Any issues please let me know,thank you. Command line is following as below, /usr/libexec/qemu-kvm -m 2G -smp 2 -cpu cpu64-rhel6,+x2apic -usbdevice tablet -drive file=winxp-serial.raw,format=raw,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device rtl8139,netdev=hostnet0,mac=00:11:25:37:39:f5,bus=pci.0,addr=0x4,id=net0 -boot c -uuid 90d6ba8a-3e67-42c8-b0cc-54e75151e473 -rtc-td-hack -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/monitor-winxp-serial,server,nowait -mon chardev=111a,mode=readline -name winxp-serial -vnc :1 -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=4,bus=pci.0 -chardev socket,id=channel0,path=/var/lib/libvirt/qemu/winxp-serial.channel0,server,nowait -device virtserialport,chardev=channel0,name=org.linux-kvm.port.0,bus=virtio-serial0.0 (In reply to comment #6) > Execute 'Common Scenario Stress with IO' very slowly on Windows XP guest.It > has already spent almost 20 hours so far but it doesn't finish yet.Would you > please double check this issue ? Any issues please let me know,thank you. I'm going to look into this problem in the following days. Thank you, Vadim. Hi Qunfang, Can I see the relevant cpk file? Thank you, Vadim. Created attachment 513545 [details]
XP time
Created attachment 513546 [details]
win2k3-32 update time
Created attachment 513550 [details]
2k3-32
Created attachment 513551 [details]
xp
(In reply to comment #12) > Created attachment 513551 [details] > xp It doesn't look like a balloon submission. Do you experience the same problem with a virtio serial driver when running "common scenario stress with io (WDF preview)" as well? In any case, I would like to see the entire cpk file from balloon submission. Btw, how long does it take to hibernate and then resume the testing system manually? If it takes more than a couple of minutes, could you check that IDE is operating in DMA (not PIO) mode? Best regards, Vadim. Hi Vadim, Balloon and serial all have this issue while doing "common scenarios stress with io" but we only have serial cpk in hand.we can provide you balloon cpk after several days. Doing hibernate and resume manually is very quickly(only several minutes)Any issues,please let me know.thanks. Best regard, Min Deng (In reply to comment #13) > (In reply to comment #12) > > Created attachment 513551 [details] > > xp > > It doesn't look like a balloon submission. Do you experience the same problem > with a virtio serial driver when running "common scenario stress with io (WDF > preview)" as well? > In any case, I would like to see the entire cpk file from balloon submission. > Btw, how long does it take to hibernate and then resume the testing system > manually? If it takes more than a couple of minutes, could you check that IDE > is operating in DMA (not PIO) mode? > > Best regards, > Vadim. FYI ,I prepared another whql environment ,tried 2 times with winxp virtio balloon job of "common scenarios stress with io" both 2 times of this job finished less than 1hour . (In reply to comment #15) > (In reply to comment #13) > > (In reply to comment #12) > > > Created attachment 513551 [details] > > > xp > > > > It doesn't look like a balloon submission. Do you experience the same problem > > with a virtio serial driver when running "common scenario stress with io (WDF > > preview)" as well? > > In any case, I would like to see the entire cpk file from balloon submission. > > Btw, how long does it take to hibernate and then resume the testing system > > manually? If it takes more than a couple of minutes, could you check that IDE > > is operating in DMA (not PIO) mode? > > > > Best regards, > > Vadim. > > FYI ,I prepared another whql environment ,tried 2 times with winxp virtio > balloon job of "common scenarios stress with io" > > both 2 times of this job finished less than 1hour . Right, stress test shouldn't take more than several hours. Looking into XP cpk file I can see a huge gap between runs: 1st test - start 7/12/2011 3:45:42.932PM - end 7/12/2011 3:45:47.932PM 2nd test - start 7/12/2011 3:45:49.402PM - end 7/12/2011 7:33:29.402PM 3rd test - start 7/14/2011 2:16:18.196PM - end 7/14/2011 2:16:18.196PM It's almost two days between 2nd and 3rd tests. No other test was running at this time. Could it be that VM was just paused or switched into manual/debug mode. Best regards, Vadim. Hi Vadim,
QE used the script that you sent to Mike and re-test the issue on win2k3-32 guest.Already attached the log to this bug.
Steps,
1.sync the clock of DTM ,VMs and Host
2.submit the w2k3-32-balloon job in DTM
3.Manually check the host time before & after suspend / resume
a.Manually check from Hibernate to resume last for about 20 minutes
b.Mostly hibernate and resume time was right in vms.I found strange thing was that the hibernating began at 15:29 PM but the vm time changed to 19:50 PM after resuming.The VMs time jumped backward 4 hours.You can see this from the log.And it seemed that not all hibernate&resume had this issue.
The whole w2k3-32-balloon testing lasted for 18 hours and manually check the VM's time was the same to DTM at last.
command line is following as below.
/usr/libexec/qemu-kvm -m 2G -smp 4 -cpu cpu64-rhel6,+x2apic -usbdevice tablet -drive file=win2k3-32-balloon.raw,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device rtl8139,netdev=hostnet0,mac=00:12:4f:15:45:11,bus=pci.0,addr=0x4 -boot c -uuid 8068bdd3-4a9e-496d-af32-39e8eaa6eba8 -rtc-td-hack -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/win2k3-32-balloon,server,nowait -mon chardev=111a,mode=readline -name win2k3-32-balloon -device virtio-balloon-pci,addr=0x5,bus=pci.0 -vnc :1
/usr/libexec/qemu-kvm -m 2G -smp 2 -cpu cpu64-rhel6,+x2apic -usbdevice tablet -drive file=winxp-balloon.raw,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device rtl8139,netdev=hostnet0,mac=00:11:17:1e:35:19,bus=pci.0,addr=0x4 -boot c -uuid 949cffcd-c6cb-414c-98b8-ccb101675398 -rtc-td-hack -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/winxp-balloon,server,nowait -mon chardev=111a,mode=readline -name winxp-balloon -device virtio-balloon-pci,addr=0x5,bus=pci.0 -vnc :1
Please double check this issue.Any issues please let me know Thanks.
Best Regards,
Min Deng
Created attachment 514608 [details]
2k3-balloon-log
Created attachment 514609 [details]
xp-balloon-log
Created attachment 514610 [details]
cpk for w2k3-balloon
Created attachment 514611 [details]
cpk for wxp-balloon
Created attachment 515433 [details]
time log utility
Please find the new version of time log utility as attachment.
I need to see the times, acquired with this utility, while time
is jumping back and forward.
I would like to see contents of HKLM\System\CurrentControlSet\Services\W32Time
registry key. Could you save it as a Registry Export and send as attachment?
Best regards,
Vadim.
(In reply to comment #22) > Created attachment 515433 [details] > time log utility > > Please find the new version of time log utility as attachment. > I need to see the times, acquired with this utility, while time > is jumping back and forward. > I would like to see contents of HKLM\System\CurrentControlSet\Services\W32Time > registry key. Could you save it as a Registry Export and send as attachment? > > Best regards, > Vadim. Forget to mention that Gleb asked to see the host times, every time you start or resume the VM after hibernate. Thank you, Vadim. Created attachment 515630 [details]
Time1
Created attachment 515631 [details]
Time2
Created attachment 515632 [details]
Logs
QE provide related info that were asked by developer.So please have a look at them Best Regards, Min Deng (In reply to comment #27) > QE provide related info that were asked by developer.So please have a look at > them > > Best Regards, > Min Deng Thank you, Min Deng. Unfortunately, seems like the old version of the time log application was used in this test. Can you reproduce the problem and log the times with the latest version of pwrtime utility https://bugzilla.redhat.com/attachment.cgi?id=515433 ? We also need to know the host time on every VM restart. Another question is how you resume VM? Do you have some sort of script which does it automatically, or you do it manually? Best regards, Vadim. Hi Vadim, QE will rerun the job and provide them to you as soon as possible. Resume and Hibernate are all controlled by DTM ,which is a WHQL testing tool from MS.In a word,there is no manually action in the whole process. Any issues please let me know,thanks. Best Regards, Min Deng (In reply to comment #29) > Hi Vadim, > QE will rerun the job and provide them to you as soon as possible. Than you, Min Deng. > Resume and Hibernate are all controlled by DTM ,which is a WHQL testing tool > from MS.In a word,there is no manually action in the whole process. Yes, but hibernating a VM means that a (Linux) process, which hosting that VM, will be eventually closed. Then someone needs to run this process again, at least one time, to resume this VM. It can be done inside of script, which re-starts qemu process automatically. Alternatively, it can be done manually, by a QE engineer. How does it work in your case? Best regards, Vadim > Any issues please let me know,thanks. > > Best Regards, > Min Deng we used a script in this case and attach one for you. Created attachment 515656 [details]
CMD
(In reply to comment #32) > Created attachment 515656 [details] > CMD Thank you. It's what I need. I see you have "date" in you script. Next time, could you provide us with the VM restart times, when you reproduce the time jump problem. Best regards, Vadim. Hi Vadim, I submit a job for windows xp 's balloon and get a log for you. The common scenarios Stress with IO job last for about 2.5 hours this time.and the guest restart 6 times.BTW,I sync the Host&Guest&DTM time before testing. Hi Vadim, I submit a job for windows xp 's balloon and get a log for you. The common scenarios Stress with IO job last for about 2.5 hours this time.and the guest restart 6 times.BTW,I sync the Host&Guest&DTM time before testing. Best regards Min Created attachment 515684 [details]
New Tool Log
(In reply to comment #36) > Created attachment 515684 [details] > New Tool Log Got it. Best regards, Vadim. Hi dengmin, A side note: The first line of the script that restarts qemu seems to be corrupted. Since we try several directions, Gleb and I would like to test the suspend/resume on the same machine, but this time manually, and unrelated to WHQL. Please try the following: Take the machine where the problem is reproduced, unchanged, including the connection to the network and the registry. Do the following: 1. Boot Windows 2. Loop 20 times 3. Print the time/date on guest & Host 4. Wait approximately 20 minutes 5. Print the time/date on guest & Host 6. Hibernate (S4) guest 7. Immediately resume guest (Use the current script that reruns QEMU) Thanks, Ronen. (In reply to comment #35) > Hi Vadim, > > I submit a job for windows xp 's balloon and get a log for you. > The common scenarios Stress with IO job last for about 2.5 hours this time.and > the guest restart 6 times.BTW,I sync the Host&Guest&DTM time before testing. Did you synchronize the Guest with an Internet Time Server? Was it successfully updated? Could you export into a file and post HKLM\System\CurrentControlSet\Control\TimeZoneInformation registry key on your Guest? Kind regards, Vadim. > > Best regards > Min (In reply to comment #39) > (In reply to comment #35) > > Hi Vadim, > > > > I submit a job for windows xp 's balloon and get a log for you. > > The common scenarios Stress with IO job last for about 2.5 hours this time.and > > the guest restart 6 times.BTW,I sync the Host&Guest&DTM time before testing. > Did you synchronize the Guest with an Internet Time Server? > Was it successfully updated? No,I don't synchronize the time with Internet Time Server and just make sure the DTM,Host and Guest time are the same. > > Could you export into a file and post > HKLM\System\CurrentControlSet\Control\TimeZoneInformation registry key on your > Guest? please see the attachment. > > Kind regards, > Vadim. > > > > > Best regards > > Min Created attachment 515813 [details]
Regedit file
(In reply to comment #41) > Created attachment 515813 [details] > Regedit file Hi Min, Can you make sure that DTM controller, DTM client and Host all use the same time-zone and synchronized with Internet time server? Best regards, Vadim. Created attachment 516808 [details]
job hangs
(In reply to comment #38) > Hi dengmin, > > A side note: The first line of the script that restarts qemu seems to be > corrupted. > > Since we try several directions, > Gleb and I would like to test the suspend/resume on the same machine, but this > time manually, and unrelated to WHQL. > > Please try the following: > Take the machine where the problem is reproduced, unchanged, including the > connection to the network and the registry. > Do the following: > 1. Boot Windows > 2. Loop 20 times > 3. Print the time/date on guest & Host > 4. Wait approximately 20 minutes > 5. Print the time/date on guest & Host > 6. Hibernate (S4) guest > 7. Immediately resume guest (Use the current script that reruns QEMU) > > Thanks, Ronen. Hi Ronen, According to your steps I manually record the time that you need.The waiting time I set to 5 minutes,20 minutes.please refer to the attachment. It is strange that the guest time can be corrected by the guest itself if the waiting time is enough. Best Regard, Min Created attachment 516876 [details]
tables-results
It is summarized by me
Created attachment 516877 [details]
pwrtime-log
(In reply to comment #44) > (In reply to comment #38) > > Hi dengmin, > > > > A side note: The first line of the script that restarts qemu seems to be > > corrupted. > > > > Since we try several directions, > > Gleb and I would like to test the suspend/resume on the same machine, but this > > time manually, and unrelated to WHQL. > > > > Please try the following: > > Take the machine where the problem is reproduced, unchanged, including the > > connection to the network and the registry. > > Do the following: > > 1. Boot Windows > > 2. Loop 20 times > > 3. Print the time/date on guest & Host > > 4. Wait approximately 20 minutes > > 5. Print the time/date on guest & Host > > 6. Hibernate (S4) guest > > 7. Immediately resume guest (Use the current script that reruns QEMU) > > > > Thanks, Ronen. > > Hi Ronen, > According to your steps I manually record the time that you need.The > waiting time I set to 5 minutes,20 minutes.please refer to the attachment. > It is strange that the guest time can be corrected by the guest itself if > the waiting time is enough. It must be W32Time service's work. Min, please tell me which time zone was set on this VM. It doesn't look like GMT + 08.00 (Beijing). IIUC it must be GMT - 07.00 set on your system. Do you have any particular reason for doing this? Kind regards, Vadim. > > > Best Regard, > Min Hi Min, Please send us data of the last run from the events-log, under control-panel system-administration (sorry, I only have Windows7, so it might be slightly different). Events-viewer (local) --> Windows-logs --> System (select the events from a date range, and save them as csv file) The next experiments would be: 1. Please repeat the last experiment, but this time disconnected from the internet. No need to repeat all of it. We just need 4 "5 minutes intervals" (or shorter) and 2 "20 minutes intervals" 2. Please do 6 times shutdown/boot instead of suspend/resume, and sample the time ASAP after the boot. Please run 3 of the shutdown/reboot connected to the internet, and 3 disconnected. Since we suspect that there is a problem with the time-zone. Can you please add the time-zone to every time that is reported (Host & Guest). Thanks, Ronen. (In reply to comment #48) > Hi Min, > > Please send us data of the last run from the events-log, under control-panel > system-administration (sorry, I only have Windows7, so it might be slightly > different). > Events-viewer (local) --> Windows-logs --> System (select the events from a > date > range, and save them as csv file) Please see the attachment named xp.csv > > The next experiments would be: > 1. Please repeat the last experiment, but this time disconnected from the > internet. > No need to repeat all of it. We just need 4 "5 minutes intervals" (or > shorter) > and 2 "20 minutes intervals" The first experiment I did last week should include this.Because the guest I used cannot connect to internet at all.All the testing clients are in a testing domain so they cannot reach to internet directly. > 2. Please do 6 times shutdown/boot instead of suspend/resume, and sample the > time > ASAP after the boot. > Please run 3 of the shutdown/reboot connected to the internet, and 3 > disconnected. > Ok,I will do it. > Since we suspect that there is a problem with the time-zone. Can you please add > the time-zone to every time that is reported (Host & Guest). > Yes,I can do it for you.I make sure the time zone are the same from all the machines before testing at every time. > Thanks, Ronen. Created attachment 517082 [details]
xp.csv
(In reply to comment #47) > (In reply to comment #44) > > (In reply to comment #38) > > > Hi dengmin, > > > > > > A side note: The first line of the script that restarts qemu seems to be > > > corrupted. > > > > > > Since we try several directions, > > > Gleb and I would like to test the suspend/resume on the same machine, but this > > > time manually, and unrelated to WHQL. > > > > > > Please try the following: > > > Take the machine where the problem is reproduced, unchanged, including the > > > connection to the network and the registry. > > > Do the following: > > > 1. Boot Windows > > > 2. Loop 20 times > > > 3. Print the time/date on guest & Host > > > 4. Wait approximately 20 minutes > > > 5. Print the time/date on guest & Host > > > 6. Hibernate (S4) guest > > > 7. Immediately resume guest (Use the current script that reruns QEMU) > > > > > > Thanks, Ronen. > > > > Hi Ronen, > > According to your steps I manually record the time that you need.The > > waiting time I set to 5 minutes,20 minutes.please refer to the attachment. > > It is strange that the guest time can be corrected by the guest itself if > > the waiting time is enough. > It must be W32Time service's work. > > Min, please tell me which time zone was set on this VM. It doesn't look like > GMT + 08.00 (Beijing). IIUC it must be GMT - 07.00 set on your system. Do you > have any particular reason for doing this? > Kind regards, > Vadim. Hi Vadim, There is no any special reason for this.Just keep the default time zone after installation(UTC-08.00).As far as I know, in before whql testing,It's just how it is. BTW,QE shouldn't change DTM time free and easy.Because there are many testing clients in the DTM machine pool at the same time,we don't know whether it effects other testing clients(jobs) or not if DTM time changes.At least,after finishing all the left jobs. Thanks, Min > > > > > > Best Regard, > > Min (In reply to comment #49) > The first experiment I did last week should include this.Because the guest I > used cannot connect to internet at all.All the testing clients are in a testing > domain so they cannot reach to internet directly. Which role (or roles) does WIN-66UFGK5UD7I.virtio.com play in your testing domain? Best regards, Vadim. (In reply to comment #52) > (In reply to comment #49) > > The first experiment I did last week should include this.Because the guest I > > used cannot connect to internet at all.All the testing clients are in a testing > > domain so they cannot reach to internet directly. > > Which role (or roles) does WIN-66UFGK5UD7I.virtio.com play in your testing > domain? > It's Active Directory Domino Services. > Best regards, > Vadim. We installed AD and DHCP in the same machine.so it take both AD and DHCP role.(In reply to comment #53) > (In reply to comment #52) > > (In reply to comment #49) > > > The first experiment I did last week should include this.Because the guest I > > > used cannot connect to internet at all.All the testing clients are in a testing > > > domain so they cannot reach to internet directly. > > > > Which role (or roles) does WIN-66UFGK5UD7I.virtio.com play in your testing > > domain? > > > It's Active Directory Domino Services. We installed AD and DHCP in the same machine.so it take both AD and DHCP role. > > > Best regards, > > Vadim. (In reply to comment #52) > (In reply to comment #49) > > The first experiment I did last week should include this.Because the guest I > > used cannot connect to internet at all.All the testing clients are in a testing > > domain so they cannot reach to internet directly. > > Which role (or roles) does WIN-66UFGK5UD7I.virtio.com play in your testing > domain? > > Best regards, > Vadim. Capturing a screen shot for you from our server,which lists more detail info.Hope it can help you. Best Regards, Min Created attachment 517113 [details]
Roles
(In reply to comment #54) > We installed AD and DHCP in the same machine.so it take both AD and DHCP > role.(In reply to comment #53) > > (In reply to comment #52) > > > (In reply to comment #49) > > > > The first experiment I did last week should include this.Because the guest I > > > > used cannot connect to internet at all.All the testing clients are in a testing > > > > domain so they cannot reach to internet directly. > > > > > > Which role (or roles) does WIN-66UFGK5UD7I.virtio.com play in your testing > > > domain? > > > > > It's Active Directory Domino Services. > We installed AD and DHCP in the same machine.so it take both AD and DHCP > role. Thank you, Min Could you check whether it is also in Pacific time-zone? What about DTM Controller? Is it installed on a different system? Which time-zone does it use? Best regards, Vadim. > > > > > Best regards, > > > Vadim. Yes,it is also in Pacific time-zone DTM controller is in Pacific time-zone too.(In reply to comment #57) > (In reply to comment #54) > > We installed AD and DHCP in the same machine.so it take both AD and DHCP > > role.(In reply to comment #53) > > > (In reply to comment #52) > > > > (In reply to comment #49) > > > > > The first experiment I did last week should include this.Because the guest I > > > > > used cannot connect to internet at all.All the testing clients are in a testing > > > > > domain so they cannot reach to internet directly. > > > > > > > > Which role (or roles) does WIN-66UFGK5UD7I.virtio.com play in your testing > > > > domain? > > > > > > > It's Active Directory Domino Services. > > We installed AD and DHCP in the same machine.so it take both AD and DHCP > > role. > > Thank you, Min > > Could you check whether it is also in Pacific time-zone? > > What about DTM Controller? Is it installed on a different system? > Which time-zone does it use? > > Best regards, > Vadim. > Yes,it is also in Pacific time-zone DTM controller is in Pacific time-zone too. Best regards, Min > > > > > > > > > Best regards, > > > > Vadim. Hi Min, Since we suspect that it is about time zones of the guest&host, can you try running the original test with the time-zones of the host and guest set to UTC/GDT +0. I don't know about summer/winter time. If it doesn't seem to solve the problem, then you can stop it in the middle. Thanks, Ronen. (In reply to comment #59) > Hi Min, > Since we suspect that it is about time zones of the guest&host, can you try > running the original test with the time-zones of the host and guest set to > UTC/GDT +0. > I don't know about summer/winter time. > If it doesn't seem to solve the problem, then you can stop it in the middle. > Thanks, Ronen. I would rather ask for adding -rtc base=localtime to the command line. (In reply to comment #60) > (In reply to comment #59) > > Hi Min, > > Since we suspect that it is about time zones of the guest&host, can you try > > running the original test with the time-zones of the host and guest set to > > UTC/GDT +0. > > I don't know about summer/winter time. > > If it doesn't seem to solve the problem, then you can stop it in the middle. > > Thanks, Ronen. > > I would rather ask for adding -rtc base=localtime to the command line. Forget to mention that there is no reason to change time-zone, because VM belongs to domain. Best regards, Vadim. Min, Just to be clear, Please act according to Vadim's comment #60 and do not change the time zone. Thanks, Ronen. Created attachment 517510 [details]
the second booting testing
It's for Ronen.
(In reply to comment #60) > (In reply to comment #59) > > Hi Min, > > Since we suspect that it is about time zones of the guest&host, can you try > > running the original test with the time-zones of the host and guest set to > > UTC/GDT +0. > > I don't know about summer/winter time. > > If it doesn't seem to solve the problem, then you can stop it in the middle. > > Thanks, Ronen. > > I would rather ask for adding -rtc base=localtime to the command line. Hi, Vadim May I know why to use base=localtime if VM belongs to domain? I think we need to isolate guest's time from host in case that the time of host influenced guest's time .I think we need to use -rtc base=localtime,clock=vm Am I right ? Best Regards, Mike (In reply to comment #64) > (In reply to comment #60) > > (In reply to comment #59) > > > Hi Min, > > > Since we suspect that it is about time zones of the guest&host, can you try > > > running the original test with the time-zones of the host and guest set to > > > UTC/GDT +0. > > > I don't know about summer/winter time. > > > If it doesn't seem to solve the problem, then you can stop it in the middle. > > > Thanks, Ronen. > > > > I would rather ask for adding -rtc base=localtime to the command line. > > > Hi, Vadim > > May I know why to use base=localtime if VM belongs to domain? > I think we need to isolate guest's time from host in case that the time of host > influenced guest's time .I think we need to use -rtc base=localtime,clock=vm > Am I right ? > Hi Mike, If the local time is not explicitly specified in command line, QEMU presents RTC time in UTC format, while Windows expects to get it in local time format on boot. (btw, Virtual Machine Manger can choose between UTC and local, based on the "OS Type" information). If you don't specify localtime - you get a backward time jump (3 or 4 hours) on start or resume from hibernation. The forward time jump is the result of time synchronisation between a VM's "soft clock" and a peer in your domain. I really don't know what "clock=vm" does, but specifying the rtc base as localtime will eliminate the backward jump. Without the backward jump, the time difference between a VM's and a peer clocks should be relatively small and be be corrected with a clock skew as an opposite to jump. Kind regards, Vadim. > Best Regards, > Mike (In reply to comment #65) > (In reply to comment #64) > > (In reply to comment #60) > > > (In reply to comment #59) > > > > Hi Min, > > > > Since we suspect that it is about time zones of the guest&host, can you try > > > > running the original test with the time-zones of the host and guest set to > > > > UTC/GDT +0. > > > > I don't know about summer/winter time. > > > > If it doesn't seem to solve the problem, then you can stop it in the middle. > > > > Thanks, Ronen. > > > > > > I would rather ask for adding -rtc base=localtime to the command line. > > > > > > Hi, Vadim > > > > May I know why to use base=localtime if VM belongs to domain? > > I think we need to isolate guest's time from host in case that the time of host > > influenced guest's time .I think we need to use -rtc base=localtime,clock=vm > > Am I right ? > > > Hi Mike, > > If the local time is not explicitly specified in command line, QEMU presents > RTC time in UTC format, while Windows expects to get it in local time format on > boot. (btw, Virtual Machine Manger can choose between UTC and local, based on > the "OS Type" information). If you don't specify localtime - you get a backward > time jump (3 or 4 hours) on start or resume from hibernation. The forward time > jump is the result of time synchronisation between a VM's "soft clock" and a > peer in your domain. > I really don't know what "clock=vm" does, but specifying the rtc base as > localtime will eliminate the backward jump. Without the backward jump, the > time difference between a VM's and a peer clocks should be relatively small > and be be corrected with a clock skew as an opposite to jump. > > Kind regards, > Vadim. > > > Best Regards, > > Mike Hi, Vadim FYI,following is the clock description of man page in qemu-kvm By default the RTC is driven by the host system time. This allows to use the RTC as accurate reference clock inside the guest, specifically if the host time is smoothly following an accurate external reference clock, e.g. via NTP. If you want to isolate the guest time from the host, even prevent it from progressing during suspension, you can set clock to "vm" instead (In reply to comment #66) > Hi, Vadim > > FYI,following is the clock description of man page in qemu-kvm > > By default the RTC is driven by the host system time. This allows to use the > RTC > as accurate reference clock inside the guest, specifically if the host time is > smoothly following an accurate external reference clock, e.g. via NTP. If you > want to isolate the guest time from the host, even prevent it from progressing > during suspension, you can set clock to "vm" instead The default RTC setting works good for Linux guest, because Linux expect that time retrieved by reading RTC CMOS is in UTC format. Windows treats this time as local. Regarding NTP, or, in case of domain, a domain time source - is is also perfectly fine and even recommended to be synchronized with an external time source. Even more, IIRC, w32time tries to save this time in RTC CMOS on shutdown, but QEMU has no place to save it anywhere. So it is a good chance that time will jump on the next boot. Time jump is not so dramatic in the "normal" life, but it can screw up WHQL tests. Hi all, I changed all the HOST&guest and DTM time zone to UTC zero time zone,add parameter'-rtc base=localtime' to my guest command line.I summitted two jobs(both win2k3-64-serial) ,one was in domain another was in WORKGROUP (can reach Internet).So far,the two jobs didn't finished.The whole process last more than 24 hours.I'm afraid the issue still can be reproduced.Could you please double check ? Thank you in advanced. Best Regard, Min (In reply to comment #68) > Hi all, > I changed all the HOST&guest and DTM time zone to UTC zero time zone,add > parameter'-rtc base=localtime' to my guest command line.I summitted two > jobs(both win2k3-64-serial) ,one was in domain another was in WORKGROUP (can > reach Internet).So far,the two jobs didn't finished.The whole process last more > than 24 hours.I'm afraid the issue still can be reproduced.Could you please > double check ? Thank you in advanced. > > Best Regard, > Min Hi Min, Could you post cpk files as soon as the jobs will be finished? thank you, Vadim. Created attachment 518214 [details]
win2k3-64-serail
(In reply to comment #70) > Created attachment 518214 [details] > win2k3-64-serail Thank you Min. There is still something I cannot understand. according to the CPK file the test were running: on 08/12/2011 from 02:41:18 till 02:42:21, then on 08/12/2011 from 12:05:27 till 12:30:11, then on 08/14/2011 from 11:25:38 till 12:09:38, then on 08/15/2011 from 08:11:29 till 08:11:29. Do you know whether VM was running on 08/12/2011 between 02:42:21 and 12:05:27, and then on weekend between 08/12/2011 12:30:11 and 08/14/2011 11:25:38, an finally on Sunday/Monday between 08/14/2011 12:09:38 and 08/15/2011 08:11:29. Best regards, Vadim. (In reply to comment #71) > (In reply to comment #70) > > Created attachment 518214 [details] > > win2k3-64-serail > > Thank you Min. > > There is still something I cannot understand. > according to the CPK file the test were running: > > on 08/12/2011 from 02:41:18 till 02:42:21, then > on 08/12/2011 from 12:05:27 till 12:30:11, then > on 08/14/2011 from 11:25:38 till 12:09:38, then > on 08/15/2011 from 08:11:29 till 08:11:29. > > Do you know whether VM was running on 08/12/2011 > between 02:42:21 and 12:05:27, and then on weekend > between 08/12/2011 12:30:11 and 08/14/2011 11:25:38, > an finally on Sunday/Monday between 08/14/2011 12:09:38 > and 08/15/2011 08:11:29. > Hi Vadim, The VM should be running during the first and the second time bucket. For the last,it hit an BSOD issue and I reset the VM manually .We are investigating the issue now. (Job named Disable and Enable with IO). Best regards, Min Deng(邓敏) > Best regards, > Vadim. Hi Min, I know that you are investigating the issue now. Can you also get us the logs of the last run so that we can verify that the time changes are over (which is what we were chasing prior to this BSOD). Thanks, Ronen. (In reply to comment #73) > Hi Min, > I know that you are investigating the issue now. > Can you also get us the logs of the last run so that we can verify that the > time changes are over (which is what we were chasing prior to this BSOD). > Thanks, Ronen. Hi Ronen, Re-run Job named Disbable and Enable with IO ,it has passed in another job. Thanks, Min Created attachment 520326 [details]
2k3-serial-time-jump
There is an BSOD for job common scenarios stress with io.Already submit a single job for it.Still running.If the job finished,I will attach it at once.
Hi Min, In order to see the time jumps we need the equivalent of the files pwrtime.log and/or xp.csv that you sent us in the past. Assuming that you do not have the pwrtime.log from the last run, I think that the easiest would be to create the csv file from the Windows events log, and it might be enough. Thanks. Created attachment 520733 [details]
Logs
Hi Ronen,
Please see Logs.csv.
Min
(In reply to comment #76) > Hi Min, > > In order to see the time jumps we need the equivalent of the files pwrtime.log > and/or xp.csv that you sent us in the past. > Assuming that you do not have the pwrtime.log from the last run, I think that > the easiest would be to create the csv file from the Windows events log, and it > might be enough. > > Thanks. Hi, Ronen files referring to comment #77 , Best Regards, Mike Hi mike, Min, QE, Following the changes that I made to the clock on the involved machines, can you please check that it is fixed? I think that in order to be on the safe side, we better set the time zone of all the guests (images) to Beijing. If it is not fixed, please attach the event ".csv" file as well. Thanks, Ronen. (In reply to comment #79) > Hi mike, Min, QE, > > Following the changes that I made to the clock on the involved machines, can > you please check that it is fixed? > I think that in order to be on the safe side, we better set the time zone of > all the guests (images) to Beijing. > > If it is not fixed, please attach the event ".csv" file as well. > > Thanks, Ronen. Hi Ronen, According to your suggestion,I changed the guest and DTM (we had better change it too) time zone to Bejing's.It isn't reproduced at this time. Thanks Min Created attachment 521560 [details]
Attached submission for this bug
(In reply to comment #80) > (In reply to comment #79) > > Hi mike, Min, QE, > > > > Following the changes that I made to the clock on the involved machines, can > > you please check that it is fixed? > > I think that in order to be on the safe side, we better set the time zone of > > all the guests (images) to Beijing. > > > > If it is not fixed, please attach the event ".csv" file as well. > > > > Thanks, Ronen. > > Hi Ronen, > > According to your suggestion,I changed the guest and DTM (we had better > change it too) time zone to Bejing's.It isn't reproduced at this time. > > Thanks > Min It isn't reproduced on window 2003 64 bits serial at the first time but I still need to re-confirmed it.I re-submit a new job for it this morning. Thanks Min Hi Ronen, The second job finished so far,and I collected the time log via Vadim's tool.you can also get entire log from attachment. 0 5:9:2011-13:24:31:213 12959702671213 5:9:2011-5:24:31:213 12959673871213 109656 108845 0 unknown 1 5:9:2011-13:24:31:345 12959702671345 5:9:2011-5:24:31:345 12959673871345 109781 108968 4 suspend 2 5:9:2011-13:24:34:581 12959702674581 5:9:2011-5:24:34:581 12959673874581 113031 112351 18 resumeautomatic 3 5:9:2011-13:24:34:596 12959702674596 5:9:2011-5:24:34:596 12959673874596 113046 112359 7 resumesuspend 4 5:9:2011-13:25:31:763 12959702731763 5:9:2011-5:25:31:763 12959673931763 168531 169765 0 unknown 5 5:9:2011-13:25:31:958 12959702731958 5:9:2011-5:25:31:958 12959673931958 168718 169771 4 suspend As far as I concerned,the time was strange.Though the bug wasn't reproduced at this time we cannot think the problem disappear. Created attachment 521618 [details]
Guest's time log
Created attachment 521619 [details]
Host time
Hi Min, At least the time is not moving backwards. good news. What is it that was strange to you? How about the original bug? How long did it take to run the test? Thanks, Ronen. (In reply to comment #86) > Hi Min, > > At least the time is not moving backwards. good news. > What is it that was strange to you? > > How about the original bug? How long did it take to run the test? > > Thanks, Ronen. Hi Ronen and Vadim, please have a look on this - 1 5:9:2011-13:24:31:345 12959702671345 5:9:2011-5:24:31:345 12959673871345 109781 108968 4 suspend The second time is moving forward eight hours. Vadim,as you are the creator of the pwrtime.exe.could you please have a look at the logs,which is collected from win2003 64bits guest (serial) by me. 12 hours is enough for window2003 64bits serial job,as to windows xp balloon,I think,I had better submit an job for it today.I will tell you ASAP. Thanks for each developer's effort. Best regards, Min (In reply to comment #87) > (In reply to comment #86) > > Hi Min, > > > > At least the time is not moving backwards. good news. > > What is it that was strange to you? > > > > How about the original bug? How long did it take to run the test? > > > > Thanks, Ronen. > > Hi Ronen and Vadim, > > please have a look on this - > 1 5:9:2011-13:24:31:345 12959702671345 5:9:2011-5:24:31:345 12959673871345 > 109781 108968 4 suspend > The second time is moving forward eight hours. Hi Min, Sorry that I didn't provide a good explanation regarding the collected data. Here it is: - first column - run number - second column - 5:9:2011-13:24:31:345 - local time - third column - 12959702671345 - local time in milliseconds - fourth column - 5:9:2011-5:24:31:345 - system time - sixth column - 12959673871345 - system time in milliseconds - seventh column - 109781 - tick count The second column is the most important. If you see that time is increasing continuously between runs, then everything works fine. Kind regards, Vadim. |
Description of problem: Running xp-balloon job "common scenario stress with io (WDF preview)" took 68 hours this time on qemu-kvm-0.12.1.2-2.158.el6.x86_64. Tested on the qemu-kvm-0.12.1.2-2.147.el6.x86_64 before, this job can finish within a few hours (usually less than 6 hours). I reinstall a new host with 20110406 tree and retest with a fresh xp guest again.But the job is still running after 1 day (It does not finish currently). Version-Release number of selected component (if applicable): kernel-2.6.32-120.el6.x86_64 qemu-kvm-0.12.1.2-2.158.el6.x86_64 virtio-win-prewhql-0.1-9 The problem does not happen on the following packages: kernel-2.6.32-117.el6.x86_64 qemu-kvm-0.12.1.2-2.147.el6.x86_64 virtio-win-1.1.20 (Actually virtio-balloon driver has no change from 1.1.20 to 0.1-9, Vadim, right?) How reproducible: Always Steps to Reproduce: 1. Prepare whql test environment. 2. Boot guest with balloon device and install driver. /usr/libexec/qemu-kvm -m 2G -smp 2 -cpu cpu64-rhel6,+x2apic -usbdevice tablet -drive file=winxp-balloon-9.raw,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device rtl8139,netdev=hostnet0,mac=00:10:4a:1A:47:19,bus=pci.0,addr=0x4 -boot c -uuid dd6ea738-b43e-4c1c-9a82-6eedaac0d27a -rtc-td-hack -no-kvm-pit-reinjection -chardev socket,id=111a,path=/tmp/monitor-winxp-balloon-9,server,nowait -mon chardev=111a,mode=readline -name winxp-balloon-9 -device virtio-balloon-pci,addr=0x5,bus=pci.0 -vnc :1 3. Run job "common scenario stress with io (WDF preview)" Actual results: Took 3 days to finish the job. Expected results: The job should finish within a few hours.(less than 6 hours) Additional info: # free total used free shared buffers cached Mem: 8048548 6254056 1794492 0 248256 3193128 -/+ buffers/cache: 2812672 5235876 Swap: 10289144 0 10289144 (16 cpu in total, here only list the last one.) Host info: processor : 15 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz stepping : 5 cpu MHz : 2260.759 cache size : 8192 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 4521.30 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: