Description of problem: On Debian Squeeze (6.0), this seems to be some kind of bug with xenstored. For example, let's take our servers "vm7" and "vm10" into consideration, which are configured exactly the same, Debian Squeeze vanilla, no backports. "vm7", as stated below, has more Domain-0 entries, but executes virsh commands (f.e. virsh pwd, virsh list etc) faster (under a second) than "vm10". [vm7 ~]# ./dump-xenstore.sh | grep Domain-0 | wc -l 22 [vm10 ~]# ./dump-xenstore.sh | grep Domain-0 | wc -l 14 If you strace xenstored by attaching to it's process, you'll see that even the basic virsh command, like virsh pwd, doesn't finish until it "is done" with xenstored. When xenstored finishes passing data to virsh, virsh gives it's output. From the strace output, xenstored on vm10 gives ~20x more ENOENT errors than xenstored strace on vm7: [vm7 ~]# grep ENOENT xenstored.strace | wc -l 110 [vm10 ~]# grep ENOENT xenstored.strace | wc -l 2232 From what I can deduce, there is something "wrong" with vm10's xenstored's tdb, because it points to unexisting data and takes a lot of time on that? [vm10 ~]# grep tdb.0x17a47b0 xenstored.strace 2013 open("/var/lib/xenstored/tdb.0x17a47b0", O_WRONLY|O_CREAT|O_TRUNC, 0640) = 154 2013 open("/var/lib/xenstored/tdb.0x17a47b0", O_RDWR) = 155 2013 rename("/var/lib/xenstored/tdb.0x17a47b0", "/var/lib/xenstored/tdb") = 0 2013 unlink("/var/lib/xenstored/tdb.0x17a47b0") = -1 ENOENT (No such file or directory) 2013 open("/var/lib/xenstored/tdb.0x17a47b0", O_WRONLY|O_CREAT|O_TRUNC, 0640) = 153 2013 open("/var/lib/xenstored/tdb.0x17a47b0", O_RDWR) = 156 2013 rename("/var/lib/xenstored/tdb.0x17a47b0", "/var/lib/xenstored/tdb") = 0 2013 unlink("/var/lib/xenstored/tdb.0x17a47b0") = -1 ENOENT (No such file or directory) ---snip--- (last 4 lines repeated 80 times) NB: same slowdown appears on more dom0s. Version-Release number of selected component (if applicable): 0.8.3-5+squeeze2 How reproducible: virsh list or any other virsh sub-command thath involves interaction with managing domains. eg. "virsh help" works just fine. Steps to Reproduce: 1. virsh define 2. virsh list 3. virsh console Actual results: all commands lag > 10 secs. Expected results: under 1-2 secs response Additional info:
I'm sorry this never received a response, but given the age of this bug and usage of the now-deprecated xend toolstack, I'm closing this as DEFERRED. If anyone can still reproduce with modern libvirt, please reopen