Bug 1901081

Summary: virt-install race: Domain not found: no domain with matching uuid <unrelated domain>
Product: [Fedora] Fedora Reporter: Martin Pitt <mpitt>
Component: virt-managerAssignee: Cole Robinson <crobinso>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 33CC: berrange, crobinso, jkonecny, philip.wyett
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
URL: https://github.com/virt-manager/virt-manager/pull/194
Whiteboard:
Fixed In Version: virt-manager-3.2.0-2.fc33 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-01-19 02:23:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
reproducer none

Description Martin Pitt 2020-11-24 13:03:02 UTC 
Created attachment 1733001 [details]
reproducer

Description of problem: This is very similar to bug 1894359, except that virt-install cashes on a TOCTOU for domains. Unfortunately I overlooked that case in https://github.com/virt-manager/virt-manager/pull/179


Version-Release number of selected component (if applicable):

virt-manager-common-3.2.0-1.fc33.noarch

How reproducible: Often


Steps to Reproduce:
1. Run attached reproducer in an environment with virt-install

    ATTENTION: THIS DELETES ALL DOMAINS!! So run it in a throwaway environment. The comment at the top of the reproducer shows how to run it in podman from a minimal fedora:33 container.

If it doesn't bite right away, run it several times:

    while /h/reproducer-domain.sh; do sleep 1; done

Actual results: Sometimes crashes with

[Tue, 24 Nov 2020 12:32:49 virt-install 521] DEBUG (cli:208) Launched with command line: /usr/share/virt-manager/virt-install --name real1 --debug --network user --memory 128 --disk path=real1/disk.img,cache=unsafe
[Tue, 24 Nov 2020 12:32:49 virt-install 521] DEBUG (virt-install:207) Distilled --network options: ['user']
[Tue, 24 Nov 2020 12:32:49 virt-install 521] DEBUG (virt-install:139) Distilled --disk options: ['path=real1/disk.img,cache=unsafe']
[Tue, 24 Nov 2020 12:32:49 virt-install 521] DEBUG (cli:224) Requesting libvirt URI default
[Tue, 24 Nov 2020 12:32:49 virt-install 521] DEBUG (cli:227) Received libvirt URI qemu:///session
[Tue, 24 Nov 2020 12:32:50 virt-install 521] DEBUG (guest:463) Setting Guest osinfo name <_OsVariant name=generic>
[Tue, 24 Nov 2020 12:32:50 virt-install 521] DEBUG (installer:396) No media for distro detection.
[Tue, 24 Nov 2020 12:32:50 virt-install 521] DEBUG (installer:398) installer.detect_distro returned=None
[Tue, 24 Nov 2020 12:32:51 virt-install 521] DEBUG (cli:263)   File "/usr/share/virt-manager/virt-install", line 1012, in <module>
    fail(main_e)
  File "/usr/share/virt-manager/virtinst/cli.py", line 263, in fail
    log.debug("".join(traceback.format_stack()))

[Tue, 24 Nov 2020 12:32:51 virt-install 521] ERROR (cli:264) Domain not found: no domain with matching uuid 'f61016e6-1f28-40d1-8e3a-a6f8c00de1f8' (test9)
[Tue, 24 Nov 2020 12:32:51 virt-install 521] DEBUG (cli:266) 
Traceback (most recent call last):
  File "/usr/share/virt-manager/virt-install", line 1005, in <module>
    sys.exit(main())
  File "/usr/share/virt-manager/virt-install", line 993, in main
    guest, installer = build_guest_instance(conn, options)
  File "/usr/share/virt-manager/virt-install", line 560, in build_guest_instance
    installer.set_install_defaults(guest)
  File "/usr/share/virt-manager/virtinst/install/installer.py", line 340, in set_install_defaults
    guest.set_defaults(None)
  File "/usr/share/virt-manager/virtinst/guest.py", line 750, in set_defaults
    dev.set_defaults(self)
  File "/usr/share/virt-manager/virtinst/devices/interface.py", line 271, in set_defaults
    self.macaddr = self.generate_mac(self.conn)
  File "/usr/share/virt-manager/virtinst/devices/interface.py", line 133, in generate_mac
    DeviceInterface.is_conflict_net(conn, mac)
  File "/usr/share/virt-manager/virtinst/devices/interface.py", line 147, in is_conflict_net
    vms = conn.fetch_all_domains()
  File "/usr/share/virt-manager/virtinst/connection.py", line 213, in fetch_all_domains
    self._fetch_cache[key] = self._fetch_all_domains_raw()
  File "/usr/share/virt-manager/virtinst/connection.py", line 201, in _fetch_all_domains_raw
    return [Guest(weakref.proxy(self), parsexml=obj.XMLDesc(0))
  File "/usr/share/virt-manager/virtinst/connection.py", line 201, in <listcomp>
    return [Guest(weakref.proxy(self), parsexml=obj.XMLDesc(0))
  File "/usr/lib64/python3.8/site-packages/libvirt.py", line 544, in XMLDesc
    if ret is None: raise libvirtError ('virDomainGetXMLDesc() failed', dom=self)
libvirt.libvirtError: Domain not found: no domain with matching uuid 'f61016e6-1f28-40d1-8e3a-a6f8c00de1f8' (test9)

Expected results: Domain installation works reliably

I'll cook up a patch and send an upstream PR soon.
Comment 1 Martin Pitt 2020-11-24 13:31:33 UTC 
Sent a fix upstream: https://github.com/virt-manager/virt-manager/pull/194
Comment 2 Jiri Konecny 2021-01-08 13:30:55 UTC 
Hello, the patch was already merged. Could we please solve also this bugzilla?
Comment 3 Fedora Update System 2021-01-11 19:00:32 UTC 
FEDORA-2021-590900ef32 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-590900ef32
Comment 4 Fedora Update System 2021-01-12 01:26:41 UTC 
FEDORA-2021-590900ef32 has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-590900ef32`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-590900ef32

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 5 Fedora Update System 2021-01-19 02:23:53 UTC 
FEDORA-2021-590900ef32 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.