Bug 1868271

Summary: from time to time virsh net-dhcp-leases <net> does not show all leases [rhel-8.2.1.z]
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: Oneata Mircea Teodor <toneata>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: yalzhang <yalzhang>
Severity: high Docs Contact:
Priority: high    
Version: 8.2CC: afazekas, berrange, ehadley, fhubik, jdenemar, jpretori, jsuchane, lmen, lmiccini, mburns, michele, mprivozn, mtessun, pkrempa, toneata, virt-maint, wznoinsk, yalzhang
Target Milestone: rcKeywords: Upstream, ZStream
Target Release: 8.3Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-6.0.0-25.2.el8 Doc Type: Bug Fix
Doc Text:
Cause: When libvirt starts a NATed virtual network (that is with dnsmasq), it configures dnsmasq so that so called leases helper is ran on DHCP traffic. This helper binary updates a JSON file corresponding to the network, e.g. on new lease it adds IP address into the file, on lease timeout it removes it. The file is then used by 'virsh net-dhcp-leases' to report assigned IP addresses. However, to avoid two helpers writing the same file at once (which will certainly result in corrupted JSON) a PID file was used - the leases helper will try to acquire it (note that only one process can lock a file = pid file at once). Consequence: Due to a bug in implementation, the acquisition of the PID file was a single attempt without any wait only => if two dnsmasq-s from two different network decided to run the binary at once, the first process will acquire the PID file and starts updating a JSON file, while the other will fail to acquire it and exits immediately without updating the JSON file => the assigned IP address is not recorded and thus not reported later in virsh (or corresponding C API). Fix: The fix consists of making the PID file acquisition wait until the other process releases it. In code it's passing true instead of false to a function. Result: In the end, since the PID file acquisition waits, lease helpers will mutually exclude with each other, and wait for the data to be updated in the JSON file and thus virsh reports IP addresses correctly, without missing one.
Story Points: ---
Clone Of: 1840307 Environment:
Last Closed: 2020-09-01 09:38:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1840307    
Bug Blocks:    

Comment 7 yalzhang@redhat.com 2020-08-22 12:51:58 UTC
Reproduce the failure on libvirt-6.0.0-25.module+el8.2.1+7154+47ffd890.x86_64 with the test steps in https://bugzilla.redhat.com/show_bug.cgi?id=1840307#c9
And test on libvirt-6.0.0-25.2.module+el8.2.1+7722+a9e38cf3.x86_64 with the same steps, run 20 times, and no failures, the result is as expected.

Comment 9 errata-xmlrpc 2020-09-01 09:38:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: virt:8.2 and virt-devel:8.2 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3586