*** Description of problem: The libvirt documentation at <https://libvirt.org/formatdomain.html#userspace-slirp-or-passt-connection> suggests that switching from slirp to passt is a simple matter of adding a <backend type='passt'/> child element to <interface>. However, this backend change results in guest-visible differences. Assume that we have <ip family='ipv4' address='169.254.0.0' prefix='16'/> as the sole <ip> child element within <interface>, and we change the userspace network stack from slirp to passt, by inserting <backend type='passt'/> That causes the following guest-visible changes: (1) The guest IP address changes from 169.254.2.15 to 169.254.0.0. This is for the following reason: libvirt and QEMU *both* have preexistent bugs wherein their documentations (independently) state that the slirp IP address specification is the *exact* IP address of the guest, but their *implementations* actually take the guest IP address as a *subnet base address*. Then slirp goes ahead and assigns .2.15 within that subnet to the guest, and this slirp behavior *masks* the discrepancy between the documentation and the implementation (for both QEMU and libvirt). With passt enabled however, no such masking occurs. The IP address 169.254.0.0 is passed to passt with the "--address" option (libvirt commit a56f0168d576), and passt interprets that *verbatim*. Therefore the guest IP address changes to 169.254.0.0 -- which is invalid, a zero-suffix IP address is not good for an endpoint, it's only good as a subnet base address. This issue is BTW easy enough to work around: we just need to specify <ip family='ipv4' address='169.254.2.15' prefix='16'/> in the domain XML. That's what I'm doing for bug 2184967. (2) The host IP address, as seen by the guest, changes from 169.254.2.2 to the *default gateway IP address* that is enabled on the *host*. Slirp defaults to .2.2 as the host address. Libvirt does not pass the "--gateway" option to passt, therefore passt copies the gateway address from the host to the guest (with passt's goal being to imitate the host environment as closely as possible in the guest). Note that the domain XML <https://libvirt.org/formatdomain.html#ip-configuration> provides an element such as <route family='ipv4' address='...' prefix='...' gateway='...'/> which *in theory* could be used for setting the default gateway in the guest. However, <route> is not hooked up to the QEMU driver at all; the documentation states it is only used by the LXC driver. Indeed, if <route> were hooked up to QEMU, then it would have to be passed to both slirp (with the "-netdev user,host=..." property) and to passt (with the "--gateway" option). (3) The host MAC address, as seen by the guest, changes from 52:56:00:00:00:02 (which is the SLIRP default), to the *guest NIC's* MAC, which is wrong. This is a direct consequence of libvirt commit a56f0168d576 passing a bogus "--mac-addr" option to passt. Libvirt fetches the guest-side MAC from the domain config, and passes it to passt as the host-side MAC (to be seen by the guest). That's doubly wrong: not only is it incompatible with slirp, but it leads to MAC duplication (both the guest and the host will have identical MACs, as seen by the guest). (4) The guest subnet mask changes as well, from /16 to /1, in my case anyway. This is a consequence of issues (1) and (2), taken together. Under (2), my default gateway (on my host) is 192.168.0.1. Under (1), my guest IP address is 169.254.2.15, the guest subnet is 169.254.0.0, and the guest netmask is /16. Now, the default gateway for an endpoint (192.168.0.1) is supposed to be in the same subnet as the endpoint itself (169.254.2.15). Therefore, I *think* passt calculates the subnet mask by AND-ing 169 with 192 (in the most significant byte). The result is 128.0.0.0, hence the derived /1 mask. *** Version-Release number of selected component (if applicable): libvirt-daemon-9.0.0-10.1.el9_2.x86_64 passt-0^20230222.g4ddbcb9-2.el9_2.x86_64 *** How reproducible: Always. *** Steps to Reproduce: 1. Create a domain XML with the following <interface> element: <interface type="user"> <mac address='52:54:00:41:2c:49'/> <source network='default'/> <model type="virtio"/> <ip family="ipv4" address="169.254.0.0" prefix="16"/> </interface> 2. Boot a Linux guest in the domain. 3. Run the following commands in the guest, and take note of the outputs: ping -c 3 8.8.8.8 ip neighbor ip addr ip route 4. Insert the following child element in the domain XML, under <interface>: <backend type='passt'/> 5. Repeat the commands from step 3 in the guest. *** Actual results: - The ping command succeeds in both steps 3 and 5. - The "ip neighbor" output changes, from step 3 to step 5, from "52:56:00:00:00:02" to "52:54:00:41:2c:49", with the latter being the virtio-net NIC's MAC. - The "ip addr" output changes from "169.254.2.15/16" to "169.254.0.0/1"; both the IP address and the subnet mask are wrong. - The "ip route" output changes from "169.254.2.2" to whatever is your default IPv4 gateway on your host. *** Expected results: - Nothing to change for the "ping" command, as it works in both cases. - The "ip neighbor" output should remain the same, in step 5, as it was in step 3 -- 52:56:00:00:00:02. - The "ip route" output should remain "169.254.2.2". - In the "ip addr" output, the netmask should remain /16. - In the "ip addr" output, the IP address should *maybe* remain 169.254.2.15. However, this specific detail is ambiguous. As I said above, it's *already* a bug in both QEMU and libvirt that an IP address spec of 169.254.0.0 results in the guest seeing 169.254.2.15. Therefore, we can argue that, if a user wants 169.254.2.15, they should *specify* 169.254.2.15 too. If that advice is followed, then the "ip addr" output is actually *correct* in step 5 (not the netmask part, just the address part). *** Additional info: I don't know how important these issues are in practice. I've noticed them because I'm trying to keep the libvirt and direct backends of libguestfs as consistent as possible, while enabling passt for libguestfs (under bug 2184967). I can have passt provide the guest with an environment that is identical to SLIRP's, using the --address, --netmask, --mac-addr, and --gateway options, in the direct backend. But in the libvirt backend, the above issues appear; I can't control the details well enough via the domain XML.
Wow! We needed you around to keep me in line when doing the original passt support! I would consider these each separate issues possibly worthy of separate BZes, but here's a quick response to each: 1) Yes, this is due to SLIRP having an "odd" idea of how the configuration for a guest IP should be done vs. passt's more sane idea, combined with me ASSuming that SLIRP would be doing the sane thing. In my opinion, replicating SLIRP's mistakes for passt would not be a good idea, and we should instead document the change in behavior (is there a way to force SLIRP to assign a specific IP? If so, maybe a "common configuration" could be achieved by detecting that the specified IP was a host address (rather than network address) and making whatever (if any) change was necessary to the SLIRP/qemu commandline. At the very least, the documentation in formatdomain.html needs to be changed to reflect reality. 2) How does the guest know what the host's IP address is in the case of SLIRP? Is it (or maybe something in libguestfs?) just implying it based on the network address that's provided? Is the host's IP actually used for something in the guest or host? I'm not sure what's being requested here; just that the default gateway remain the same? 3) This is definitely just a bonehead mistake on my part, again caused by making an assumption rather than actually reading and comprehending the full documentation. I just posted a fix here: https://listman.redhat.com/archives/libvir-list/2023-July/240713.html Let me know if you'd rather not have yourself listed in the Reported-by: tag. 4) So, the only reason that the guest-side <route> entries isn't implemented for QEMU is because it was originally added (by me? I can't remember for sure) for LXC, and it's not possible for QEMU to directly manipulate guest routing tables (or IP addresses for that matter) for tap-based networking, which was the only kind of QEMU networking I was interested in at the time. With passt, it would certainly be reasonable to implement the one special case of <route> that can be configured - setting the --gateway based on <route family='ipv4' address='0.0.0.0' prefix='0' gateway='blah'/> in the XML. I don't know (yet) if that would solve the difference in interface netmask in the guest though (I need time and neural bandwidth to test it out).
(1) From libguestfs's perspective, where we certainly need to preserve SLIRP support, I'd prefer if SLIRP-versus-passt would be "transparent", as seen from the appliance. So I'm fine with any changes to the defaults, as long as the libvirt XML lets me specify such details for passt that I get a guest-side (appliance-side) environment that is very similar to SLIRP's. NB I'm not implying that all of these details actually *matter* for libguestfs networking. There is one bit that certainly matters (the host's IP address, as seen from the guest), and from there, I went for all the other details too that I could collect. (2) The host's IP address, as seen by the guest, is exposed via DHCP; it is used at least for name resolving, for setting the default route, and (dependent on guest "payload") by guest applications too. For example, one of the tests in the libguestfs test suite sets up an rsync server on the host, and connects to it from the guest. That depends on the test suite knowing the host's IP address, as seen by the guest, in advance. It's admittedly quite hacky, so if we can make that test case smarter (so that the appliance determine the host IP address automatically...), that might as well obviate this whole BZ, perhaps. (3) I've replied in-thread. (4) This seems like a lot of work, and again its utility depends on (2). Thanks!
Replying to (1) again: as I said above, this problem can be worked around (or mitigated) quite easily, in fact; and it's part of my libguestfs series that I just sent (see <https://bugzilla.redhat.com/show_bug.cgi?id=2184967#c12>, patch 1 out of 7). The trick is to just use the specific "host address" 169.254.2.15 ("host" in the networking sense, not in the virtualization sense...), as opposed to the network address 169.254.0.0, in the domain XML. SLIRP will behave identically, and passt will start doing the right thing. So this may only need a documentation change, or not even that.
Freshly back from vacation, I've pushed the patch to remove --mac-addr setting. (I unfortunately forgot to link this BZ in the commit log) commit a8262cb331db0f1a706edbadffc2cd5b01e44be6 Author: Laine Stump <laine> Date: Thu Jul 13 11:51:28 2023 -0400 qemu: don't add --mac-addr option to passt commandline