Bug 2222766

*** Description of problem:

The libvirt documentation at
<https://libvirt.org/formatdomain.html#userspace-slirp-or-passt-connection>
suggests that switching from slirp to passt is a simple matter of adding
a <backend type='passt'/> child element to <interface>. However, this
backend change results in guest-visible differences.

Assume that we have

  <ip family='ipv4' address='169.254.0.0' prefix='16'/>

as the sole <ip> child element within <interface>, and we change the
userspace network stack from slirp to passt, by inserting

  <backend type='passt'/>

That causes the following guest-visible changes:

(1) The guest IP address changes from 169.254.2.15 to 169.254.0.0.

This is for the following reason: libvirt and QEMU *both* have
preexistent bugs wherein their documentations (independently) state that
the slirp IP address specification is the *exact* IP address of the
guest, but their *implementations* actually take the guest IP address as
a *subnet base address*. Then slirp goes ahead and assigns .2.15 within
that subnet to the guest, and this slirp behavior *masks* the
discrepancy between the documentation and the implementation (for both
QEMU and libvirt).

With passt enabled however, no such masking occurs. The IP address
169.254.0.0 is passed to passt with the "--address" option (libvirt
commit a56f0168d576), and passt interprets that *verbatim*. Therefore
the guest IP address changes to 169.254.0.0 -- which is invalid, a
zero-suffix IP address is not good for an endpoint, it's only good as a
subnet base address.

This issue is BTW easy enough to work around: we just need to specify

  <ip family='ipv4' address='169.254.2.15' prefix='16'/>

in the domain XML. That's what I'm doing for bug 2184967.

(2) The host IP address, as seen by the guest, changes from 169.254.2.2
to the *default gateway IP address* that is enabled on the *host*.

Slirp defaults to .2.2 as the host address. Libvirt does not pass the
"--gateway" option to passt, therefore passt copies the gateway address
from the host to the guest (with passt's goal being to imitate the host
environment as closely as possible in the guest).

Note that the domain XML
<https://libvirt.org/formatdomain.html#ip-configuration> provides an
element such as

  <route family='ipv4' address='...' prefix='...' gateway='...'/>

which *in theory* could be used for setting the default gateway in the
guest. However, <route> is not hooked up to the QEMU driver at all; the
documentation states it is only used by the LXC driver. Indeed, if
<route> were hooked up to QEMU, then it would have to be passed to both
slirp (with the "-netdev user,host=..." property) and to passt (with the
"--gateway" option).

(3) The host MAC address, as seen by the guest, changes from
52:56:00:00:00:02 (which is the SLIRP default), to the *guest NIC's*
MAC, which is wrong.

This is a direct consequence of libvirt commit a56f0168d576 passing a
bogus "--mac-addr" option to passt. Libvirt fetches the guest-side MAC
from the domain config, and passes it to passt as the host-side MAC (to
be seen by the guest).

That's doubly wrong: not only is it incompatible with slirp, but it
leads to MAC duplication (both the guest and the host will have
identical MACs, as seen by the guest).

(4) The guest subnet mask changes as well, from /16 to /1, in my case
anyway.

This is a consequence of issues (1) and (2), taken together. Under (2),
my default gateway (on my host) is 192.168.0.1. Under (1), my guest IP
address is 169.254.2.15, the guest subnet is 169.254.0.0, and the guest
netmask is /16. Now, the default gateway for an endpoint (192.168.0.1)
is supposed to be in the same subnet as the endpoint itself
(169.254.2.15). Therefore, I *think* passt calculates the subnet mask by
AND-ing 169 with 192 (in the most significant byte). The result is
128.0.0.0, hence the derived /1 mask.


*** Version-Release number of selected component (if applicable):
libvirt-daemon-9.0.0-10.1.el9_2.x86_64
passt-0^20230222.g4ddbcb9-2.el9_2.x86_64


*** How reproducible:
Always.


*** Steps to Reproduce:
1. Create a domain XML with the following <interface> element:

  <interface type="user">
    <mac address='52:54:00:41:2c:49'/>
    <source network='default'/>
    <model type="virtio"/>
    <ip family="ipv4" address="169.254.0.0" prefix="16"/>
  </interface>

2. Boot a Linux guest in the domain.

3. Run the following commands in the guest, and take note of the
outputs:

ping -c 3 8.8.8.8
ip neighbor
ip addr
ip route

4. Insert the following child element in the domain XML, under
<interface>:

    <backend type='passt'/>

5. Repeat the commands from step 3 in the guest.

*** Actual results:

- The ping command succeeds in both steps 3 and 5.

- The "ip neighbor" output changes, from step 3 to step 5, from
  "52:56:00:00:00:02" to "52:54:00:41:2c:49", with the latter being the
  virtio-net NIC's MAC.

- The "ip addr" output changes from "169.254.2.15/16" to
  "169.254.0.0/1"; both the IP address and the subnet mask are wrong.

- The "ip route" output changes from "169.254.2.2" to whatever is your
  default IPv4 gateway on your host.


*** Expected results:

- Nothing to change for the "ping" command, as it works in both cases.

- The "ip neighbor" output should remain the same, in step 5, as it was
  in step 3 -- 52:56:00:00:00:02.

- The "ip route" output should remain "169.254.2.2".

- In the "ip addr" output, the netmask should remain /16.

- In the "ip addr" output, the IP address should *maybe* remain
  169.254.2.15. However, this specific detail is ambiguous. As I said
  above, it's *already* a bug in both QEMU and libvirt that an IP
  address spec of 169.254.0.0 results in the guest seeing 169.254.2.15.
  Therefore, we can argue that, if a user wants 169.254.2.15, they
  should *specify* 169.254.2.15 too. If that advice is followed, then
  the "ip addr" output is actually *correct* in step 5 (not the netmask
  part, just the address part).


*** Additional info:

I don't know how important these issues are in practice. I've noticed
them because I'm trying to keep the libvirt and direct backends of
libguestfs as consistent as possible, while enabling passt for
libguestfs (under bug 2184967). I can have passt provide the guest with
an environment that is identical to SLIRP's, using the --address,
--netmask, --mac-addr, and --gateway options, in the direct backend. But
in the libvirt backend, the above issues appear; I can't control the
details well enough via the domain XML.