Created attachment 1703236 [details]
RHV shows ip info

Created attachment 1703237 [details]
Firstboot log

Please provide engine and vdsm log as well

I looked at the VM and it seems like a bug in qemu-ga. When I query the agent directly I get:

virsh # qemu-agent-command esx6.7-win2019-static-ip '{"execute":"guest-network-get-interfaces"}'
{"return":[{"name":"Ethernet","ip-addresses":[{"ip-address-type":"ipv6","ip-address":"fe80::3429:9746:f288:9279%8","prefix":64},{"ip-address-type":"ipv4","ip-address":"169.254.146.121","prefix":16}],"statistics":{"tx-packets":0,"tx-errs":0,"rx-bytes":0,"rx-dropped":0,"rx-packets":0,"rx-errs":0,"tx-bytes":0,"tx-dropped":0},"hardware-address":"00:50:56:ac:90:79"},{"name":"Loopback Pseudo-Interface 1","ip-addresses":[{"ip-address-type":"ipv6","ip-address":"::1","prefix":128},{"ip-address-type":"ipv4","ip-address":"127.0.0.1","prefix":8}],"statistics":{"tx-packets":0,"tx-errs":0,"rx-bytes":0,"rx-dropped":0,"rx-packets":0,"rx-errs":0,"tx-bytes":0,"tx-dropped":0}}]}

This matches what we see in engine. I wonder where does the agent get the information from.

Created attachment 1710467 [details]
vdsm log

Created attachment 1710468 [details]
engine log

The vdsm.log is empty. Can you please check?

Created attachment 1711062 [details]
new vdsm.log

In light of https://bugzilla.redhat.com/show_bug.cgi?id=1867575#c15, do I understand this correctly that the story goes like this:
1. We have a VM in VMware that is set with static IP
2. We ask virt-v2v to change the static IP of that NIC (00:50:56:ac:90:79) to 10.66.147.203
3. After the conversion, the NIC is not set with requested static IP 10.66.147.203
4. Instead, the NIC gets to the following state:
4.1 The "Internet Protocol Version 4 (TCP/IPv4) Properties" dialog shows that the NIC is set to use an empty static IP - something that is typically blocked by Windows UI
4.2 The NIC is displayed as 'Media disconnected' in the output of 'ipconfig' within the guest
4.3 The NIC is assigned with a private IP 169.254.146.121, as if it can't access a DHCP server
5. RHV exposes the private IP 169.254.146.121 that the NIC is assigned with

While we can argue on whether that private IP should be displayed in RHV or not in this case (and if not, which component should filter it out), I wonder how interesting that is.
Isn't the real issue here that the requested static IP (10.66.147.203) is not properly set (#3) and instead we end up with some invalid state (#4)?

In my perspective the questions is why link is disconnected ("Media disconnected")? Does management layer set's the link to down state?
On other hand the value of IP is actually "valid" (according to MS documentation): https://docs.microsoft.com/en-us/windows-server/troubleshoot/how-to-use-automatic-tcpip-addressing-without-a-dh

(In reply to Arik from comment #10)
> While we can argue on whether that private IP should be displayed in RHV or
> not in this case (and if not, which component should filter it out), I
> wonder how interesting that is.

That depends. The fact that we report IPs that are unreachable makes it difficult for other tools to connect to the guest. If the NIC is unpluged in oVirt configuration then oVirt knows that it should not report the IP. But if the interface is down for other reasons... ?

But as it stands now it is definitely very low priority. I am still puzzled how often can such situation happen apart from the guest being miss-configured.

> Isn't the real issue here that the requested static IP (10.66.147.203) is
> not properly set (#3) and instead we end up with some invalid state (#4)?

Yes and yes.



(In reply to Yan Vugenfirer from comment #11)
> In my perspective the questions is why link is disconnected ("Media
> disconnected")? Does management layer set's the link to down state?
> On other hand the value of IP is actually "valid" (according to MS
> documentation):
> https://docs.microsoft.com/en-us/windows-server/troubleshoot/how-to-use-
> automatic-tcpip-addressing-without-a-dh

My initial assumption was that the interface was disabled by Windows because it was miss-configured, but if that's not the case then this is a good question.

(In reply to Tomáš Golembiovský from comment #12)
> (In reply to Arik from comment #10)
> > While we can argue on whether that private IP should be displayed in RHV or
> > not in this case (and if not, which component should filter it out), I
> > wonder how interesting that is.
> 
> That depends. The fact that we report IPs that are unreachable makes it
> difficult for other tools to connect to the guest. If the NIC is unpluged in
> oVirt configuration then oVirt knows that it should not report the IP.

Ok, it seems it's not like that. If I unplug the NIC in oVirt it is disabled by NetworkManager and that's why it is not reported (even by qemu-ga). But when I bring it up manually it shows in oVirt UI even though the link is still down. In oVirt API we seem to report some cached IP address even if the address is no longer reported by guest.


> But if the interface is down for other reasons... ?

(In reply to Tomáš Golembiovský from comment #13)
> (In reply to Tomáš Golembiovský from comment #12)
> > (In reply to Arik from comment #10)
> > > While we can argue on whether that private IP should be displayed in RHV or
> > > not in this case (and if not, which component should filter it out), I
> > > wonder how interesting that is.
> > 
> > That depends. The fact that we report IPs that are unreachable makes it
> > difficult for other tools to connect to the guest. If the NIC is unpluged in
> > oVirt configuration then oVirt knows that it should not report the IP.
> 
> Ok, it seems it's not like that. If I unplug the NIC in oVirt it is disabled
> by NetworkManager and that's why it is not reported (even by qemu-ga). But
> when I bring it up manually it shows in oVirt UI even though the link is
> still down. In oVirt API we seem to report some cached IP address even if
> the address is no longer reported by guest.

You brought up the NIC with the same MAC address, right?
Take into account that it takes some time for the change to propagate to the engine -
once you unplug the NIC, it takes time until the information gets from gqa to vdsm and until engine polls vdsm for the information
It may even take more than a minute - and only then the data is removed.
So if you brought up the NIC before the data was removed and the NIC was set with the same MAC address then it will be correlated with the "old" data.

In order to check if the qga data exists you can query the database:
engine=# select * from vm_guest_agent_interfaces where vm_id in (select vm_guid from vm_static where vm_name='vm-name');
                vm_id                 | interface_name |    mac_address    | ipv4_addresses |                       ipv6_addresses                       
--------------------------------------+----------------+-------------------+----------------+------------------------------------------------------------
 dcbaa892-af6c-46bf-b69a-ab88f3c92059 | eth0           | 00:1a:4a:16:10:39 | 10.35.30.57    | 2620:52:0:231e:21a:4aff:fe16:1039,fe80::21a:4aff:fe16:1039

(In reply to Tomáš Golembiovský from comment #12)
> (In reply to Yan Vugenfirer from comment #11)
> > In my perspective the questions is why link is disconnected ("Media
> > disconnected")? Does management layer set's the link to down state?
> > On other hand the value of IP is actually "valid" (according to MS
> > documentation):
> > https://docs.microsoft.com/en-us/windows-server/troubleshoot/how-to-use-
> > automatic-tcpip-addressing-without-a-dh
> 
> My initial assumption was that the interface was disabled by Windows because
> it was miss-configured, but if that's not the case then this is a good
> question.

+1
Zi, is it solved by changing the NIC's settings to "Obtain an IP address automatically" in that dialog that is shown at https://bugzilla.redhat.com/attachment.cgi?id=1703235?

Hi,Arik

(In reply to Arik from comment #10)
> While we can argue on whether that private IP should be displayed in RHV or
> not in this case (and if not, which component should filter it out), I
> wonder how interesting that is.

RHV should show the guest's ip when the guest running qemu-ga server normally.in this bug,the guest has installed qemu-ga.
So I think the real issue is that:why the guest doesn't have ip, rhv still can show the ip info?


(In reply to Arik from comment #15)
> Zi, is it solved by changing the NIC's settings to "Obtain an IP address
> automatically" in that dialog that is shown at
> https://bugzilla.redhat.com/attachment.cgi?id=1703235?

The original guest has static ip,but after conversion,the guest's NIC changed to "Obtain an IP address automatically".This state is not set by manual.
Additional,if virt-v2v change ip successfully,the new guest's will still have static ip.

you can refer to https://bugzilla.redhat.com/show_bug.cgi?id=1867575#c5 to check the guest.

(In reply to liuzi from comment #16)
> Hi,Arik
> 
> (In reply to Arik from comment #10)
> > While we can argue on whether that private IP should be displayed in RHV or
> > not in this case (and if not, which component should filter it out), I
> > wonder how interesting that is.
> 
> RHV should show the guest's ip when the guest running qemu-ga server
> normally.in this bug,the guest has installed qemu-ga.
> So I think the real issue is that:why the guest doesn't have ip, rhv still
> can show the ip info?

Right so we already have the answer for this part -
the guest actually *has* an IP - the NIC is assigned with a private IP, the one that is presented in RHV, even though it appears as disconnected in the output of 'ipconfig' within the guest.

Dominik, this is easily reproducible:
1. Add a NIC with empty profile to a VM installed with Windows and qemu-guest-agent
2. Start the VM
=> The NIC appears as disconnected from within the guest (which makes sense, it's not set with any network) and qemu-guest-agent reports the private IP that Windows assigns to this NIC and we display it in RHV.

This is a low-priority issue (it shouldn't normally happen), yet -
would it make sense to always filter the reported IPs from qemu-guest-agent in this case (of a NIC with empty profile)?
Is it something that was discussed/considered before?
I think the MAC address should not be filtered

 
> 
> (In reply to Arik from comment #15)
> > Zi, is it solved by changing the NIC's settings to "Obtain an IP address
> > automatically" in that dialog that is shown at
> > https://bugzilla.redhat.com/attachment.cgi?id=1703235?
> 
> The original guest has static ip,but after conversion,the guest's NIC
> changed to "Obtain an IP address automatically".This state is not set by
> manual.
> Additional,if virt-v2v change ip successfully,the new guest's will still
> have static ip.
> 
> you can refer to https://bugzilla.redhat.com/show_bug.cgi?id=1867575#c5 to
> check the guest.

Thanks, having access to the VM helped me understand what happened there.
Zi, do we have a bug on virt-v2v to see why the requested static IP wasn't set? I believe that's what leads to the NIC being set with the empty profile

(In reply to Arik from comment #17)

> Thanks, having access to the VM helped me understand what happened there.
> Zi, do we have a bug on virt-v2v to see why the requested static IP wasn't
> set? I believe that's what leads to the NIC being set with the empty profile

Hi,Arik
It's not a bug.Because the virt-v2v command is incomplete in this bug.After add the parameter "-b ",virt-v2v can set the static ip successfully.

(In reply to Arik from comment #17)
> (In reply to liuzi from comment #16)
> > Hi,Arik
> > 
> > (In reply to Arik from comment #10)
> > > While we can argue on whether that private IP should be displayed in RHV or
> > > not in this case (and if not, which component should filter it out), I
> > > wonder how interesting that is.
> > 
> > RHV should show the guest's ip when the guest running qemu-ga server
> > normally.in this bug,the guest has installed qemu-ga.
> > So I think the real issue is that:why the guest doesn't have ip, rhv still
> > can show the ip info?
> 
> Right so we already have the answer for this part -
> the guest actually *has* an IP - the NIC is assigned with a private IP, the
> one that is presented in RHV, even though it appears as disconnected in the
> output of 'ipconfig' within the guest.
> 
> Dominik, this is easily reproducible:
> 1. Add a NIC with empty profile to a VM installed with Windows and
> qemu-guest-agent
> 2. Start the VM
> => The NIC appears as disconnected from within the guest (which makes sense,
> it's not set with any network) and qemu-guest-agent reports the private IP
> that Windows assigns to this NIC and we display it in RHV.
> 


This behavior sounds good to me. 

> This is a low-priority issue (it shouldn't normally happen), yet -
> would it make sense to always filter the reported IPs from qemu-guest-agent
> in this case (of a NIC with empty profile)?

Why we should?
The guest OS configured this IP address for the interface, and oVirt/RHV reports it correctly.

> Is it something that was discussed/considered before?

I am not aware of this.

Zi, what is the behavior you would expect here?

In comment 0 you wrote:
> RHV cannot show ip info if guest's ip doestn't exist.

But it turns out that this IP address is configured, so RHV pointed you in the correct direction.
Do you have an idea how RHV could show you in a better way, what happens inside the guest?

(In reply to Dominik Holler from comment #19)
> (In reply to Arik from comment #17)
> > This is a low-priority issue (it shouldn't normally happen), yet -
> > would it make sense to always filter the reported IPs from qemu-guest-agent
> > in this case (of a NIC with empty profile)?
> 
> Why we should?
> The guest OS configured this IP address for the interface, and oVirt/RHV
> reports it correctly.

That's right but I think there's a matter of usability here.
We can:
1. Project the configuration as it appears from within the guest; or
2. Filter out what is unlikely to be useful for the user

The qemu-guest-agent does its part - it reports what it sees within the guest.
RHV already filters out some of what qemu-guest-agent reports - by default the loopback interface is filtered out and optionally other interfaces that the user configured RHV to filter out (using 'GuestNicNamesBlacklist' in vdc_options).

But I'm not sure if that should be applied also for the IP of NICs without a network profile.
On the one hand, such a NIC is defined in RHV without a network and as it appears in this case, I guess the assigned IP would typically not be accessible - so what's the point in showing it?
On the other hand, maybe with different configuration within the guest that NIC would be assigned with an IP that is accessible and usable from outside and then it makes sense to present it.

Maybe what I'm missing here is what's the reason for setting a NIC without network profile on the first place?

(In reply to Arik from comment #20)
> (In reply to Dominik Holler from comment #19)
> > (In reply to Arik from comment #17)
> > > This is a low-priority issue (it shouldn't normally happen), yet -
> > > would it make sense to always filter the reported IPs from qemu-guest-agent
> > > in this case (of a NIC with empty profile)?
> > 
> > Why we should?
> > The guest OS configured this IP address for the interface, and oVirt/RHV
> > reports it correctly.
> 
> That's right but I think there's a matter of usability here.
> We can:
> 1. Project the configuration as it appears from within the guest; or
> 2. Filter out what is unlikely to be useful for the user
> 
> The qemu-guest-agent does its part - it reports what it sees within the
> guest.
> RHV already filters out some of what qemu-guest-agent reports - by default
> the loopback interface is filtered out and optionally other interfaces that
> the user configured RHV to filter out (using 'GuestNicNamesBlacklist' in
> vdc_options).
> 
> But I'm not sure if that should be applied also for the IP of NICs without a
> network profile.
> On the one hand, such a NIC is defined in RHV without a network and as it
> appears in this case, I guess the assigned IP would typically not be
> accessible - so what's the point in showing it?

It might help to detect such a situation as described in comment 0 ;-)

> On the other hand, maybe with different configuration within the guest that
> NIC would be assigned with an IP that is accessible and usable from outside
> and then it makes sense to present it.
> 

Yes, if the guest would enable routing, but I would ignore such cases.

> Maybe what I'm missing here is what's the reason for setting a NIC without
> network profile on the first place?

You could attach the network after boot, without hotplugging the vNIC.
One reason might be to keep the order or the PCI address of vNICs in the guest.

(In reply to Dominik Holler from comment #19)
 
> I am not aware of this.
> 
> Zi, what is the behavior you would expect here?
> 
> In comment 0 you wrote:
> > RHV cannot show ip info if guest's ip doestn't exist.
> 
> But it turns out that this IP address is configured, so RHV pointed you in
> the correct direction.
> Do you have an idea how RHV could show you in a better way, what happens
> inside the guest?

Hi,Arik
From my part,I think RHV shouldn't show ip info if a guest's NIC with empty profile.
First, it is useless information for user. And second, it will make someone who don't know much about Windows and qemu-ga confused.

*** Bug 1867575 has been marked as a duplicate of this bug. ***

OK, so let's filter out the reported IPs for NICs that are not set with a network profile in oVirt then

Low priority issue that is not so interesting from the RHV point of view.
Feel free to reopen if this doesn't work for you.

This bugzilla is included in oVirt 4.5.0 release, published on April 20th 2022.

Since the problem described in this bug report should be resolved in oVirt 4.5.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.