Description of problem: [vmware]Getting "[WARNING] @client.py:813 - Web service reported a SOAP processing fault using an unexpected HTTP status code 200. Reporting as an internal server error" after fresh configuration of virt-who on RHEL9 vm. Version-Release number of selected component (if applicable): virt-who-1.31.22-1.el9_0.noarch How reproducible: We are trying to reproduce these warnings in our test environment and will share our observations soon. Steps to Reproduce: 1. configure virt-who service on RHEL9 vm to report host-guest mapping to customer portal. # vi /etc/virt-who.d/virt-who.conf [vmware] type=esx server=vCenter(FQDN/IP) username=vCenter_Username encrypted_password=******** 2.Change interval=1800 and debug=false in /etc/virt-who.conf 3. Add below parameters in /etc/virt-who.conf owner=org ID (From the Command # subscription-manager identity) hypervisor_id=hostname 4.Start virt-who service using command # systemctl start virt-who Actual results: After starting virt-who service,at first it was able to report number of host-guests as below 2022 Jul 5 15:58:05 varda [local0.info] /usr/bin/virt-who: 2022-07-05 15:58:05,609 [virtwho.destination_-3072355972159141989 INFO] MainProcess(153438):Thread-3 @virt.py:_send_data:836 - Sending updated Host-to-guest mapping to "xxxxxx" including 6 hypervisors and 79 guests Then,our customer got below warnings multiple times in /var/log/rhsm/rhsm.log 2022-07-07 12:07:03,838 [WARNING] @client.py:813 - Web service reported a SOAP processing fault using an unexpected HTTP status code 200. Reporting as an internal server error. All esxi hypervisors were reported to Customer portal, but for some reason the subscriptions and details weren't in the customer portal. Expected results: These warnings should not be present. There should not be much delay while subscribing esxi hosts reported by virt-who on Red Hat customer portal. Additional info: We are trying to reproduce these warnings in our test environment and will share our observations soon.
We need logs from when the error occurs, please.
This "error" is handled in the code in esx.py. It is captured and ignored by design. These warnings do not have any effect on virt-who's reporting as far as I know. It runs on its cycle and collects the data. If the data is different than the last time, it will send a report. There is no reason that anything would take a day to report. Is this a new hypervisor with new guests? Is the service getting restarted daily?
If the hypervisors are reported correctly with their mappings, then virt-who is done. It has no more input in the process as it is not responsible for the attaching of subscriptions. This will need to be brought to the team that handles the Customer Portal [teamnado]
Both virt-who [mapping] and subscription-manager [rhsmcertd] appear to be properly handling this data. For reasons unknown, when auto-attach [heal] is called on the guest machines, the host subscription attachment which allows the guest subscription attachment is not happening. It seems to take a long time or does not happen at all It seems that the rules engine is not making a match with the expected subscription. Perhaps it is a mismatch in data [the example above shows the hypervisor has 2 cores. Is the SKU 1 core and multi-entitlement?]. Perhaps it is a quantity of pool issue. That is the best place to start. Comment 16 says that if the subscription for the hypervisor is attached manually, then the guest system will auto-attach correctly.
Can you tell me how long "a long time is"? What is the frequency set to for rhsmcertd on the virtual machines? autoattachinterval = ? certcheckinterval = ? I'd love to see a log from a vm and/or hosted when there is an auto-attach that does not result in the state you expect.
If no more information is provided, this issue will be closed.
Backlog Clean up : ------------------ Unfortunately our engineering team is unable to proceed with the investigation without the additional details requested. Hence closing the bug. Please open a new bug if this is still an issue. Thanks, Rehana