Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2105994

Summary:	Subscription assignment takes excessive time after host/guest relationship is mapped
Product:	[Community] Candlepin (Migrated to Jira)	Reporter:	sgajendr
Component:	candlepin	Assignee:	candlepin-bugs
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.2	CC:	hsun, kuhuang, nmoumoul, redakkan, wpoteat
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-04-03 15:27:32 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description sgajendr 2022-07-11 13:23:41 UTC

Description of problem:

[vmware]Getting "[WARNING] @client.py:813 - Web service reported a SOAP processing fault using an unexpected HTTP status code 200. Reporting as an internal server error" after fresh configuration of virt-who on RHEL9 vm.

Version-Release number of selected component (if applicable):

virt-who-1.31.22-1.el9_0.noarch

How reproducible:
We are trying to reproduce these warnings in our test environment and will share our observations soon.

Steps to Reproduce:
1. configure virt-who service on RHEL9 vm to report host-guest mapping to customer portal.

# vi /etc/virt-who.d/virt-who.conf

[vmware]
type=esx
server=vCenter(FQDN/IP)
username=vCenter_Username
encrypted_password=********

2.Change interval=1800 and debug=false in /etc/virt-who.conf
3. Add below parameters in /etc/virt-who.conf
owner=org ID (From the Command # subscription-manager identity)
hypervisor_id=hostname

4.Start virt-who service using command

# systemctl start virt-who

Actual results:

After starting virt-who service,at first it was able to report number of host-guests as below

2022 Jul 5 15:58:05 varda [local0.info] /usr/bin/virt-who: 2022-07-05 15:58:05,609 [virtwho.destination_-3072355972159141989 INFO] MainProcess(153438):Thread-3 @virt.py:_send_data:836 - Sending updated Host-to-guest mapping to "xxxxxx" including 6 hypervisors and 79 guests

Then,our customer got below warnings multiple times in /var/log/rhsm/rhsm.log

2022-07-07 12:07:03,838 [WARNING] @client.py:813 - Web service reported a SOAP processing fault using an unexpected HTTP status code 200. Reporting as an internal server error.

All esxi hypervisors were reported to Customer portal, but for some reason the subscriptions and details weren't in the customer portal.

Expected results:

These warnings should not be present.
There should not be much delay while subscribing esxi hosts reported by virt-who on Red Hat customer portal.
Additional info:
We are trying to reproduce these warnings in our test environment and will share our observations soon.

Comment 5 William Poteat 2022-07-14 14:09:40 UTC

We need logs from when the error occurs, please.

Comment 12 William Poteat 2022-07-21 19:07:37 UTC

This "error" is handled in the code in esx.py. It is captured and ignored by design. 

These warnings do not have any effect on virt-who's reporting as far as I know. It runs on its cycle and collects the data. If the data is different than the last time, it will send a report. There is no reason that anything would take a day to report. Is this a new hypervisor with new guests? Is the service getting restarted daily?

Comment 14 William Poteat 2022-08-26 19:34:42 UTC

If the hypervisors are reported correctly with their mappings, then virt-who is done. It has no more input in the process as it is not responsible for the attaching of subscriptions.

This will need to be brought to the team that handles the Customer Portal [teamnado]

Comment 17 William Poteat 2022-09-01 17:58:54 UTC

Both virt-who [mapping] and subscription-manager [rhsmcertd] appear to be properly handling this data.

For reasons unknown, when auto-attach [heal] is called on the guest machines, the host subscription attachment which allows the guest subscription attachment is not happening. It seems to take a long time or does not happen at all

It seems that the rules engine is not making a match with the expected subscription. Perhaps it is a mismatch in data [the example above shows the hypervisor has 2 cores. Is the SKU 1 core and multi-entitlement?]. Perhaps it is a quantity of pool issue. That is the best place to start. Comment 16 says that if the subscription for the hypervisor is attached manually, then the guest system will auto-attach correctly.

Comment 18 William Poteat 2022-09-21 21:36:13 UTC

Can you tell me how long "a long time is"?
What is the frequency set to for rhsmcertd on the virtual machines?

   autoattachinterval = ?
   certcheckinterval = ?

I'd love to see a log from a vm and/or hosted when there is an auto-attach that does not result in the state you expect.

Comment 19 William Poteat 2023-03-02 13:36:32 UTC

If no more information is provided, this issue will be closed.

Comment 20 Rehana 2023-04-03 15:27:32 UTC

Backlog Clean up : 
------------------
Unfortunately our engineering team is unable to proceed with the investigation without the additional details requested. Hence closing the bug. Please open a new bug if this is still an issue.

Thanks,
Rehana

Comment 21 Red Hat Bugzilla 2023-09-18 04:41:42 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days