Bug 2208224

Summary: [Azure][RHEL-9] systemd runs in high CPU usage if unregister from RHSM
Product: Red Hat Enterprise Linux 9 Reporter: Yuxin Sun <yuxisun>
Component: redhat-cloud-client-configurationAssignee: Alba Hita <ahitacat>
Status: CLOSED ERRATA QA Contact: Craig Donnelly <cdonnell>
Severity: medium Docs Contact:
Priority: high    
Version: 9.3CC: cdonnell, litian, redakkan, xiliang, xuli, xxiong, yacao
Target Milestone: rcKeywords: Triaged
Target Release: ---Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: redhat-cloud-client-configuration-1-11.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-11-07 08:36:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yuxin Sun 2023-05-18 10:15:06 UTC
Description of problem:
redhat-cloud-client-configuration package is installed when VM boot up. Wait for the VM auto-register to RHSM successfully, and insights-client is registered as well. Then 'subscription-manager unregister'. The systemd begins to print the following logs continuously, and has >50% CPU usage.

May 18 05:34:53 localhost systemd[1]: insights-unregistered.service: Deactivated successfully.
May 18 05:34:53 localhost systemd[1]: Started Automatically check if manually Unregistered from Red Hat Insights Client.
May 18 05:34:53 localhost systemd[1]: Reloading.
May 18 05:34:53 localhost systemd-rc-local-generator[16479]: /etc/rc.d/rc.local is not marked executable, skipping.
May 18 05:34:53 localhost systemd[1]: /usr/lib/systemd/system/insights-register.service:23: Unit uses MemoryLimit=; please use MemoryMax= instead. Support for MemoryLimit= will be removed soon.
May 18 05:34:53 localhost systemd[1]: /usr/lib/systemd/system/insights-client-boot.service:24: Unit uses MemoryLimit=; please use MemoryMax= instead. Support for MemoryLimit= will be removed soon.

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                
      1 root      20   0  108248  18196  10952 R  51.2   1.0   1:48.42 systemd

Version-Release number of selected components (if applicable):
redhat-cloud-client-configuration-1-10.el9.noarch
systemd-252-13.el9_2.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare a VM on Azure with redhat-cloud-client-configuration installed.
2. Wait for it to auto-register to RHSM. And ensure the insights-client is registered.
# insights-client --status
System is registered locally via .registered file. Registered at 2023-05-18T02:59:07.128754
Insights API confirms registration.
3. Unregister from RHSM manually
# subscription-manager unregister
4. Check /var/log/messages and CPU usage

Actual results:
As in the description.

Expected results:
Do not print such logs continuously. Do not in such a high CPU usage

Additional info:
It looks similar to BZ#2183947. Please help to check if they are in the same root cause.

When this issue is happened, the insights-client has been unregistered successfully:
# insights-client --status
Unable to fetch egg url https://cert-api.access.redhat.com/r/insights/platform/module-update-router/v1/channel?module=insights-core: 401: Unauthorized. Defaulting to /release
Please ensure that the system is registered with RHSM for CERT auth, or that correct credentials are set in /etc/insights-client/insights-client.conf for BASIC auth.
System is NOT registered locally via .registered file. Unregistered at 2023-05-18T09:19:12.428156
Insights API could not be reached to confirm registration status.

# ll /etc/pki/consumer/cert.pem
ls: cannot access '/etc/pki/consumer/cert.pem': No such file or directory

No such issue in RHEL-8.9(redhat-cloud-client-configuration-1-10.el8.noarch, systemd-239-75.el8.x86_64)

Comment 6 Li Tian 2023-06-30 01:48:22 UTC
This issue is present on RHEL9.3 without any actions right after boot. dmesg and console keep logging:

[  100.535277] systemd-rc-local-generator[7214]: /etc/rc.d/rc.local is not marked executable, skipping.

constantly about twice every second.

systemd-252-15.el9.x86_64
insights-client-3.1.7-13.el9.noarch

Comment 14 errata-xmlrpc 2023-11-07 08:36:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (redhat-cloud-client-configuration bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:6548