1293821 – virt-who processes(PPID != 1) cannot be killed immediately after unregister system

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1293821 - virt-who processes(PPID != 1) cannot be killed immediately after unregister system

Summary: virt-who processes(PPID != 1) cannot be killed immediately after unregister s...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	virt-who
Sub Component:
Version:	6.8
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Radek Novacek
QA Contact:	gaoshang
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2015-12-23 07:32 UTC by Liushihui
Modified:	2016-12-01 00:35 UTC (History)
CC List:	6 users (show)
Fixed In Version:	virt-who-0.16-4.el6
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-05-10 23:57:02 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHEA-2016:0859	0	normal	SHIPPED_LIVE	virt-who bug fix and enhancement update	2016-05-10 22:44:47 UTC

Description Liushihui 2015-12-23 07:32:52 UTC

Description of problem:
After unregister system, virt-who processes which PPID is not 1 can not be killed until 30s. it also won't show system has been unregistered in the rhsm.log. 

Version-Release number of selected component (if applicable):
virt-who-0.16-1.el6.noarch
subscription-manager-1.16.6-1.el6.x86_64
python-rhsm-1.16.4-1.el6.x86_64

How reproducible:
80%

Steps to Reproduce:
1. Register system to Satellite6. make virt-who run at kvm mode, refresh interval is default.
[root@hp-z220-05 tmp]# subscription-manager register --username=admin --password=admin
Registering to: cisco-b200m1-04.rhts.eng.bos.redhat.com:443/rhsm
The system has been registered with ID: b9f6ca8a-3ae8-4b66-b293-847e39481ce0 
2. Start virt-who service 
[root@hp-z220-05 tmp]# service virt-who start
[root@hp-z220-05 tmp]# ps -ef|grep virt-who
root     10795     1  0 14:00 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root     11055 10795  0 14:05 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root     11060  9603  0 14:05 pts/0    00:00:00 grep virt-who
3. Unregister system. then check virt-who processes.
[root@hp-z220-05 tmp]# subscription-manager unregister
System has been unregistered.
[root@hp-z220-05 tmp]# ps -ef|grep virt-who
root     10795     1  0 14:00 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root     11055 10795  0 14:05 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root     11060  9603  0 14:05 pts/0    00:00:00 grep virt-who
4. Check virt-who's log in /var/log/rhsm/rhsm.log
# tail -f /var/log/rhsm/rhsm.log
5. After about 30s, check virt-who processes:
[root@hp-z220-05 tmp]# ps -ef|grep virt-who
root     10795     1  0 14:00 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root     11062  9603  0 14:06 pts/0    00:00:00 grep virt-who

Actual results:
It still show two virt-who processes after unregister system.
It will not show system is unregistered immediately.
2015-12-23 14:05:28,759 [virtwho.env_cmdline DEBUG] Libvirtd-6(11055):MainThread @libvirtd.py:_listDomains:238 - Libvirt domains found: 3c6e36b9-a88e-dacf-3d8d-782fcd8cbb39
2015-12-23 14:05:28,759 [virtwho.env_cmdline DEBUG] Libvirtd-6(11055):MainThread @virt.py:enqueue:331 - Report gathered, putting to queue for sending
2015-12-23 14:05:28,762 [virtwho.main DEBUG] MainProcess(10795):MainThread @virtwho.py:update_report_to_send:274 - Report for config "env/cmdline" updated
=============================================================================
==============After 30s, it will show "system is not registered "===========
=============================================================================                                                   
2015-12-23 14:05:59,110 [virtwho.main DEBUG] MainProcess(10795):MainThread @subscriptionmanager.py:_connect:112 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2015-12-23 14:05:59,111 [virtwho.main ERROR] MainProcess(10795):MainThread @virtwho.py:send:232 - Error in communication with subscription manager:
Traceback (most recent call last):
  File "/usr/share/virt-who/virtwho.py", line 222, in send
    self._sendGuestList(report)
  File "/usr/share/virt-who/virtwho.py", line 249, in _sendGuestList
    manager.sendVirtGuests(report.guests)
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 130, in sendVirtGuests
    self._connect()
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 115, in _connect
    "Unable to read certificate, system is not registered or you are not root")
SubscriptionManagerUnregisteredError: Unable to read certificate, system is not registered or you are not root

Expected results:
It should keep only one virt-who(PPID=1) process after unregister system. system has been unregistered should be shown on rhsm log immediately.

Additional info:

Comment 1 Radek Novacek 2016-01-28 13:04:23 UTC

Fixed in virt-who-0.16-2.el6.

Comment 3 Liushihui 2016-02-04 07:45:51 UTC

Reopen it on virt-who-0.16-3.el6.noarch since virt-who dead after unregister system. please see the detail as the following:

Checked version:
virt-who-0.16-3.el6.noarch
subscription-manager-1.16.8-1.el6.x86_64
python-rhsm-1.16.6-1.el6.x86_64

Checked processes:
1.Register system to satellite and run virt-who at libvirt mode.there are 2 virt-who threads.
[root@intel-canoepass-10 ~]# service virt-who status
virt-who (pid  13779) is running...
[root@intel-canoepass-10 ~]# ps -ef|grep virt-who 
root      13779      1  1 02:37 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root      13786  13779  0 02:37 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root      13808  10536  0 02:37 pts/1    00:00:00 grep virt-who
2. Unregister system,then check virt-who thread
[root@intel-canoepass-10 ~]# subscription-manager unregister
System has been unregistered.
[root@intel-canoepass-10 ~]# ps -ef|grep virt-who 
root      13818  10536  0 02:38 pts/1    00:00:00 grep virt-who
[root@intel-canoepass-10 ~]# service virt-who status
virt-who dead but subsys locked

Result:
virt-who dead after unregister system.

Expected result:
It should keep only one virt-who(PPID=1) process after unregister system.

Comment 4 Radek Novacek 2016-02-11 13:07:56 UTC

Fixed in virt-who-0.16-4.el6.

Comment 6 Liushihui 2016-02-25 07:47:14 UTC

Verified it on virt-who-0.16-4.el6.noarch since virt-who can work normally and system has been unregistered has shown on rhsm log immediately after unregister system. Therefore, verify it.

Verified version:
virt-who-0.16-4.el6.noarch
subscription-manager-1.16.8-2.el6.x86_64
python-rhsm-1.16.6-1.el6.x86_64

Verified process:
1.Register system to Satellite and run virt-who at libvirt mode.there are 2 virt-who threads.
[root@sgi-xe500-01 ~]# service virt-who restart
Stopping virt-who: [  OK  ]
Starting virt-who: [  OK  ]
[root@sgi-xe500-01 ~]# ps -ef|grep virt-who
root      1379     1  1 02:33 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root      1387  1379  0 02:33 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root      1392  8716  0 02:33 pts/0    00:00:00 grep virt-who

2. Unregister system,then check virt-who thread
[root@sgi-xe500-01 ~]# subscription-manager  unregister
System has been unregistered.
[root@sgi-xe500-01 ~]# ps -ef|grep virt-who
root      1379     1  0 02:33 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root      1397  1379  0 02:33 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root      1402  8716  0 02:34 pts/0    00:00:00 grep virt-who

3. Check virt-who's log after unregister system.
2016-02-25 02:35:43,162 [virtwho.env_cmdline DEBUG] HyperV-4(1494):MainThread @hyperv.py:getHostGuestMapping:521 - Unable to enumerate using root/virtualization namespace, trying root/virtualization/v2 namespace
2016-02-25 02:35:46,749 [virtwho.test-esx1 DEBUG] Esx-3(1492):MainThread @esx.py:_prepare:58 - Creating ESX event filter
2016-02-25 02:35:54,734 [virtwho.env_cmdline DEBUG] HyperV-4(1494):MainThread @virt.py:enqueue:351 - Report gathered, putting to queue for sending
2016-02-25 02:35:54,741 [virtwho.main DEBUG] MainProcess(1469):MainThread @subscriptionmanager.py:_connect:121 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2016-02-25 02:35:54,745 [virtwho.main ERROR] MainProcess(1469):MainThread @virtwho.py:_main:751 - Fatal error:
Traceback (most recent call last):
  File "/usr/share/virt-who/virtwho.py", line 748, in _main
    result = virtWho.run()
  File "/usr/share/virt-who/virtwho.py", line 333, in run
    self.send_current_report()
  File "/usr/share/virt-who/virtwho.py", line 153, in send_current_report
    if self.send(report):
  File "/usr/share/virt-who/virtwho.py", line 197, in send
    self._sendGuestAssociation(report)
  File "/usr/share/virt-who/virtwho.py", line 224, in _sendGuestAssociation
    manager.hypervisorCheckIn(report, self.options)
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 161, in hypervisorCheckIn
    self._connect(report.config)
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 124, in _connect
    "Unable to read certificate, system is not registered or you are not root")
SubscriptionManagerUnregisteredError: Unable to read certificate, system is not registered or you are not root
2016-02-25 02:35:54,747 [virtwho.main INFO] MainProcess(1469):MainThread @virtwho.py:_main:753 - Waiting for reload signal

4. Re-register system to satellite and check vir-who status and threads
[root@sgi-xe500-01 ~]# subscription-manager  register --username=admin --password=admin
Registering to: bigisis.rhts.eng.bos.redhat.com:443/rhsm
The system has been registered with ID: f901e50e-f52c-4595-a77c-3f364c2739a3 
[root@sgi-xe500-01 ~]# ps -ef|grep virt-who
root      1379     1  0 02:33 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root      1432  1379  0 02:34 ?        00:00:00 /usr/bin/python /usr/share/virt-who/virtwho.py
root      1442  8716  0 02:34 pts/0    00:00:00 grep virt-who
[root@sgi-xe500-01 ~]# service virt-who status
virt-who (pid  1469) is running...

Result:
After unregister system, virt-who run normally and "system has been unregistered" has shown on rhsm log immediately.Re-register system to server, virt-who still work normally.

Comment 8 errata-xmlrpc 2016-05-10 23:57:02 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0859.html

Note You need to log in before you can comment on or make changes to this bug.