Bug 1212674
| Summary: | Failed to unregister system as required lock is already taken by "Listen on candlepin events" task | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Liushihui <shihliu> | ||||||||
| Component: | Subscription Management | Assignee: | satellite6-bugs <satellite6-bugs> | ||||||||
| Status: | CLOSED NEXTRELEASE | QA Contact: | Katello QA List <katello-qa-list> | ||||||||
| Severity: | high | Docs Contact: | |||||||||
| Priority: | unspecified | ||||||||||
| Version: | 6.1.0 | CC: | bbuckingham, bcourt, bkearney, cwelton, ldai, mmccune, sgao, shihliu, sthirugn, tomckay | ||||||||
| Target Milestone: | Unspecified | Keywords: | Reopened, Triaged | ||||||||
| Target Release: | Unused | ||||||||||
| Hardware: | x86_64 | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2017-08-22 17:54:23 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
Since this issue was entered in Red Hat Bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release. Created attachment 1024432 [details]
Destroy Content Host
It still exist on candlepin-0.9.49.0-1.el7.noarch. failed to unregister system since it conflict with "update" task. please see the detail as the following: Checked version: candlepin-0.9.49.0-1.el7.noarch katello-2.2.0.10-1.el7sat.noarch foreman-1.7.2.25-1.el7sat.noarch subscription-manager-1.14.9-1.el6.x86_64 virt-who-0.12-9.el6.noarch 1. Unregister system [root@hp-z220-06 ~]# subscription-manager unregister Required lock is already taken by other running tasks. Please inspect their state, fix their errors and resume them. Required lock: read Conflicts with tasks: - https://hp-dl360g6-01.rhts.eng.bos.redhat.com/foreman_tasks/tasks/f095e167-d4e3-49f1-9e19-efdf8b707a5f 2. Check the register status, system still registered. [root@hp-z220-06 ~]# subscription-manager identity system identity: df66dd3d-f506-4ac7-92b0-37c15e96fd89 name: hp-z220-06.qe.lab.eng.nay.redhat.com org name: Default Organization org ID: Default_Organization environment name: Library 3. In the Satellite webUI, Open the conflict task, it will show the unregister task conflict with "update" task Id: f095e167-d4e3-49f1-9e19-efdf8b707a5f Label: Actions::Katello::System::Update Name: Update Owner: Started at: 2015-06-05 03:08:22 UTC Ended at: Status: paused Result: error Parameter: system 'hp-z220-06.qe.lab.eng.nay.redhat.com'; organization 'Default Organization' Can we get access to your environment where this is happening? Hi Mike, You can use RHEL6.7-20150617.0-Server-x86_64(IP: 10.66.144.5 : root/redhat) and Satellite-6.1.0-RHEL-7-20150707.4(IP:10.16.65.177 : root/red2015). # subscription-manager register --username=admin --password=admin When unreigster rhel system to Satellite, it still show conflict error as the following: # subscription-manager unregister Required lock is already taken by other running tasks. Please inspect their state, fix their errors and resume them. Required lock: read Conflicts with tasks: - https://tyan-gt24-01.rhts.eng.bos.redhat.com/foreman_tasks/tasks/65ca5204-7162-466e-ad12-bce296923204 Thanks. (In reply to Mike McCune from comment #7) > Can we get access to your environment where this is happening? I logged into this Satellite and saw the stuck task and tried the following: 1) Resume the task, it completed: https://tyan-gt24-01.rhts.eng.bos.redhat.com/foreman_tasks/tasks/65ca5204-7162-466e-ad12-bce296923204 2) unregister completed 3) re-register completed fine 4) restart virt-who 5) unregister completed fine 6) re-register completed fine I'm wondering if we can reproduce this again now as I can't seem to get it to error out. Can you indicate how this started as I've not heard any other reports of this situation. Mike, After researching it, the lock tasks which I mentioned on comment 8 was caused by migration guest, please see detail info on bug 1220620. About this bug, If it hasn't migration guest, unregister system won't generate locked task which mentioned on the bug description and comment 4 on the latest Satellite 6.1.0-20150716.0. Therefore, I think you can close this bug. Thanks. (In reply to Mike McCune from comment #9) > I logged into this Satellite and saw the stuck task and tried the following: > > 1) Resume the task, it completed: > > https://tyan-gt24-01.rhts.eng.bos.redhat.com/foreman_tasks/tasks/65ca5204- > 7162-466e-ad12-bce296923204 > > 2) unregister completed > > 3) re-register completed fine > > 4) restart virt-who > > 5) unregister completed fine > > 6) re-register completed fine > > > I'm wondering if we can reproduce this again now as I can't seem to get it > to error out. > > Can you indicate how this started as I've not heard any other reports of > this situation. Closing this bug per comment 10. It has been reproduced on Satellite-6.1.0-RHEL-7-20151210.0. Therefore, reopen it. Reproduced version: Satellite-6.1.0-RHEL-7-20151210.0 katello-2.2.0.16-1.el7sat.noarch foreman-1.7.2.49-1.el7sat.noarch candlepin-0.9.49.9-1.el7.noarch Reproduced process: Do as the bug's description, when unregister rhel system, it was failed and show conflict task error. #subscription-manager unregister Required lock is already taken by other running tasks. Please inspect their state, fix their errors and resume them. Required lock: read Conflicts with tasks: - https://satellite-1c5f84e0-5a8a-4d14-8039-4cf8be754093.novalocal/foreman_tasks/tasks/b343bf9a-9a35-4de7-be2f-a61c3146ca18 It still exist on Satellite-6.1.0-RHEL-7-20160107.0. when unregister system(RHEL6.8-20160111.0-x86_64), it also has the same problem. #subscription-manager unregister Required lock is already taken by other running tasks. Please inspect their state, fix their errors and resume them. Required lock: read Conflicts with tasks: - https://satellite-4df6a60a-42b5-4f29-9e4c-459656386bcf.novalocal/foreman_tasks/tasks/8356ed93-e999-401a-847c-52a26ae8cb89 Register rhel to Satellite-6.2.0-RHEL-7-20160907.6 has the similar problem. please see the webUI screenshot in attachment. # subscription-manager register --username=admin --password=admin Registering to: ibm-x3655-04.ovirt.rhts.eng.bos.redhat.com:443/rhsm Required lock is already taken by other running tasks. Please inspect their state, fix their errors and resume them. Required lock: read Conflicts with tasks: - https://ibm-x3655-04.ovirt.rhts.eng.bos.redhat.com/foreman_tasks/tasks/521989ac-6d62-4af0-8751-62a70c7c8021 Created attachment 1200727 [details]
register_conflict
We believe the bugs around this have been fixed around https://bugzilla.redhat.com/show_bug.cgi?id=1338578 https://bugzilla.redhat.com/show_bug.cgi?id=1209866 https://bugzilla.redhat.com/show_bug.cgi?id=1244309 https://bugzilla.redhat.com/show_bug.cgi?id=1276241 https://bugzilla.redhat.com/show_bug.cgi?id=1325879 I am closing this out as NEXTRELEASE. If you see additional cases with 6.3, please feel free to re-open this. |
Created attachment 1015442 [details] conflict task Description of problem: Unregister system, it will show "Required lock is already taken by other running tasks" in the log file, this task has conflicted with "Listen on candlepin events" task. Version-Release number of selected component (if applicable): subscription-manager-1.14.5-1.el6.x86_64 python-rhsm-1.14.2-1.el6.x86_64 virt-who-0.12-5.el6.noarch candlepin-0.9.48-1.el7.noarch foreman-1.7.2.15-1.el7sat.noarch Satellite-6.1.0-RHEL-7-20150409.0 How reproducible: 50% Steps to Reproduce: 1. Register system to satellite [root@rhel6 ~]# subscription-manager identity system identity: 1e3261fc-2f53-47a1-9fda-abf9cb31ba87 name: rhel6.7-20150415.redhat.com org name: Default Organization org ID: Default_Organization environment name: Library 2. Restart virt-who service, then unregister system [root@rhel6 ~]# subscription-manager unregister Required lock is already taken by other running tasks. Please inspect their state, fix their errors and resume them. Required lock: write Conflicts with tasks: 3. Check the virt-who's log 4. On the satellite webUI, Go to Monitor-->Task, filter out "result=pending" Actual results: Failed to unregister system unless force unlock the conflict task. it will show error in the log. 2015-04-16 21:29:35,259 [DEBUG] @subscriptionmanager.py:76 - Authenticating with certificate: /etc/pki/consumer/cert.pem 2015-04-16 21:29:36,637 [DEBUG] subscription-manager:3556 @connection.py:521 - Response: status=500 2015-04-16 21:29:36,637 [WARNING] subscription-manager:3556 @connection.py:525 - Clock skew detected, please check your system time 2015-04-16 21:29:36,637 [ERROR] subscription-manager:3556 @managercli.py:160 - Unregister failed 2015-04-16 21:29:36,638 [ERROR] subscription-manager:3556 @managercli.py:161 - Required lock is already taken by other running tasks. Please inspect their state, fix their errors and resume them. Required lock: write Conflicts with tasks: Traceback (most recent call last): File "/usr/share/rhsm/subscription_manager/managercli.py", line 1232, in _do_command managerlib.unregister(self.cp, self.identity.uuid) File "/usr/share/rhsm/subscription_manager/managerlib.py", line 788, in unregister uep.unregisterConsumer(consumer_uuid) File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 990, in unregisterConsumer return self.conn.request_delete(method) File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 614, in request_delete return self._request("DELETE", method, params) File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 530, in _request self.validateResponse(result, request_type, handler) File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 572, in validateResponse raise RestlibException(response['status'], error_msg) RestlibException: Required lock is already taken by other running tasks. Please inspect their state, fix their errors and resume them. Required lock: write Conflicts with tasks: ================================================================ On the satellite webUI, it will show a conflict task as the following or see the screenshot in the attachment. Id: 1a3d1a63-4552-4a56-9916-a91c02e65dbf Label: Actions::Candlepin::ListenOnCandlepinEvents Name: Listen on candlepin events Owner: Started at: 2015-04-10 11:47:38 UTC Ended at: State: running Result: - Params: {"locale"=>"en"} 50.0% Complete 50% Output: {"messages"=>"5b02c6b5-4073-3d25-8c80-b36134106f55", "last_message"=>"5b02c6b5-4073-3d25-8c80-b36134106f55 - consumer.modified", "connection"=>"Connected"} Expected results: Success to unregister system. Additional info: Sometimes do some actions such as attach or unattach subscriptions also has this problem.