Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 663326

Summary: Channel deletion doesn't check for spacewalk-repo-sync process running in background leading to tracebacks
Product: Red Hat Satellite 5 Reporter: Paresh Mutha <pmutha>
Component: ServerAssignee: Michael Mráka <mmraka>
Status: CLOSED ERRATA QA Contact: Jan Hutař <jhutar>
Severity: medium Docs Contact:
Priority: medium    
Version: 540CC: clasohm, cperry, dyordano, jhutar, mmraka, mzazrivec, tlestach, xdmoon
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: spacewalk-backend-1.2.13-39 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-06-17 02:44:16 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 677501    

Description Paresh Mutha 2010-12-15 12:26:46 UTC
Description of problem:
On using spacewalk-remove-channel to delete a channel, it doesn't remove the taskomatic scheduled sync tasks (RepoSyncTask). This leads to traceback emails like :

----
Taskomatic bunch repo-sync-bunch was scheduled to run within the repo-sync-1-1251 schedule.

Subtask repo-sync failed.

For more information check /var/lib/rhn/tasko/org1/repo-sync-bunch/repo-sync_61024_err.
----

In which the error log listed has:

----
INFO | jvm 1 | 2010/11/23 03:15:39 | 2010-11-23 03:15:39,124 [DefaultQuartzScheduler_Worker-9] ERROR com.redhat.rhn.taskomatic.task.RepoSyncTask - No such channel with channel_id 891
INFO | jvm 1 | 2010/11/23 03:15:39 | 2010-11-23 03:15:39,124 [DefaultQuartzScheduler_Worker-9] ERROR com.redhat.rhn.taskomatic.task.RepoSyncTask - 


Version-Release number of selected component (if applicable):
RHN Satellite v5.4

How reproducible:
Always

Steps to Reproduce:
1] Create an external repository (Channels > Manage Repositories)
2] Create a channel (Channels > Manage Software Channels)
3] Link repository to channel (Channels > Manage Software Channels > (Select Channel) > Repositories tab)
4] Set a scheduled sync on repository (Channels > Manage Software Channels > (Select Channel) > Repositories tab > Sync subtab)
5] Delete the channel (Channels > Manage Software Channels > (Select Channel)) 
6] ps -elf |grep spacewalk-repo-sync
  
Actual results:
/usr/bin/spacewalk-repo-sync still running in background

Expected results:
Either the spacewalk-remove-channel should not remove the channel if repo sync is in progress or it should kill/remove those tasks after the channel is deleted.

Comment 2 Paresh Mutha 2010-12-15 12:39:42 UTC
Small update in the Expected results section:
Channel deletion (either web-ui or spacewalk-remove-channel) should not remove the channel if repo sync is in progress or it should kill/remove that process/task after the channel is deleted.

Comment 3 Jan Pazdziora (Red Hat) 2010-12-15 13:06:04 UTC
Wouldn't there be similar problem with satellite-sync and rhnpush? Basically a race condition of one process removing entity which is being updated by other process.

Comment 4 Paresh Mutha 2010-12-16 07:23:31 UTC
Yes I agree there could be similar problem with satellite-sync or rhnpush. 

Checking this further :
1]The code of channel deletion through web interface, I couldn't see any checks on satellite-sync or repo-sync pid's. 
2] satellite-sync as well doesn't check for spacewalk-remove-channel or rhnpush.

But if we check the code for spacewalk-remove-channel, it already checks for satellite-sync process. If its already locked then spacewalk-remove-channel errors out with message "ERROR: Either another instance of spacewalk-remove-channel is running, or a satellite-sync is currently taking place."

So on these lines I have proposed the additional check on spacewalk-repo-sync as well. But this doesn't addresses the concern that if the channel is deleted from web-ui when repo sync was running it would still lead to tracebacks.

So here is what I propose few options :
1] Before deletion of channel, we check for lock on spacewalk-repo-sync (since it leads to traceback when channel not present)
2] After channel deletion, if we have the repo sync tasks on the same channel we clean/remove those as well.

Also before the start of satellite-sync, rhnpush, spacewalk-remove channel - should we add conditions to check if any of these are already locked and if so then exit with the error ? This would ensure a consistent lock handling across the channel management tools.

Comment 7 Michael Mráka 2011-02-04 13:19:37 UTC
This has been already fixed in spacewalk:

# spacewalk-remove-channel -v -c centos5-i386
ERROR: Either another instance of spacewalk-remove-channel is running, or a satellite-sync is currently taking place.
# rpm -q spacewalk-backend
spacewalk-backend-1.3.54-1.el6.noarch

It's been fixed by
commit b59374531ee9a977b9e50395196e1f6047ffa92c
    enhancing the spacewalk-remove-channel script
    - adding locking to conflict with itself and satellite-sync
    - improved verbage of unsubscribing process to show the number of systems per channel
    - fixed bug where rhnReleaseChannelMap entries would not be deleted
    - fixed a typo or two

which is included in spacewalk-backend-0.9.7-1 so it should be also included in Satellite 5.4 (spacewalk-backend-1.2.*).

Comment 8 Michael Mráka 2011-04-06 13:44:27 UTC
Oops, the above commit solves simultaneous run of satellite-sync, not spacewalk-repo-sync.

Comment 9 Michael Mráka 2011-04-06 13:47:46 UTC
Fixed in spacewalk nightly by
commit 291ce7c5d447933242bf9386edafd4d32b998d3f
    663326 - return doesn't correctly trigger releaseLOCK()
commit 1b7958ded41b086debbc8d6fe7fb502755fcf99e
    663326 - prevent spacewalk-remove-channel to run when spacewalk-repo-sync is runnig

Fixed spacewalk package: spacewalk-backend-1.4.28-1

Comment 10 Michael Mráka 2011-04-06 13:55:12 UTC
Backported to SATELLITE-5.4 as
commit 0ddd1302ddbafd29b522ef08f7a7fe7c5a050f5b
    663326 - return doesn't correctly trigger releaseLOCK()
    (cherry picked from commit 291ce7c5d447933242bf9386edafd4d32b998d3f)
    
    Conflicts:
    
        backend/satellite_tools/spacewalk-remove-channel
commit 3e6073571d0917bcf8aef02f6e6b64d7580dda1f
    663326 - prevent spacewalk-remove-channel to run when spacewalk-repo-sync is runnig
    (cherry picked from commit 1b7958ded41b086debbc8d6fe7fb502755fcf99e)

Comment 20 Milan Zázrivec 2011-06-06 12:04:49 UTC
Verified in stage w/ spacewalk-backend-1.2.13-52 -> release pending.

Comment 21 Clifford Perry 2011-06-17 02:44:16 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

https://rhn.redhat.com/errata/RHEA-2011-0875.html