Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 549553

Summary:	Channel->Packages->Remove packages is slow (sometimes UI times out and actual removal of ~6000pkgs take over 1 hr)
Product:	Red Hat Satellite 5	Reporter:	Xixi <xdmoon>
Component:	Server	Assignee:	Justin Sherrill <jsherril>
Status:	CLOSED ERRATA	QA Contact:	Garik Khachikyan <gkhachik>
Severity:	high	Docs Contact:
Priority:	urgent
Version:	520	CC:	cperry, gkhachik, mkoci, tao, xdmoon
Target Milestone:	---	Keywords:	Regression
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2010-02-16 08:39:22 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	518256

Description Xixi 2009-12-21 23:15:05 UTC

Description of problem:
Originated from same customer report as Bug 548208.
After a recent upgrade from RHN Satellite 5.0.x to 5.3.0, customer encountered
some performance related problems w/r/t deleting packages from a channel.

A process that would normally take about 10 minutes prior to the upgrade now
takes on the order of 1.5 hours or more!

Version-Release number of selected component (if applicable):
Red Hat Network (RHN) Satellite 5.3.0

How reproducible:
Always.

Steps to Reproduce:
1. have a satellite with base channel A, then child channel B with ~6100 pkgs (shouldn't really matter if it's child or another base channel though).
2. select "Channels" from the top RHN bar
3. select "Manage Software Channels" on menu at left
4. select channel B
5. select "Packages" from list at top
6. select "List / Remove Packages"
7. click on "Select All" at bottom of page
8. confirm on the next page.

Actual results:
"Select all" itself is fast but the actual deletion after confirmation takes 1.5 hrs for customer, and in internal reproducer the page times out with Internal Server Error, and top shows high java cpu usage for over 1 hour, then refreshing the channel UI shows pkg deletion has completed.

Expected results:
Action completes fast and no internal server error.

Additional info:
Suggested workaround to use the API - e.g., under channel.software there're these calls among others:
  * listAllPackages
  * removePackages
so do a listAllPackages and then removePackages.

Comment 2 Xixi 2009-12-21 23:19:05 UTC

Oracle does not show high resource consumption as java does under top, and Oracle statspack report confirms it.  This seems to be a performance issue in the java layer, possibly inefficiency in the following code?

spacewalk-java-0.5.44/code/src/com/redhat/rhn/frontend/action/channel/manage/ChannelPackagesRemoveAction.java
...
    /** {@inheritDoc} */
    public ActionForward execute(ActionMapping mapping,
            ActionForm formIn,
            HttpServletRequest request,
            HttpServletResponse response) {
...
        RhnSet set =  RhnSetDecl.PACKAGES_TO_REMOVE.get(user);
        DataResult result = PackageManager.packageIdsInSet(user, set.getLabel(), null);


        TagHelper.bindElaboratorTo(LIST_ACTION, result.getElaborator(), request);
        request.setAttribute("cid", chan.getId());
        request.setAttribute("channel_name", chan.getName());
        request.setAttribute(ListTagHelper.PARENT_URL, request.getRequestURI());        request.setAttribute("pageList", result);

        String button = LocalizationService.getInstance().getMessage(
        "channel.jsp.package.confirmbutton");

        if (button.equals(request.getParameter("confirm")) && set.size() > 0) {
            removePackages(user, chan, set);
...
    private void removePackages(User user, Channel chan, RhnSet set) {
        for (Long id : set.getElementValues()) {
            chan.removePackage(PackageFactory.lookupByIdAndUser(id, user));
        }
        List<Long> packList = new ArrayList<Long>();
        packList.addAll(set.getElementValues());
        ErrataCacheManager.deleteCacheEntriesForChannelPackages(chan.getId(), packList);
        ChannelManager.refreshWithNewestPackages(chan, "web.channel_package_remove");
    }
...

Comment 6 Justin Sherrill 2010-01-11 16:03:14 UTC

fixed in master.  baf5885465437d3797cbf53144998f996db74358

Package removal now should take a couple of seconds.

Comment 10 Justin Sherrill 2010-01-21 19:21:35 UTC

Test Packages now available

Comment 15 Garik Khachikyan 2010-02-12 13:20:14 UTC

# VERIFIED

Tried the reproducer for a fresh installed Satellite 5.3.0 and for a cloned RHEL 5 i386 server base channel.

After 2 minute of loading it gave an ISE and only service restart helped to bring system be operational.

With the packages fix, it does removes the ~6300 packages in about 25 sec without any problem and still stays operational :)

Many thanks for the fix.

Checked against Satellite installed on RHEL 4 AS, RHEL 5 Server.

Comment 17 errata-xmlrpc 2010-02-16 08:39:22 UTC

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0105.html