Bug 502581 - Sync of rhn-tools-rhel-4-as-i386 interrupted on package metadata import
Sync of rhn-tools-rhel-4-as-i386 interrupted on package metadata import
Status: CLOSED CURRENTRELEASE
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Satellite Synchronization (Show other bugs)
530
i386 Linux
low Severity low
: ---
: ---
Assigned To: Pradeep Kilambi
Garik Khachikyan
:
Depends On:
Blocks: 456985
  Show dependency treegraph
 
Reported: 2009-05-26 05:23 EDT by Garik Khachikyan
Modified: 2015-01-04 16:56 EST (History)
4 users (show)

See Also:
Fixed In Version: sat530
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-09-10 16:37:20 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
an email content about the error (2.24 KB, text/plain)
2009-05-26 05:23 EDT, Garik Khachikyan
no flags Details

  None (edit)
Description Garik Khachikyan 2009-05-26 05:23:52 EDT
Created attachment 345427 [details]
an email content about the error

Description of problem:
Synchronization of the channel "rhn-tools-rhel-4-as-i386" failed with an error: "array size too large" on importing package metadata. It was requested to sync just this channel only (no more ones from satellite-sync command line). After re-running the same task again (when the rpm-s were already fetched) it gone without an error.

Version-Release number of selected component (if applicable):
Satellite 5.3.0

How reproducible:


Steps to Reproduce:
1. Get channels "satellite-sync -crh-x86_64-desktop-4 -crhel-i386-as-4"
2. Try to sync  "satellite-sync -crhn-tools-rhel-4-as-i386"
  
Actual results:
Exception is thrown, process interrupts.

Expected results:
The channel should get sync-ed in Satellite 5.3.0

Additional info:
Here is a out dump, and attached is the log from the sent email about the failure.
------------------------
04:33:52     1333/1336 Fetch successful: spacewalk-proxy-installer-0.5.25-5.el4sat.noarch.rpm (36678 bytes)
04:33:52     1334/1336 Fetch successful: spacewalk-koan-0.1.11-7.el4sat.noarch.rpm (16170 bytes)
04:33:53     1335/1336 Fetch successful: spacewalk-proxy-installer-0.5.25-6.el4sat.noarch.rpm (38396 bytes)
04:33:53     1336/1336 Fetch successful: spacewalk-koan-0.1.11-8.el4sat.noarch.rpm (16614 bytes)
04:33:53 Processing rpm packages complete
04:33:53 No handler for step srpms
04:33:53 
04:33:53 Downloading package metadata
04:33:54    Retrieving / parsing *relevant* package metadata: rhn-tools-rhel-4-as-i386 (1336)
04:33:54    * WARNING: this may be a slow process.
            ________________________________________
Downloading:######################################## - complete
04:35:48 
04:35:48 Downloading errata data
04:35:53    Retrieving / parsing errata data: rhn-tools-rhel-4-as-i386 (0)
04:35:53 Downloading errata data complete
04:35:53 
04:35:53 Downloading kickstartable trees metadata
04:35:53    Retrieving / parsing kickstart data: rhn-tools-rhel-4-as-i386 (NONE RELEVANT)
04:35:53 
04:35:53 Downloading kickstartable trees files
04:35:53    Retrieving / parsing kickstart tree files: rhn-tools-rhel-4-as-i386 (NONE RELEVANT)
04:35:53 
04:35:53 Importing package metadata
04:35:53    Importing *relevant* package metadata: rhn-tools-rhel-4-as-i386 (1336)
04:35:53    * WARNING: this may be a slow process.
            ________________________________________
Importing:  #####################################+++ sending log as an email +++

SYNC ERROR: unhandled exception occurred:

(Check logs/email for potentially more detail)

array size too large
------------------------
Comment 1 Jan Pazdziora 2009-05-27 03:11:17 EDT
Brandon, I'm afraid I won't be able to cover this bugzilla before my departure (and thus before stage time). Is there something specific in it for me, or can I reassign it?
Comment 2 Brandon Perkins 2009-05-28 21:43:32 EDT
Reassigning to dgoodwin.
Comment 3 Pradeep Kilambi 2009-06-02 11:36:29 EDT
some first thoughts:

Oracle has a limit on number of args you can pass to a query. Which by default is 1000 iirc. Now the above error happens when this limit exceeded. 

dissecting the traceback:

File "/usr/share/rhn/server/importlib/backendLib.py", line 461, in executeStatement
    count = count + apply(statement.executemany, (), tempdict)
  File "/usr/share/rhn/server/rhnSQL/sql_base.py", line 170, in executemany
    return apply(self._execute_wrapper, (self._executemany, ) + p, kw)
  File "/usr/share/rhn/server/rhnSQL/driver_cx_Oracle.py", line 98, in _execute_wrapper
    retval = apply(function, p, kw)
  File "/usr/share/rhn/server/rhnSQL/driver_cx_Oracle.py", line 186, in _executemany
    self._real_cursor.executemany(None, arr)
ValueError: array size too large


executeStatement in backendLib.py already takes care of that by defining a chunk size of 1000 and we only extract 1000 elements from the list at a time. Also once the data reaches the oracle driver, we trim this even further to 100 by grabbing the min(max_array_size, array_size) where max_array_size = 100. So unless the default value in the db has changed or if this is perhaps an express edition or something where defaults might be different I dont see how this can happen. 

Having said that I'm in the process of running these syncs to see if this is an isolated incident or indeed something is wrong. Will post my results.
Comment 4 Garik Khachikyan 2009-06-03 07:00:08 EDT
# COMMENT

BTW: after restart of the command (running sync again when the packages are fetched already) it passes okey without that exception.
Comment 11 Garik Khachikyan 2009-06-22 08:49:38 EDT
# VERIFIED

Checked for installers: Satellite-5.3.0-RHEL4-re20090616.1 and Satellite-5.3.0-RHEL5-re20090616.1 (platform - i386).

For both cases packages metadata importing passed okey.
Comment 12 John Sefler 2009-08-21 10:43:26 EDT
Syncing a staged iso (Satellite-5.3.0-RHEL5-re20090724.0) to server.satellite.rhn_parent = satellite.rhn.stage.redhat.com


[root@sun-x4200-01 tmp]# satellite-sync -crhn-tools-rhel-4-as-i386
...<cut out 1 hours worth of syncing>...
2009/08/20 19:03:45 -04:00     1097/1098 Fetch successful: spacewalk-utils-0.5.1-6.el4sat.noarch.rpm (16548 bytes)
2009/08/20 19:03:46 -04:00     1098/1098 Fetch successful: spacewalk-utils-0.5.1-8.el4sat.noarch.rpm (20122 bytes)
2009/08/20 19:03:46 -04:00 Processing rpm packages complete
2009/08/20 19:03:46 -04:00 No handler for step srpms
2009/08/20 19:03:46 -04:00
2009/08/20 19:03:46 -04:00 Downloading package metadata
2009/08/20 19:03:46 -04:00    Retrieving / parsing *relevant* package metadata: rhn-tools-rhel-4-as-i386 (1098)
2009/08/20 19:03:46 -04:00    * WARNING: this may be a slow process.
2009/08/20 19:05:11 -04:00
2009/08/20 19:05:11 -04:00 Downloading errata data
2009/08/20 19:05:11 -04:00    Retrieving / parsing errata data: rhn-tools-rhel-4-as-i386 (0)
2009/08/20 19:05:11 -04:00 Downloading errata data complete
2009/08/20 19:05:11 -04:00
2009/08/20 19:05:11 -04:00 Downloading kickstartable trees metadata
2009/08/20 19:05:11 -04:00    Retrieving / parsing kickstart data: rhn-tools-rhel-4-as-i386 (NONE RELEVANT)
2009/08/20 19:05:11 -04:00
2009/08/20 19:05:11 -04:00 Downloading kickstartable trees files
2009/08/20 19:05:11 -04:00    Retrieving / parsing kickstart tree files: rhn-tools-rhel-4-as-i386 (NONE RELEVANT)
2009/08/20 19:05:11 -04:00
2009/08/20 19:05:11 -04:00 Importing package metadata
2009/08/20 19:05:11 -04:00    Importing *relevant* package metadata: rhn-tools-rhel-4-as-i386 (1098)
2009/08/20 19:05:11 -04:00    * WARNING: this may be a slow process.
2009/08/20 19:06:15 -04:00
2009/08/20 19:06:15 -04:00 Linking packages to channels
2009/08/20 19:06:19 -04:00 No handler for step source-packages
2009/08/20 19:06:19 -04:00
2009/08/20 19:06:19 -04:00 Importing channel errata
2009/08/20 19:06:19 -04:00    Importing *relevant* errata: rhn-tools-rhel-4-as-i386 (NONE RELEVANT)
2009/08/20 19:06:19 -04:00    No new kickstartable tree to import
    Import complete:
        Begin time: Thu Aug 20 18:05:13 2009
        End time:   Thu Aug 20 19:06:19 2009
        Elapsed:    1 hours, 1 minutes, 6 seconds


No SYNC ERROR reported.
moving to RELEASE_PENDING
Comment 13 Brandon Perkins 2009-09-10 16:37:20 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-1434.html

Note You need to log in before you can comment on or make changes to this bug.