Bug 1127293 - rhnpush with large rpms fails with Internal Server Error
Summary: rhnpush with large rpms fails with Internal Server Error
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Spacewalk
Classification: Community
Component: Server
Version: 2.2
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Michael Mráka
QA Contact: Red Hat Satellite QA List
URL:
Whiteboard:
Depends On: 1127291
Blocks: space23
TreeView+ depends on / blocked
 
Reported: 2014-08-06 14:33 UTC by Tasos Papaioannou
Modified: 2015-04-14 19:02 UTC (History)
4 users (show)

Fixed In Version: spacewalk-backend-2.3.9-1
Doc Type: Bug Fix
Doc Text:
Clone Of: 1127291
Environment:
Last Closed: 2015-04-14 19:02:48 UTC


Attachments (Terms of Use)

Description Tasos Papaioannou 2014-08-06 14:33:39 UTC
+++ This bug was initially created as a clone of Bug #1127291 +++

Description of problem:

Using rhnpush on an rpm with an archive size of > 4GB fails with an Internal Server Error and traceback email:

****
Exception reported from satellite.example.com
Time: Tue Aug  5 17:25:24 2014
Exception type <type 'exceptions.TypeError'>
Exception while handling function server.apacheUploadServer._wrapper
Request object information:
URI: /PACKAGE-PUSH
[...]
Exception Handler Information
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/spacewalk/server/apacheUploadServer.py", line 100, in _wrapper
    ret = function(req)
  File "/usr/share/rhn/upload_server/handlers/package_push/package_push.py", line 125, in handler
    relative_path=self.rel_package_path, org_id=self.org_id)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnPackageUpload.py", line 130, in push_package
    header_end=a_pkg.header_end, channels=channels)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/mpmSource.py", line 224, in create_package
    header_end=header_end, channels=channels)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/headerSource.py", line 427, in createPackage
    channels)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/headerSource.py", line 152, in populate
    header_start, header_end)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/headerSource.py", line 74, in populate
    val = long(val) + 2 ** 32
TypeError: long() argument must be a string or a number, not 'NoneType'
****


Version-Release number of selected component (if applicable):

spacewalk-backend-server-2.0.3-29.el6sat.noarch

How reproducible:

100%

Steps to Reproduce:

1.) Build an rpm with more than 4GB worth of files. For example:

$ ll ./rpmbuild/RPMS/x86_64/tpapaioa-1.0-1.x86_64.rpm 
-rw-r--r--. 1 root root 572570597 Aug  4 15:52 ./rpmbuild/RPMS/x86_64/tpapaioa-1.0-1.x86_64.rpm

$ rpm -qip ./rpmbuild/RPMS/x86_64/tpapaioa-1.0-1.x86_64.rpm 
Name        : tpapaioa                     Relocations: (not relocatable)
Version     : 1.0                               Vendor: None
Release     : 1                             Build Date: Mon 04 Aug 2014 03:29:20 PM EDT
Install Date: (not installed)               Build Host: satellite.example.com
Group       : RHN/Security                  Source RPM: tpapaioa-1.0-1.src.rpm
Size        : 7725768704                       License: GPL
Signature   : (none)
Packager    : None
Summary     : Tasos test
Description :
Tasos test

$ rpm -qp ./rpmbuild/RPMS/x86_64/tpapaioa-1.0-1.x86_64.rpm --dump
/root/core1 3862884352 1407180299 844a42cc1a0ad1e8fe87d19d10c61de5 0100600 root root 0 0 0 X
/root/core2 3862884352 1407180401 844a42cc1a0ad1e8fe87d19d10c61de5 0100600 root root 0 0 0 X

2.) Push the rpm to a channel on the Satellite:

# rhnpush 
# rhnpush --server localhost -c tpapaioa-1 --nosig ./rpmbuild/RPMS/x86_64/tpapaioa-1.0-1.x86_64.rpm
Internal server error 500 Internal Server Error
Error pushing /root/rpmbuild/RPMS/x86_64/tpapaioa-1.0-1.x86_64.rpm:  (500)
Waiting 4 seconds and trying again...
Internal server error 500 Internal Server Error
Error pushing /root/rpmbuild/RPMS/x86_64/tpapaioa-1.0-1.x86_64.rpm:  (500)
Waiting 1 seconds and trying again...
Internal server error 500 Internal Server Error
Error pushing /root/rpmbuild/RPMS/x86_64/tpapaioa-1.0-1.x86_64.rpm:  (500)
Waiting 2 seconds and trying again...
Giving up after 3 attempts

Actual results:

Internal Server Error, package push fails.

Expected results:

No Internal Server Error, package pushed successfully.

Additional info:

The code responsible for the traceback is:

./backend/server/importlib/headerSource.py:

            if f == 'payload_size':
                # workaround for older rpms where signed
                # attributes go negative for size > 2G
                if val < 0:
                    val = long(val) + 2 ** 32

The value of "payload_size" is pulled from the "archivesize" field of the rpm header. This field is 32 bits long, and is populated if the archive is smaller than 4GB. For large rpms, the "longarchivesize" field is populated instead, resulting in a None value for the value associated with "payload_size". "val < 0" evaluates as True, resulting in the TypeError when trying to pass it to long().

A proposed Spacewalk patch available at https://github.com/spacewalkproject/spacewalk/pull/133 checks for "longarchivesize" of the "archivesize" field has a None value. This prevents NoneType values from falling through to the "val < 0" check. I don't know if the check for negative values is still necessary, but I've left it in.

Comment 1 Michael Mráka 2014-08-29 09:28:33 UTC
Pull request merged to master as
commit 33d17aa2cd8698a1acdd31cdaff53245750382cc
    fix traceback when pushing rpms with archive size > 4GB

Comment 2 Grant Gainey 2015-03-23 16:58:51 UTC
Moving bugs to ON_QA as we move to release Spacewalk 2.3

Comment 3 Grant Gainey 2015-04-14 19:02:48 UTC
Spacewalk 2.3 has been released. See

https://fedorahosted.org/spacewalk/wiki/ReleaseNotes23


Note You need to log in before you can comment on or make changes to this bug.