Bug 1002041 - spacewalk-abrt: large file upload fails
Summary: spacewalk-abrt: large file upload fails
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Server
Version: 560
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
Assignee: Milan Zázrivec
QA Contact: Lukas Pramuk
URL:
Whiteboard:
Depends On:
Blocks: sat560-blockers sat560-abrt 982642
TreeView+ depends on / blocked
 
Reported: 2013-08-28 11:28 UTC by Matej Kollar
Modified: 2013-10-03 12:10 UTC (History)
4 users (show)

Fixed In Version: spacewalk-abrt-2.0.1-4
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-10-01 19:59:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Matej Kollar 2013-08-28 11:28:44 UTC
Description of problem:

  When crash directory contains large files,

  > spacewalk-abrt --sync

  ends either with "Internal Server Error" or "Network error: Unexpected EOF".
  This is issue for abrt-addon-vmcore crash-directories


Version-Release number of selected component:

  spacewalk-backend-2.0.3-10.el6sat.noarch

How reproducible: almost always/deterministic
sometime few (observed max 2) vmcore crashes are synced.

Steps to Reproduce:

  1. sleep 100 &
  2. kill -s SEGV $!
  3. abrt-cli list # determine crash-direcotry
  4. dd if=/dev/urandom of=$crashdir/reason bs=1M count=100  # pump up some file
  5. spacewalk-abrt --sync

Actual results:
  Upload starts, few files are uploaded, then either
  "Internal Server Error" or "Network error: Unexpected EOF" error is produced.

Expected results:
  Files are uploaded, if limit to file-size is set, it should be evaluated sooner than
  file is fully uploaded. No crash...

Additional info:
  First observed while reproducing
  https://bugzilla.redhat.com/show_bug.cgi?id=982642

Comment 3 Milan Zázrivec 2013-09-03 11:16:06 UTC
The problem here is that whenever the client tries to upload a big file
to the Spacewalk / Satellite server, it takes a long time for the server
to actually process the request (in case of a 100MB file it took ~ 30 minutes),
so by the time the server is ready to respond, the client connection times out
and the whole upload on the client side finishes with the following traceback:

Traceback (most recent call last):
  File "/usr/bin/spacewalk-abrt", line 86, in <module>
    cli.run()
  File "/usr/share/rhn/up2date_client/rhncli.py", line 96, in run
    sys.exit(self.main() or 0)
  File "/usr/bin/spacewalk-abrt", line 72, in main
    spacewalk_abrt.abrt.report(self.options.report)
  File "/usr/share/rhn/spacewalk_abrt/abrt.py", line 117, in report
    server.abrt.upload_crash_file(systemid, basename, crash_file_data)
  File "/usr/share/rhn/up2date_client/rhnserver.py", line 63, in __call__
    return rpcServer.doCall(method, *args, **kwargs)
  File "/usr/share/rhn/up2date_client/rpcServer.py", line 204, in doCall
    ret = method(*args, **kwargs)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/share/rhn/up2date_client/rpcServer.py", line 38, in _request1
    ret = self._request(methodname, params)
  File "/usr/lib/python2.7/site-packages/rhn/rpclib.py", line 384, in _request
    self._handler, request, verbose=self._verbose)
  File "/usr/lib/python2.7/site-packages/rhn/transports.py", line 171, in request
    headers, fd = req.send_http(host, handler)
  File "/usr/lib/python2.7/site-packages/rhn/transports.py", line 725, in send_http
    response = self._connection.getresponse()
  File "/usr/lib/python2.7/site-packages/rhn/connections.py", line 94, in getresponse
    response.begin()
  File "/usr/lib64/python2.7/httplib.py", line 409, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 365, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python2.7/site-packages/rhn/SSL.py", line 287, in readline
    self._poll(select.POLLIN, 'readline')
  File "/usr/lib/python2.7/site-packages/rhn/SSL.py", line 218, in _poll
    raise TimeoutException, "Connection timed out on %s" % caller_name
<class 'up2date_client.up2dateErrors.CommunicationError'>: Error communicating with server. The message was:
Connection timed out on readline

One way to help the situation is make the client not to upload
files if they're not desired -- i.e. if they're over the size limit
or if the file uploading is not allowed at all.

spacewalk.git master: 1d83f1eb87814b1c4b8715197daa50ef7c3d1cc0

Comment 11 Matej Kollar 2013-09-23 11:35:41 UTC
Spacewalk.git master: 3e7bc878aed77e04f92ff91875ffa436b2c2a907

Comment 14 Lukas Pramuk 2013-09-30 09:16:03 UTC
VERIFIED. spacewalk-abrt-2.0.1-4.el6sat

Newly spacewalk-abrt doesn't try to upload huge files.
These files aren't uploaded on SAT anymore.
Used initial reproducer in comment #0

Comment 15 Clifford Perry 2013-10-01 19:59:25 UTC
This bug was fixed with updated packages being released within the RHN Tools channels. 


RHN Tools (for Sat 5.6 GA) Errata text:

https://rhn.redhat.com/errata/RHEA-2013-1391.html


Note You need to log in before you can comment on or make changes to this bug.