Red Hat Bugzilla – Bug 1002041
spacewalk-abrt: large file upload fails
Last modified: 2013-10-03 08:10:49 EDT
Description of problem:
When crash directory contains large files,
> spacewalk-abrt --sync
ends either with "Internal Server Error" or "Network error: Unexpected EOF".
This is issue for abrt-addon-vmcore crash-directories
Version-Release number of selected component:
How reproducible: almost always/deterministic
sometime few (observed max 2) vmcore crashes are synced.
Steps to Reproduce:
1. sleep 100 &
2. kill -s SEGV $!
3. abrt-cli list # determine crash-direcotry
4. dd if=/dev/urandom of=$crashdir/reason bs=1M count=100 # pump up some file
5. spacewalk-abrt --sync
Upload starts, few files are uploaded, then either
"Internal Server Error" or "Network error: Unexpected EOF" error is produced.
Files are uploaded, if limit to file-size is set, it should be evaluated sooner than
file is fully uploaded. No crash...
First observed while reproducing
The problem here is that whenever the client tries to upload a big file
to the Spacewalk / Satellite server, it takes a long time for the server
to actually process the request (in case of a 100MB file it took ~ 30 minutes),
so by the time the server is ready to respond, the client connection times out
and the whole upload on the client side finishes with the following traceback:
Traceback (most recent call last):
File "/usr/bin/spacewalk-abrt", line 86, in <module>
File "/usr/share/rhn/up2date_client/rhncli.py", line 96, in run
sys.exit(self.main() or 0)
File "/usr/bin/spacewalk-abrt", line 72, in main
File "/usr/share/rhn/spacewalk_abrt/abrt.py", line 117, in report
server.abrt.upload_crash_file(systemid, basename, crash_file_data)
File "/usr/share/rhn/up2date_client/rhnserver.py", line 63, in __call__
return rpcServer.doCall(method, *args, **kwargs)
File "/usr/share/rhn/up2date_client/rpcServer.py", line 204, in doCall
ret = method(*args, **kwargs)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__
return self.__send(self.__name, args)
File "/usr/share/rhn/up2date_client/rpcServer.py", line 38, in _request1
ret = self._request(methodname, params)
File "/usr/lib/python2.7/site-packages/rhn/rpclib.py", line 384, in _request
self._handler, request, verbose=self._verbose)
File "/usr/lib/python2.7/site-packages/rhn/transports.py", line 171, in request
headers, fd = req.send_http(host, handler)
File "/usr/lib/python2.7/site-packages/rhn/transports.py", line 725, in send_http
response = self._connection.getresponse()
File "/usr/lib/python2.7/site-packages/rhn/connections.py", line 94, in getresponse
File "/usr/lib64/python2.7/httplib.py", line 409, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python2.7/httplib.py", line 365, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/site-packages/rhn/SSL.py", line 287, in readline
File "/usr/lib/python2.7/site-packages/rhn/SSL.py", line 218, in _poll
raise TimeoutException, "Connection timed out on %s" % caller_name
<class 'up2date_client.up2dateErrors.CommunicationError'>: Error communicating with server. The message was:
Connection timed out on readline
One way to help the situation is make the client not to upload
files if they're not desired -- i.e. if they're over the size limit
or if the file uploading is not allowed at all.
spacewalk.git master: 1d83f1eb87814b1c4b8715197daa50ef7c3d1cc0
Spacewalk.git master: 3e7bc878aed77e04f92ff91875ffa436b2c2a907
Newly spacewalk-abrt doesn't try to upload huge files.
These files aren't uploaded on SAT anymore.
Used initial reproducer in comment #0
This bug was fixed with updated packages being released within the RHN Tools channels.
RHN Tools (for Sat 5.6 GA) Errata text: