Bug 984009

Summary: ssl timeout while adding large number of systems
Product: [Retired] Subscription Asset Manager Reporter: Vitaly Kuznetsov <vkuznets>
Component: SpliceAssignee: Chris Duryee <cduryee>
Status: CLOSED ERRATA QA Contact: mkovacik
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 1.3CC: bkearney, cduryee
Target Milestone: rc   
Target Release: 1.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-10-01 11:03:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 833466    

Description Vitaly Kuznetsov 2013-07-12 13:58:14 UTC
Description of problem:
During SST sync with katello ssl timeout can happen:
  File "/usr/bin/spacewalk-splice-checkin", line 93, in <module>
    checkin.main(opts)
  File "/usr/lib/python2.6/site-packages/spacewalk_splice_tool/checkin.py", line 188, in main
    spacewalk_sync(options)
  File "/usr/lib/python2.6/site-packages/spacewalk_splice_tool/checkin.py", line 146, in spacewalk_sync
    katello_consumer_list = katello_client.get_consumers(with_details=False)
  File "/usr/lib/python2.6/site-packages/spacewalk_splice_tool/katello_connect.py", line 166, in get_consumers
    consumer_list.append(self.systemapi.systems_by_org(orgId=org_id))
  File "/usr/lib/python2.6/site-packages/katello/client/api/system.py", line 171, in systems_by_org
    return self.server.GET(path, query)[1]
  File "/usr/lib/python2.6/site-packages/katello/client/server.py", line 425, in GET
    return self._request('GET', path, queries, custom_headers=custom_headers)
  File "/usr/lib/python2.6/site-packages/katello/client/server.py", line 249, in _request
    return self._process_response(connection.getresponse())
  File "/usr/lib64/python2.6/httplib.py", line 990, in getresponse
    response.begin()
  File "/usr/lib64/python2.6/httplib.py", line 391, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.6/httplib.py", line 349, in _read_status
    line = self.fp.readline()
  File "/usr/lib64/python2.6/socket.py", line 433, in readline
    data = recv(1)
  File "/usr/lib64/python2.6/ssl.py", line 215, in recv
    return self.read(buflen)
  File "/usr/lib64/python2.6/ssl.py", line 136, in read
    return self._sslobj.read(len)
ssl.SSLError: The read operation timed out


Version-Release number of selected component (if applicable):
spacewalk-splice-tool-0.27-1

How reproducible:
Always for e.g. 300 systems

Steps to Reproduce:
1. Create 300 systems in spacewalk (splice-export report)
2. Run SST
3. Update 300 systems (new checkins,...)
4. Run SST again

Actual results:
Traceback (above)

Expected results:
Sync goes normally

Additional info:

Comment 1 Chris Duryee 2013-07-12 14:10:13 UTC
The proposed fix for this is to call the "systems_by_org" API with pagination, so we don't hit a timeout.

Comment 2 Chris Duryee 2013-07-24 20:17:55 UTC
spacewalk-splice-tool ea56dad16d5 0.35

if you set debug logging on sst, you'll see pagination messages like:

2013-07-24 20:11:10,417 DEBUG spacewalk_splice_tool.katello_connect katello_connect:get_consumers: retrieving systems for org satellite-3, offset 0
2013-07-24 20:11:15,390 DEBUG spacewalk_splice_tool.katello_connect katello_connect:get_consumers: found 25 systems
2013-07-24 20:11:15,391 DEBUG spacewalk_splice_tool.katello_connect katello_connect:get_consumers: retrieving systems for org satellite-3, offset 25
2013-07-24 20:11:19,552 DEBUG spacewalk_splice_tool.katello_connect katello_connect:get_consumers: found 20 systems

Comment 3 Bryan Kearney 2013-08-02 18:51:34 UTC
SNAP0 contains these bug fixes. Moving to ON_QA.

Comment 5 Vitaly Kuznetsov 2013-08-21 14:24:43 UTC
Verified with 300 systems upload,

10 uploads took 4460 seconds (446 sec./upload, 1.5 sec./system)

Comment 7 errata-xmlrpc 2013-10-01 11:03:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2013-1390.html