Bug 1004954 - getting "KeyError: '1000010848'" when syncing from big satellite instance
getting "KeyError: '1000010848'" when syncing from big satellite instance
Product: Subscription Asset Manager
Classification: Red Hat
Component: Splice (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Splice Developers
Depends On:
Blocks: sam13-tracker
  Show dependency treegraph
Reported: 2013-09-05 16:31 EDT by Jan Hutař
Modified: 2013-10-01 07:24 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-10-01 07:24:55 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Jan Hutař 2013-09-05 16:31:25 EDT
Description of problem:
Getting "KeyError: '1000010848'" when syncing from big satellite instance

Version-Release number of selected component (if applicable):

How reproducible:
1 of 1 attempt

Steps to Reproduce:
1. Setup Splice to sync from Satellite 5.6.0 to SAM 1.3 snapshot #4
2. $ spacewalk-splice-checkin

Actual results:
Traceback (most recent call last):
  File "/usr/bin/spacewalk-splice-checkin", line 97, in <module>
  File "/usr/lib/python2.6/site-packages/spacewalk_splice_tool/checkin.py", line 206, in main
  File "/usr/lib/python2.6/site-packages/spacewalk_splice_tool/checkin.py", line 159, in spacewalk_sync
    kps.upload_host_guest_mapping(hosts_guests, katello_consumer_list)
  File "/usr/lib/python2.6/site-packages/spacewalk_splice_tool/katello_sync.py", line 231, in upload_host_guest_mapping
KeyError: '1000010848'

Expected results:
No traceback.
Comment 2 Chris Duryee 2013-09-05 17:13:00 EDT
According to a katello developer, we are likely pushing data with too many threads, and it's having trouble keeping up. Reducing the num_threads count in checkin.conf should resolve this.

Let me know if num_threads=2 works better, and if it does I'll change the default in checkin.conf.
Comment 3 Jan Hutař 2013-09-06 02:31:09 EDT
Yes. This helped. I have decreased num_threads in /etc/splice/checkin.conf from 4 to 2 and re-run the command (note I have not purged SAM DB to be 100% sure all data gets inserted again) and it ended without traceback.
Comment 4 Chris Duryee 2013-09-06 11:12:07 EDT
eb0c566 spacewalk-splice-tool 0.42+

I changed the default num_threads from 4 to 2. This makes creates and updates somewhat slower, but avoids the issues mentioned above.
Comment 5 Chris Duryee 2013-09-12 12:41:31 EDT
After discussion with vkuznets, we changed this to 1
Comment 6 Vitaly Kuznetsov 2013-09-13 10:25:57 EDT
In Snapshot 6:

# grep num_threads /etc/splice/checkin.conf 
num_threads = 1

Jan, can you please confirm you're not hitting the initial issue with Snapshot6?
Comment 7 Jan Hutař 2013-09-16 04:55:57 EDT
Is there a way how to list all the systems in the SAM instance on the command line? I want to compare it to list in satellite. Would prefer some CSV file.
Comment 8 Vitaly Kuznetsov 2013-09-16 05:02:44 EDT
(In reply to Jan Hutař from comment #7)
> Is there a way how to list all the systems in the SAM instance on the
> command line? I want to compare it to list in satellite. Would prefer some
> CSV file.

Yes, it's possible.
E.g. :
#headpin -u admin -p admin system list --org satellite-1 -g --noheading -d ','

You can list all your org labels with 'headpin org list'.
Comment 9 Jan Hutař 2013-09-16 08:45:58 EDT

Note it will take some time to me (day or so) waiting for a Beaker to prepare mine environment, will report back with results later.
Comment 11 Vitaly Kuznetsov 2013-09-23 08:05:34 EDT
Verifying based on Comment 6 and Comment 10.
Comment 15 errata-xmlrpc 2013-10-01 07:24:55 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.