| Summary: | Not all content is being mirrored to the SE repos | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Og Maciel <omaciel> | ||||
| Component: | Content Management | Assignee: | Lukas Zapletal <lzap> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Og Maciel <omaciel> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | urgent | ||||||
| Version: | 6.0.0 | CC: | bkearney, inecas, mmccune, msolberg, mwagner | ||||
| Target Milestone: | Unspecified | Keywords: | Triaged | ||||
| Target Release: | Unused | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2012-08-22 18:27:20 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Bug Depends On: | 797195 | ||||||
| Bug Blocks: | 747354 | ||||||
| Attachments: |
|
||||||
|
Description
Og Maciel
2012-02-14 22:07:55 UTC
will do some digging into this one Still investigating, added a new product KatelloAll to my existing provider (already has a product named SystemEngine) and added 2 repositories, each pointing to katello beta2 and katello-tools beta2 for the agent. Synced and promoted this product alongside my RHEL 6 product (6.2 and 6Server), which failed with several OS errors (but the content is there, scroll all the way down):
# tail -n 100 /var/log/pulp/pulp.log
2012-02-14 22:35:25,050 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:254 Sync invoked for repo <NYDateCenter-Development-KatelloAll-Beta02>
2012-02-14 22:35:25,056 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:268 Sync of NYDateCenter-Development-KatelloAll-Beta02 starting, skip_dict = {}
2012-02-14 22:35:25,062 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1207 sync of /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Beta02 for repo NYDateCenter-Development-KatelloAll-Beta02
2012-02-14 22:35:25,125 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:851 Found 252 packages in /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Beta02
2012-02-14 22:35:25,134 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:863 Found 0 delta rpm packages in /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Beta02
2012-02-14 22:35:25,140 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:851 Found 252 packages in /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Beta02
2012-02-14 22:35:25,375 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1044 Finished cloning 252 packages
2012-02-14 22:35:25,381 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:863 Found 0 delta rpm packages in /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Beta02
2012-02-14 22:35:25,382 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1098 No image files to import
2012-02-14 22:35:25,382 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1265 Copying repodata from /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Beta02/repodata to /var/lib/pulp//repos/NYDateCenter/Development/custom/KatelloAll/Beta02/repodata
2012-02-14 22:35:25,907 11655:140629690537728: pulp.server.api.repo_sync:INFO: repo_sync:95 Creating [parent] feed repo [NYDateCenter-Development-KatelloAll-Agent] cloned from [NYDateCenter-KatelloAll-Agent] with relative_path [NYDateCenter/Development/custom/KatelloAll/Agent]
2012-02-14 22:35:29,354 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:440 No image files to import to repo..
2012-02-14 22:35:29,368 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:420 No metadata for 'File Sync' present; no files to import to repo..
2012-02-14 22:35:29,388 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:410 Skipping errata imports from sync process
2012-02-14 22:35:29,397 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:275 Sync returned 252 packages, 0 errata in 4.34034800529 seconds
2012-02-14 22:35:29,405 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:286 0 old packages to process, 252 new packages to process
2012-02-14 22:35:29,482 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:301 Examining 0 errata from repo NYDateCenter-Development-KatelloAll-Beta02
2012-02-14 22:35:29,488 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:305 Removing 0 old errata from repo NYDateCenter-Development-KatelloAll-Beta02
2012-02-14 22:35:29,536 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:314 Adding 0 new errata to repo NYDateCenter-Development-KatelloAll-Beta02
2012-02-14 22:35:29,633 11655:140627899578112: pulp.server.api.keystore:INFO: keystore:201 mkdir: /var/lib/pulp/published/gpg/NYDateCenter/Library/custom/KatelloAll/Beta02
2012-02-14 22:35:29,678 11655:140627899578112: pulp.server.api.keystore:INFO: keystore:201 mkdir: /var/lib/pulp/published/gpg/NYDateCenter/Development/custom/KatelloAll/Beta02
2012-02-14 22:35:29,685 11655:140627899578112: pulp.server.api.repo:INFO: repo:1764 repository (NYDateCenter-Development-KatelloAll-Beta02), added keys: []
2012-02-14 22:35:29,727 11655:140627899578112: pulp.server.api.repo:INFO: repo:2007 Successfully added files [] to repo NYDateCenter-Development-KatelloAll-Beta02
2012-02-14 22:35:29,729 11655:140627899578112: pulp.server.tasking.task:INFO: task:456 Task succeeded: Task 189ff2a8-5786-11e1-b393-5254001dfa20: _clone(NYDateCenter-Development-KatelloAll-Beta02, feed=parent, groupid=['product:1329276678403', 'env:3', 'org:2', 'content:1329276728229'], synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7fe6c8241b50>, progress_callback=<bound method RepoCloneTask.progress_callback of <pulp.server.api.repo_clone_task.RepoCloneTask object at 0x7fe6c8241fd0>>, filters=[], id=NYDateCenter-KatelloAll-Beta02, clone_name=Beta02, relative_path=NYDateCenter/Development/custom/KatelloAll/Beta02)
2012-02-14 22:35:30,122 11655:140629690537728: pulp.server.util:INFO: util:502 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha', '--update', '/var/lib/pulp//repos/NYDateCenter/Development/custom/KatelloAll/Agent']
2012-02-14 22:35:31,257 11655:140629690537728: pulp.server.util:INFO: util:553 createrepo on /var/lib/pulp//repos/NYDateCenter/Development/custom/KatelloAll/Agent finished
2012-02-14 22:35:31,257 11655:140629690537728: pulp.server.util:INFO: util:555 Nothing further to check; we got our fresh metadata
2012-02-14 22:35:31,339 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:254 Sync invoked for repo <NYDateCenter-Development-KatelloAll-Agent>
2012-02-14 22:35:31,342 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:268 Sync of NYDateCenter-Development-KatelloAll-Agent starting, skip_dict = {}
2012-02-14 22:35:31,348 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1207 sync of /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Agent for repo NYDateCenter-Development-KatelloAll-Agent
2012-02-14 22:35:31,455 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:851 Found 25 packages in /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Agent
2012-02-14 22:35:31,456 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:863 Found 0 delta rpm packages in /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Agent
2012-02-14 22:35:31,457 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:851 Found 25 packages in /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Agent
2012-02-14 22:35:31,476 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1044 Finished cloning 25 packages
2012-02-14 22:35:31,477 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:863 Found 0 delta rpm packages in /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Agent
2012-02-14 22:35:31,477 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1098 No image files to import
2012-02-14 22:35:31,477 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1265 Copying repodata from /var/lib/pulp//repos/NYDateCenter/Library/custom/KatelloAll/Agent/repodata to /var/lib/pulp//repos/NYDateCenter/Development/custom/KatelloAll/Agent/repodata
2012-02-14 22:35:31,582 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:440 No image files to import to repo..
2012-02-14 22:35:31,587 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:420 No metadata for 'File Sync' present; no files to import to repo..
2012-02-14 22:35:31,593 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:410 Skipping errata imports from sync process
2012-02-14 22:35:31,594 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:275 Sync returned 25 packages, 0 errata in 0.251482009888 seconds
2012-02-14 22:35:31,597 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:286 0 old packages to process, 25 new packages to process
2012-02-14 22:35:31,604 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:301 Examining 0 errata from repo NYDateCenter-Development-KatelloAll-Agent
2012-02-14 22:35:31,607 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:305 Removing 0 old errata from repo NYDateCenter-Development-KatelloAll-Agent
2012-02-14 22:35:31,619 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:314 Adding 0 new errata to repo NYDateCenter-Development-KatelloAll-Agent
2012-02-14 22:35:31,643 11655:140627899578112: pulp.server.api.keystore:INFO: keystore:201 mkdir: /var/lib/pulp/published/gpg/NYDateCenter/Library/custom/KatelloAll/Agent
2012-02-14 22:35:31,646 11655:140627899578112: pulp.server.api.keystore:INFO: keystore:201 mkdir: /var/lib/pulp/published/gpg/NYDateCenter/Development/custom/KatelloAll/Agent
2012-02-14 22:35:31,647 11655:140627899578112: pulp.server.api.repo:INFO: repo:1764 repository (NYDateCenter-Development-KatelloAll-Agent), added keys: []
2012-02-14 22:35:31,652 11655:140627899578112: pulp.server.api.repo:INFO: repo:2007 Successfully added files [] to repo NYDateCenter-Development-KatelloAll-Agent
2012-02-14 22:35:31,653 11655:140627899578112: pulp.server.tasking.task:INFO: task:456 Task succeeded: Task 1c787b40-5786-11e1-8cc7-5254001dfa20: _clone(NYDateCenter-Development-KatelloAll-Agent, feed=parent, groupid=['product:1329276678403', 'env:3', 'org:2', 'content:1329276769221'], synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7fe6d1010550>, progress_callback=<bound method RepoCloneTask.progress_callback of <pulp.server.api.repo_clone_task.RepoCloneTask object at 0x7fe6d1010b10>>, filters=[], id=NYDateCenter-KatelloAll-Agent, clone_name=Agent, relative_path=NYDateCenter/Development/custom/KatelloAll/Agent)
2012-02-14 22:35:32,013 11655:140629577299712: pulp.server.webservices.controllers.repositories:INFO: repositories:1175 sync timeout passed : None
2012-02-14 22:35:32,269 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:254 Sync invoked for repo <NYDateCenter-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2>
2012-02-14 22:35:32,541 11655:140629577299712: pulp.server.webservices.controllers.repositories:INFO: repositories:1175 sync timeout passed : None
2012-02-14 22:35:32,654 11655:140627899578112: pulp.server.api.repo_sync:INFO: repo_sync:268 Sync of NYDateCenter-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2 starting, skip_dict = {}
2012-02-14 22:35:33,407 11655:140629566809856: pulp.server.api.repo_sync:INFO: repo_sync:254 Sync invoked for repo <NYDateCenter-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6Server>
2012-02-14 22:35:34,454 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:1207 sync of /var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6.2/x86_64/os for repo NYDateCenter-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2
2012-02-14 22:35:34,478 11655:140629566809856: pulp.server.api.repo_sync:INFO: repo_sync:268 Sync of NYDateCenter-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6Server starting, skip_dict = {}
2012-02-14 22:35:36,115 11655:140629566809856: pulp.server.api.synchronizers:INFO: synchronizers:1207 sync of /var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6Server/x86_64/os for repo NYDateCenter-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6Server
2012-02-14 22:37:41,226 11655:140629566809856: pulp.server.api.synchronizers:INFO: synchronizers:851 Found 4005 packages in /var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6Server/x86_64/os
2012-02-14 22:37:44,338 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:851 Found 4094 packages in /var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6.2/x86_64/os
2012-02-14 22:38:00,252 11655:140629566809856: pulp.server.api.synchronizers:INFO: synchronizers:863 Found 0 delta rpm packages in /var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6Server/x86_64/os
2012-02-14 22:38:39,706 11655:140629566809856: pulp.server.api.synchronizers:INFO: synchronizers:959 Working on 0/6840
2012-02-14 22:38:41,622 11655:140629566809856: pulp.server.tasking.task:ERROR: task:474 Task failed: Task 1db75eab-5786-11e1-94f1-5254001dfa20: _sync(NYDateCenter-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6Server, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7fe68f049810>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7fe6cdfb2cd0>>)
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 420, in run
result = self.callable(*self.args, **self.kwargs)
File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 272, in _sync
progress_callback, synchronizer, max_speed, threads)
File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 355, in fetch_content
progress_callback, max_speed, threads)
File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 163, in sync
max_speed=max_speed, threads=threads)
File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 1241, in local
blacklist_packages, progress_callback)
File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 977, in _sync_rpms
item_size = self._calculate_bytes(src_repo_dir, [pkg.relativepath])
File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 673, in _calculate_bytes
bytes += os.stat(os.path.join(dir, pkg))[6]
OSError: [Errno 2] No such file or directory: '/var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6Server/x86_64/os/Packages/ConsoleKit-0.4.1-3.el6.x86_64.rpm'
2012-02-14 22:39:01,972 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:863 Found 0 delta rpm packages in /var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6.2/x86_64/os
2012-02-14 22:39:21,937 11655:140627899578112: pulp.server.api.synchronizers:INFO: synchronizers:959 Working on 0/6840
2012-02-14 22:39:22,082 11655:140627899578112: pulp.server.tasking.task:ERROR: task:474 Task failed: Task 1d0e80fd-5786-11e1-909e-5254001dfa20: _sync(NYDateCenter-Development-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7fe6d0ae0310>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7fe6cdfb2e10>>)
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 420, in run
result = self.callable(*self.args, **self.kwargs)
File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 272, in _sync
progress_callback, synchronizer, max_speed, threads)
File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 355, in fetch_content
progress_callback, max_speed, threads)
File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 163, in sync
max_speed=max_speed, threads=threads)
File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 1241, in local
blacklist_packages, progress_callback)
File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 977, in _sync_rpms
item_size = self._calculate_bytes(src_repo_dir, [pkg.relativepath])
File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 673, in _calculate_bytes
bytes += os.stat(os.path.join(dir, pkg))[6]
OSError: [Errno 2] No such file or directory: '/var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6.2/x86_64/os/Packages/389-ds-base-1.2.9.14-1.el6.x86_64.rpm'
[root@qetello01 ~]# ls -l /var/lib/pulp//repos/NYDateCenter/Library/content/dist/rhel/server/6/6.2/x86_64/os/Packages/389-ds-base-
389-ds-base-1.2.8.2-1.el6_1.3.x86_64.rpm 389-ds-base-libs-1.2.8.2-1.el6.x86_64.rpm
389-ds-base-1.2.8.2-1.el6.x86_64.rpm 389-ds-base-libs-1.2.9.14-1.el6_2.2.i686.rpm
389-ds-base-1.2.9.14-1.el6_2.2.x86_64.rpm 389-ds-base-libs-1.2.9.14-1.el6_2.2.x86_64.rpm
389-ds-base-libs-1.2.8.2-1.el6_1.3.i686.rpm 389-ds-base-libs-1.2.9.14-1.el6.i686.rpm
389-ds-base-libs-1.2.8.2-1.el6_1.3.x86_64.rpm 389-ds-base-libs-1.2.9.14-1.el6.x86_64.rpm
389-ds-base-libs-1.2.8.2-1.el6.i686.rpm
# tail -n 100 /var/log/katello/delayed_job.log 2012-02-14T22:41:45-0500: [Worker(delayed_job host:qetello01.usersys.redhat.com pid:11807)] Repository#index_after_sync failed with NoMethodError: undefined method `[]' for nil:NilClass - 0 failed attempts 2012-02-14T22:41:45-0500: [Worker(delayed_job host:qetello01.usersys.redhat.com pid:11807)] PERMANENTLY removing Repository#index_after_sync because of 1 consecutive failures. undefined method `[]' for nil:NilClass /usr/share/katello/app/models/pulp_task_status.rb:53:in `error' /usr/share/katello/app/models/pulp_task_status.rb:77:in `any_task_running' /usr/share/katello/app/models/pulp_task_status.rb:71:in `each' /usr/share/katello/app/models/pulp_task_status.rb:71:in `any_task_running' /usr/share/katello/app/models/pulp_task_status.rb:22:in `wait_for_tasks' /usr/share/katello/app/models/glue/pulp/repo.rb:371:in `index_after_sync' /usr/share/katello/app/models/async_operation.rb:41:in `send' /usr/share/katello/app/models/async_operation.rb:41:in `perform' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/backend/base.rb:87:in `invoke_job' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:120:in `run' /usr/lib/ruby/1.8/timeout.rb:67:in `timeout' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:120:in `run' /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:119:in `run' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:177:in `reserve_and_run_one_job' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:104:in `work_off' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:103:in `times' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:103:in `work_off' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:78:in `start' /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:77:in `start' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:74:in `loop' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/worker.rb:74:in `start' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:104:in `run' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:83:in `run_process' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/application.rb:249:in `call' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/application.rb:249:in `start_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/daemonize.rb:197:in `call' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/daemonize.rb:197:in `call_as_daemon' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/application.rb:253:in `start_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/application.rb:293:in `start' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/controller.rb:70:in `run' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons.rb:195:in `run_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/cmdline.rb:109:in `call' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons/cmdline.rb:109:in `catch_exceptions' /usr/lib/ruby/gems/1.8/gems/daemons-1.1.4/lib/daemons.rb:194:in `run_proc' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:81:in `run_process' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:75:in `daemonize' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:73:in `times' /usr/lib/ruby/gems/1.8/gems/delayed_job-2.1.4/lib/delayed/command.rb:73:in `daemonize' script/delayed_job:5 2012-02-14T22:41:45-0500: [Worker(delayed_job host:qetello01.usersys.redhat.com pid:11807)] 3 jobs processed at 0.0078 j/s, 3 failed ... *** Bug 796787 has been marked as a duplicate of this bug. *** Reproduced on Pulp side, Should be fixed in next pulp build. See https://bugzilla.redhat.com/show_bug.cgi?id=795819 Although the recent fix on the Pulp side probably won't fix this issue. See https://bugzilla.redhat.com/show_bug.cgi?id=795819#c6 for what the real cause might be. Issue on Pulp probably causing this problem is filed in https://bugzilla.redhat.com/show_bug.cgi?id=797195 Are you able to try it again with Grinder 0.139 when it comes out? Verified: * candlepin-0.5.24-1.el6.noarch * candlepin-tomcat6-0.5.24-1.el6.noarch * katello-0.1.303-1.el6.noarch * katello-all-0.1.303-1.el6.noarch * katello-candlepin-cert-key-pair-1.0-1.noarch * katello-certs-tools-1.0.4-1.el6.noarch * katello-cli-0.1.102-1.el6.noarch * katello-cli-common-0.1.102-1.el6.noarch * katello-common-0.1.303-1.el6.noarch * katello-configure-0.1.104-1.el6.noarch * katello-glue-candlepin-0.1.303-1.el6.noarch * katello-glue-foreman-0.1.303-1.el6.noarch * katello-glue-pulp-0.1.303-1.el6.noarch * katello-qpid-broker-key-pair-1.0-1.noarch * katello-qpid-client-key-pair-1.0-1.noarch * katello-selinux-0.1.8-1.el6.noarch * pulp-1.0.0-4.el6.noarch * pulp-common-1.0.0-4.el6.noarch * pulp-selinux-server-1.0.0-4.el6.noarch |