Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
When trigger rpm updates from the Satelite web ui that also update the gofer rpm, the task will stuck forever and cause duplicate rpms and incomplete yum transaction on client side.
This is because updating the gofer rpm will cause goferd to restart and cause incomplete yum transaction.
/var/log/messages shows that goferd got restarted during the update:
goferd: [INFO][worker-0] gofer.rmi.dispatcher:603 - call: Content.update() sn=5ca337fe-8f24-4e0e-83a5-7c6c806f26f9 data={u'task_id': u'782659fb-3480-4431-a90c-7c6b7f642e62', u'consumer_id': u'2e77ff64-07b6-4326-99f7-dc31d23d38cf'}
systemd: Reloading.
systemd: Current command vanished from the unit file, execution of the command list won't be resumed.
systemd: Adding 1h 57min 40.716370s random time.
systemd: Adding 3h 52min 30.842836s random time.
systemd: Started Flexible Branding Service.
systemd: Starting Flexible Branding Service...
systemd: Reloading.
systemd: Adding 1h 36min 41.182648s random time.
systemd: Adding 1h 5min 41.218908s random time.
systemd: Started Flexible Branding Service.
systemd: Starting Flexible Branding Service...
systemd: Stopping Gofer Agent...
systemd: goferd.service: main process exited, code=exited, status=1/FAILURE
systemd: Unit goferd.service entered failed state.
systemd: goferd.service failed.
systemd: Started Gofer Agent.
systemd: Starting Gofer Agent...
goferd: [INFO][Thread-1] gofer.rmi.store:108 - Using: /var/lib/gofer/messaging/pending/demo
goferd: [WARNING][MainThread] gofer.agent.plugin:647 - plugin:demo, DISABLED
goferd: [INFO][Thread-2] gofer.rmi.store:108 - Using: /var/lib/gofer/messaging/pending/katello
goferd: [INFO][MainThread] gofer.agent.plugin:692 - plugin:katello loaded using: katello.agent.goferd.plugin
goferd: [INFO][MainThread] rhsm.connection:868 - Connection built: host=<satellite> port=443 handler=/rhsm auth=identity_cert ca_dir=/etc/rhsm/ca/ insecure=False
...
package-cleanup command shows many duplicate rpms.
# package-cleanup --dupes
Loaded plugins: langpacks, product-id, subscription-manager
audit-2.8.1-3.el7_5.1.x86_64
audit-2.8.1-3.el7.x86_64
gofer-2.7.8-1.el7.noarch
gofer-2.12.1-1.el7sat.noarch
audit-libs-2.8.1-3.el7.x86_64
audit-libs-2.8.1-3.el7_5.1.x86_64
audit-libs-2.8.1-3.el7.i686
audit-libs-2.8.1-3.el7_5.1.i686
python-gofer-proton-2.12.1-1.el7sat.noarch
python-gofer-proton-2.7.8-1.el7.noarch
python-pulp-common-2.16.4.1-1.el7sat.noarch
python-pulp-common-2.13.4.16-1.el7sat.noarch
python-gofer-2.7.8-1.el7.noarch
python-gofer-2.12.1-1.el7sat.noarch
Steps to Reproduce:
1. Prepare a host that has some packages to update including gofer rpms and katello-agent
2. Go to the web ui -> content hosts -> <host> -> Packages tab -> Actions -> click "Update all packages" button.
I never realised this is what was causing all my incomplete transactions with Satellite! I see this a lot.
Indeed, when I upgraded from 6.3.5 to 6.4.2, and owing to bad UI design (see: https://bugzilla.redhat.com/show_bug.cgi?id=1678666) accidentally applied all updates to all my servers, I then had to manually go through each and every one separately to unpick the mess yum had made with regard to incomplete transactions. It was a complete nightmare.
I'd appreciate a fix for this very much.
I had the same problem. I narrowed it down to when the gofer package is being updated, and goferd restarts, the update stops before the cleanup, and causes duplicate packages, and not generating iniramfs.
There are 2 workarounds.
1. Update goferd manually from the command line before applying any updates via satellite.
2. Create a custom package that will run a script to update gofer, and push that out via satellite.
I chose to go with the 2nd option, and it worked out fine.
Thanks,
David
Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in a month. This message may be a repeat of a previous update and the bug is again being considered to be closed. If you have any concerns about this, please contact your Red Hat Account team. Thank you.
Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this feel free to contact your Red Hat Account Team. Thank you.
Description of problem: When trigger rpm updates from the Satelite web ui that also update the gofer rpm, the task will stuck forever and cause duplicate rpms and incomplete yum transaction on client side. This is because updating the gofer rpm will cause goferd to restart and cause incomplete yum transaction. /var/log/messages shows that goferd got restarted during the update: goferd: [INFO][worker-0] gofer.rmi.dispatcher:603 - call: Content.update() sn=5ca337fe-8f24-4e0e-83a5-7c6c806f26f9 data={u'task_id': u'782659fb-3480-4431-a90c-7c6b7f642e62', u'consumer_id': u'2e77ff64-07b6-4326-99f7-dc31d23d38cf'} systemd: Reloading. systemd: Current command vanished from the unit file, execution of the command list won't be resumed. systemd: Adding 1h 57min 40.716370s random time. systemd: Adding 3h 52min 30.842836s random time. systemd: Started Flexible Branding Service. systemd: Starting Flexible Branding Service... systemd: Reloading. systemd: Adding 1h 36min 41.182648s random time. systemd: Adding 1h 5min 41.218908s random time. systemd: Started Flexible Branding Service. systemd: Starting Flexible Branding Service... systemd: Stopping Gofer Agent... systemd: goferd.service: main process exited, code=exited, status=1/FAILURE systemd: Unit goferd.service entered failed state. systemd: goferd.service failed. systemd: Started Gofer Agent. systemd: Starting Gofer Agent... goferd: [INFO][Thread-1] gofer.rmi.store:108 - Using: /var/lib/gofer/messaging/pending/demo goferd: [WARNING][MainThread] gofer.agent.plugin:647 - plugin:demo, DISABLED goferd: [INFO][Thread-2] gofer.rmi.store:108 - Using: /var/lib/gofer/messaging/pending/katello goferd: [INFO][MainThread] gofer.agent.plugin:692 - plugin:katello loaded using: katello.agent.goferd.plugin goferd: [INFO][MainThread] rhsm.connection:868 - Connection built: host=<satellite> port=443 handler=/rhsm auth=identity_cert ca_dir=/etc/rhsm/ca/ insecure=False ... package-cleanup command shows many duplicate rpms. # package-cleanup --dupes Loaded plugins: langpacks, product-id, subscription-manager audit-2.8.1-3.el7_5.1.x86_64 audit-2.8.1-3.el7.x86_64 gofer-2.7.8-1.el7.noarch gofer-2.12.1-1.el7sat.noarch audit-libs-2.8.1-3.el7.x86_64 audit-libs-2.8.1-3.el7_5.1.x86_64 audit-libs-2.8.1-3.el7.i686 audit-libs-2.8.1-3.el7_5.1.i686 python-gofer-proton-2.12.1-1.el7sat.noarch python-gofer-proton-2.7.8-1.el7.noarch python-pulp-common-2.16.4.1-1.el7sat.noarch python-pulp-common-2.13.4.16-1.el7sat.noarch python-gofer-2.7.8-1.el7.noarch python-gofer-2.12.1-1.el7sat.noarch Steps to Reproduce: 1. Prepare a host that has some packages to update including gofer rpms and katello-agent 2. Go to the web ui -> content hosts -> <host> -> Packages tab -> Actions -> click "Update all packages" button.