Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1457977 - goferd failed to connect to satellite amqps service
Summary: goferd failed to connect to satellite amqps service
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Qpid
Version: 6.3.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: Unspecified
Assignee: Mike Cressman
QA Contact: Katello QA List
URL:
Whiteboard:
Depends On: 1457938
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-01 16:25 UTC by Djebran Lezzoum
Modified: 2021-07-22 07:58 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
The 'hammer host-collection erratum install` installation failed with a sub-task error. With the latest update to qpid, this is now fixed.
Clone Of: 1457938
Environment:
Last Closed: 2018-02-21 12:57:54 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0338 0 normal SHIPPED_LIVE Red Hat Satellite 6.3 Tools 2018-02-21 19:30:02 UTC

Description Djebran Lezzoum 2017-06-01 16:25:56 UTC 
+++ This bug was initially created as a clone of Bug #1457938 +++

Description of problem:
when trying to do hammer host-collection erratum install, installation sub task error returned

Version-Release number of selected component (if applicable):
* candlepin-0.9.54.21-1.el7.noarch
* candlepin-selinux-0.9.54.21-1.el7.noarch
* foreman-1.11.0.76-1.el7sat.noarch
* foreman-compute-1.11.0.76-1.el7sat.noarch
* foreman-debug-1.11.0.76-1.el7sat.noarch
* foreman-discovery-image-3.1.1-23.el7sat.noarch
* foreman-ec2-1.11.0.76-1.el7sat.noarch
* foreman-gce-1.11.0.76-1.el7sat.noarch
* foreman-installer-1.11.0.15-2.el7sat.noarch
* foreman-installer-katello-3.0.0.84-1.el7sat.noarch
* foreman-libvirt-1.11.0.76-1.el7sat.noarch
* foreman-openstack-1.11.0.76-1.el7sat.noarch
* foreman-ovirt-1.11.0.76-1.el7sat.noarch
* foreman-postgresql-1.11.0.76-1.el7sat.noarch
* foreman-proxy-1.11.0.7-1.el7sat.noarch
* foreman-rackspace-1.11.0.76-1.el7sat.noarch
* foreman-selinux-1.11.0.4-1.el7sat.noarch
* foreman-vmware-1.11.0.76-1.el7sat.noarch
* katello-3.0.0-20.el7sat.noarch
* katello-certs-tools-2.4.0-1.el7sat.noarch
* katello-client-bootstrap-1.3.0-1.el7sat.noarch
* katello-common-3.0.0-20.el7sat.noarch
* katello-debug-3.0.0-20.el7sat.noarch
* katello-default-ca-1.0-1.noarch
* katello-installer-base-3.0.0.84-1.el7sat.noarch
* katello-selinux-3.0.1.2-1.el7sat.noarch
* katello-server-ca-1.0-1.noarch
* katello-service-3.0.0-20.el7sat.noarch
* openldap-2.4.40-13.el7.x86_64
* pulp-client-1.0-1.noarch
* pulp-docker-plugins-2.0.3.1-1.el7sat.noarch
* pulp-katello-1.0.2-1.el7sat.noarch
* pulp-ostree-plugins-1.1.3.3-1.el7sat.noarch
* pulp-puppet-plugins-2.8.7.1-1.el7sat.noarch
* pulp-puppet-tools-2.8.7.1-1.el7sat.noarch
* pulp-rpm-plugins-2.8.7.13-1.el7sat.noarch
* pulp-selinux-2.8.7.11-1.el7sat.noarch
* pulp-server-2.8.7.11-1.el7sat.noarch
* python-ldap-2.4.15-2.el7.x86_64
* tfm-rubygem-ldap_fluff-0.4.5-1.el7sat.noarch
* tfm-rubygem-net-ldap-0.13.0-1.el7sat.noarch


How reproducible:
Always

Steps to Reproduce:
1. Setup 2 hosts and a host-collection subscribed to some products containing erratum
2. run:
   hammer host-collection erratum install --id="col-id" --organization-id="org-id" --errata="errata-id"
<<< Message,Id
Successfully scheduled installation of errata,263eea3b-3ac1-4576-9b66-85ce46ec93e9

3. to wait the task to finish run:
    hammer -v -u admin -p changeme  task progress --id="263eea3b-3ac1-4576-9b66-85ce46ec93e9"
     

Actual results:
2 task(s), 0 success, 2 fail

Expected results:
2 task(s), 2 success, 0 fail
and errata successfully installed

Additional info:
1- in the UI Minitor > tasks 
    the tasks error is:
     Host did not respond within 20 seconds. The task has been cancelled. Is katello-agent installed and goferd running on the Host?

2- at one of the virtual machine  /var/log/messages contain error
   Jun  1 10:44:19 139890008902672 goferd: [ERROR][worker-0] gofer.messaging.adapter.connect:33 - connect: proton+amqps://dell-pe-fm120-2a.rhts.eng.bos.redhat.com:5647, failed: Connection amqps://dell-pe-fm120-2a.rhts.eng.bos.redhat.com:5647 disconnected
Jun  1 10:44:19 139890008902672 goferd: [INFO][worker-0] gofer.messaging.adapter.connect:35 - retry in 51 seconds

--- Additional comment from Djebran Lezzoum on 2017-06-01 11:18 EDT ---



--- Additional comment from Djebran Lezzoum on 2017-06-01 11:20 EDT ---



--- Additional comment from Djebran Lezzoum on 2017-06-01 11:22:13 EDT ---

related bugs: 
https://bugzilla.redhat.com/show_bug.cgi?id=1405428
https://bugzilla.redhat.com/show_bug.cgi?id=1418026

--- Additional comment from Djebran Lezzoum on 2017-06-01 11:25:19 EDT ---

full automation test log:
http://pastebin.test.redhat.com/490019

--- Additional comment from RHEL Product and Program Management on 2017-06-01 11:25:51 EDT ---

Since this issue was entered in Red Hat Bugzilla, the pm_ack has been
set to + automatically for the next planned release
Comment 1 Djebran Lezzoum 2017-06-01 16:27:58 UTC 
6.3 
Version-Release number of selected component (if applicable):
* candlepin-2.0.35-1.el7.noarch
* candlepin-selinux-2.0.35-1.el7.noarch
* foreman-1.15.0-1.el7sat.noarch
* foreman-compute-1.15.0-1.el7sat.noarch
* foreman-debug-1.15.0-1.el7sat.noarch
* foreman-discovery-image-3.1.1-22.el7sat.noarch
* foreman-ec2-1.15.0-1.el7sat.noarch
* foreman-gce-1.15.0-1.el7sat.noarch
* foreman-installer-1.15.0-1.el7sat.noarch
* foreman-installer-katello-3.4.0-2.el7sat.noarch
* foreman-libvirt-1.15.0-1.el7sat.noarch
* foreman-openstack-1.15.0-1.el7sat.noarch
* foreman-ovirt-1.15.0-1.el7sat.noarch
* foreman-postgresql-1.15.0-1.el7sat.noarch
* foreman-proxy-1.15.0-2.el7sat.noarch
* foreman-rackspace-1.15.0-1.el7sat.noarch
* foreman-selinux-1.15.0-1.el7sat.noarch
* foreman-vmware-1.15.0-1.el7sat.noarch
* katello-3.4.0-3.el7sat.noarch
* katello-ca-consumer-sat-r220-02.lab.eng.rdu2.redhat.com-1.0-1.noarch
* katello-certs-tools-2.4.0-1.el7sat.noarch
* katello-client-bootstrap-1.3.0-1.el7sat.noarch
* katello-common-3.4.0-3.el7sat.noarch
* katello-debug-3.4.0-3.el7sat.noarch
* katello-default-ca-1.0-1.noarch
* katello-installer-base-3.4.0-2.el7sat.noarch
* katello-selinux-3.0.2-1.el7sat.noarch
* katello-server-ca-1.0-1.noarch
* katello-service-3.4.0-3.el7sat.noarch
* openldap-2.4.40-13.el7.x86_64
* pulp-client-1.0-1.noarch
* pulp-docker-plugins-2.3.0-1.el7sat.noarch
* pulp-katello-1.0.2-1.el7sat.noarch
* pulp-ostree-plugins-1.2.1-1.el7sat.noarch
* pulp-puppet-plugins-2.12.2-1.el7sat.noarch
* pulp-puppet-tools-2.12.2-1.el7sat.noarch
* pulp-rpm-plugins-2.12.2-1.el7sat.noarch
* pulp-selinux-2.12.2-1.el7sat.noarch
* pulp-server-2.12.2-1.el7sat.noarch
* python-ldap-2.4.15-2.el7.x86_64
* tfm-rubygem-ldap_fluff-0.4.6-1.el7sat.noarch
* tfm-rubygem-net-ldap-0.15.0-1.el7sat.noarch
Comment 5 Evgeni Golov 2017-06-22 11:17:04 UTC 
I see similar behaviour in my tests: the connection from goferd to satellite is fine, but the messages are not picked up from the queue:

  pulp.agent.91ab5c1b-9cc9-49e6-9022-495761a518d2                           Y                      6     6      0    3.52k  3.52k       0         0     1

I also tried installing the whole client stack (katello-agent, gofer, proton) on my client VM, but this did not help.
Comment 11 Evgeni Golov 2017-09-14 14:24:01 UTC 
moving to ON_DEV as qpid 1.36 was delivered in latest snap (15) and fixes the issues for me
Comment 12 Roman Plevka 2017-09-20 12:17:15 UTC 
VERIFIED
on sat6.3.0-16


systemd[1]: Started Gofer Agent.
systemd[1]: Starting Gofer Agent...
goferd[4777]: [INFO][Thread-1] gofer.rmi.store:114 - Using: /var/lib/gofer/messaging/pending/demo
goferd[4777]: [WARNING][MainThread] gofer.agent.plugin:639 - plugin:demo, DISABLED
goferd[4777]: [INFO][Thread-2] gofer.rmi.store:114 - Using: /var/lib/gofer/messaging/pending/katelloplugin
goferd[4777]: [INFO][Thread-3] gofer.rmi.store:114 - Using: /var/lib/gofer/messaging/pending/katelloplugin
goferd[4777]: [INFO][MainThread] gofer.agent.plugin:682 - plugin:katelloplugin loaded using: /usr/lib/gofer/plugins/katelloplugin.py
goferd[4777]: [INFO][MainThread] rhsm.connection:830 - Connection built: host=sat-server port=443 handler=/rhsm auth=identity_cert ca_dir=/etc/rhsm/ca/ verify=False
goferd[4777]: [INFO][MainThread] katelloplugin:164 - Using /etc/rhsm/ca/katello-default-ca.pem as the ca cert for qpid connection
goferd[4777]: [INFO][worker-0] gofer.messaging.adapter.connect:28 - connecting: proton+amqps://sat-server:5647
goferd[4777]: [INFO][MainThread] gofer.agent.main:87 - agent started.
goferd[4777]: [INFO][worker-0] gofer.messaging.adapter.proton.connection:87 - open: URL: amqps://sat-server:5647|SSL: ca: /etc/rhsm/ca/katello-default-ca.pem|key: None|certificate: /etc/pki/consumer/bundle.pem|host-validation: None
goferd[4777]: [INFO][worker-0] gofer.messaging.adapter.proton.connection:92 - opened: proton+amqps://sat-server:5647
goferd[4777]: [INFO][worker-0] gofer.messaging.adapter.connect:30 - connected: proton+amqps://sat-server:5647
goferd[4777]: [INFO][worker-0] gofer.messaging.adapter.proton.connection:131 - closed: proton+amqps://sat-server:5647
goferd[4777]: [INFO][pulp.agent.c90cc938-16e7-4bc4-b9f8-39c73bc77cab] gofer.messaging.adapter.connect:28 - connecting: proton+amqps://sat-server:5647
goferd[4777]: [INFO][worker-0] gofer.agent.plugin:368 - plugin:katelloplugin, attached => pulp.agent.c90cc938-16e7-4bc4-b9f8-39c73bc77cab
goferd[4777]: [INFO][pulp.agent.c90cc938-16e7-4bc4-b9f8-39c73bc77cab] gofer.messaging.adapter.proton.connection:87 - open: URL: amqps://sat-server:5647|SSL: ca: /etc/rhsm/ca/katello-default-ca.pem|key: None|certificate: /etc/pki/consumer/bundle.pem|host-validation: None
goferd[4777]: [INFO][pulp.agent.c90cc938-16e7-4bc4-b9f8-39c73bc77cab] gofer.messaging.adapter.proton.connection:92 - opened: proton+amqps://sat-server:5647
goferd[4777]: [INFO][pulp.agent.c90cc938-16e7-4bc4-b9f8-39c73bc77cab] gofer.messaging.adapter.connect:30 - connected: proton+amqps://sat-server:5647


# updating a package

sat-servergoferd[4777]: [INFO][worker-0] gofer.messaging.adapter.connect:28 - connecting: proton+amqps://sat-server:5647
sat-servergoferd[4777]: [INFO][worker-0] gofer.messaging.adapter.proton.connection:87 - open: URL: amqps://sat-server:5647|SSL: ca: /etc/rhsm/ca/katello-default-ca.pem|key: None|certificate: /etc/pki/consumer/bundle.pem|host-validation: None
sat-servergoferd[4777]: [INFO][worker-0] gofer.messaging.adapter.proton.connection:92 - opened: proton+amqps://sat-server:5647
sat-servergoferd[4777]: [INFO][worker-0] gofer.messaging.adapter.connect:30 - connected: proton+amqps://sat-server:5647
sat-servergoferd[4777]: [INFO][worker-0] gofer.rmi.dispatcher:603 - call: Content.update() sn=206e036f-6f18-4b4b-985f-e7004b7e576a data={u'task_id': u'10c459a8-9774-4b9b-87e5-626d11969b02', u'consumer_id': u'c90cc938-16e7-4bc4-b9f8-39c73bc77cab'}
sat-servergoferd[4777]: Loaded plugins: enabled_repos_upload, langpacks, package_upload, product-id,
sat-servergoferd[4777]: : search-disabled-repos, subscription-manager, tracer_upload
sat-servergoferd[4777]: Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
sat-servergoferd[4777]: Running Transaction Check
Sep 20 08:04:00 sat-servergoferd[4777]: Uploading Package Profile
Sep 20 08:04:00 sat-servergoferd[4777]: Uploading Tracer Profile
Sep 20 08:04:03 sat-servergoferd[4777]: [INFO][worker-0] pulp_rpm.handlers.rpmtools:232 - Updated: python-urllib3-1.10.2-3.el7sat.noarch
Sep 20 08:04:03 sat-serveryum[4777]: Updated: python-urllib3-1.10.2-3.el7sat.noarch
Sep 20 08:04:03 sat-servergoferd[4777]: [INFO][worker-0] gofer.agent.rmi:193 - Request: 206e036f-6f18-4b4b-985f-e7004b7e576a, committed
Sep 20 08:04:03 sat-servergoferd[4777]: [INFO][worker-0] gofer.agent.rmi:147 - Request: 206e036f-6f18-4b4b-985f-e7004b7e576a processed in: 8.371 (seconds)
Sep 20 08:04:03 sat-servergoferd[4777]: [INFO][worker-0] gofer.messaging.adapter.proton.connection:131 - closed: proton+amqps://sat-server:5647
Comment 15 errata-xmlrpc 2018-02-21 12:57:54 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0338
Note You need to log in before you can comment on or make changes to this bug.