Description of problem: https://issues.apache.org/jira/browse/PROTON-1394 is missing from the build available in EPEL7 causing goferd to eventually fail with the error in the additional info section below. Version-Release number of selected component (if applicable): qpid-proton-c-0.17.0-4.el7.x86_64 How reproducible: goferd running while katello is offline or otherwise unavailable Steps to Reproduce: 1. install katello-agent on a subscribed host 2. wait with katello offline or otherwise unavailable 3. systemctl status goferd OR lsof -n -p $pid_of_goferd Actual results: goferd unable to connect Expected results: goferd functions normally Additional info: Oct 19 10:44:02 foo.example.com goferd[1420]: [INFO][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.connect:28 - connecting: proton+amqps://katello.example.com:5647 Oct 19 10:44:02 foo.example.com goferd[1420]: [INFO][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.proton.connection:87 - open: URL: amqps://katello.example.com:5647|SSL: ca: /etc/rhsm/ca/katello-default-ca.pem|key: None|certificate: /etc/pki/consumer/bundle.pem|host-validation: None Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - Wrapper failed to create wrapped object. Check for file descriptor or memory exhaustion. Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - Traceback (most recent call last): Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/model.py", line 39, in _fn Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - return fn(*args, **keywords) Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/model.py", line 593, in open Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - self._impl.open() Oct 19 10:44:02 foo.example.com goferd[1420]: Exception KeyError: ('_attrs',) in ignored Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/proton/reliability.py", line 44, in _fn Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - return fn(messenger, *args, **kwargs) Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/proton/consumer.py", line 72, in open Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - self.connection.open() Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/connect.py", line 29, in inner Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - impl = fn(connection) Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/proton/connection.py", line 91, in open Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - ssl_domain=domain) Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib64/python2.7/site-packages/proton/utils.py", line 207, in __init__ Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - self.container = container or Container() Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib64/python2.7/site-packages/proton/reactor.py", line 655, in __init__ Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - super(Container, self).__init__(*handlers, **kwargs) Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib64/python2.7/site-packages/proton/reactor.py", line 85, in __init__ Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - Wrapper.__init__(self, kwargs.get("impl", pn_reactor), pn_reactor_attachments) Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - File "/usr/lib64/python2.7/site-packages/proton/wrapper.py", line 43, in __init__ Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - raise ProtonException("Wrapper failed to create wrapped object. Check for file descriptor or memory exhaustion.") Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.adapter.model:43 - ProtonException: Wrapper failed to create wrapped object. Check for file descriptor or memory exhaustion. Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.consumer:74 - pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.consumer:74 - Traceback (most recent call last): Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.consumer:74 - File "/usr/lib/python2.7/site-packages/gofer/messaging/consumer.py", line 71, in open Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.consumer:74 - self.reader.open() Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.consumer:74 - File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/model.py", line 44, in _fn Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.consumer:74 - raise ModelError(*e.args) Oct 19 10:44:02 foo.example.com goferd[1420]: [ERROR][pulp.agent.1a53ca26-2e5f-47da-9954-b0036ce6c93c] gofer.messaging.consumer:74 - ModelError: Wrapper failed to create wrapped object. Check for file descriptor or memory exhaustion.
Can confirm same behaviour/problem in another project that was fixed by updating to 0.18 upstream version of qpid-proton
qpid-proton-0.18.1-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-f4b0a3b0ea
qpid-proton-0.18.1-1.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-f4b0a3b0ea
I will note that the update was posted for F27 - but this bug is for EPEL. Not saying F27 is not affected but the update in F27 will be irrelevant for people affected by the bug in EPEL.
qpid-proton-0.18.1-1.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-2f3343f663
qpid-proton-0.18.1-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-2f3343f663
qpid-proton-0.18.1-1.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.
qpid-proton-0.18.1-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.