Bug 1130019 - pulp agent goferd ignores /etc/pulp/consumer/conf.d/*.conf files, broken config-parsing
Summary: pulp agent goferd ignores /etc/pulp/consumer/conf.d/*.conf files, broken conf...
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: async/tasks
Version: 2.4.0
Hardware: All
OS: Linux
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: pulp-bugs
QA Contact: pulp-qe-list
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-08-14 07:56 UTC by Moritz Rogalli
Modified: 2014-08-20 03:31 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-08-14 14:56:59 UTC
Embargoed:


Attachments (Terms of Use)

Description Moritz Rogalli 2014-08-14 07:56:27 UTC
Description of problem:

The agent (goferd) does not pick up the correct hostname for the server when using /etc/pulp/consumer/conf.d/*.conf to overwrite values in /etc/pulp/consumer/consumer.conf.

Parsing of the value in /etc/pulp/consumer/consumer.conf is broken. goferd picks up comments in the host directive in config file and treats them as part of the host address.

Version-Release number of selected component (if applicable):

2.4

How reproducible:

Always

Steps to Reproduce:
1. yum groupinstall pulp-consumer-qpid
2. echo -e "[server]\nhost = vanilla.intnet\nverify_ssl = False" > /etc/pulp/consumer/conf.d/server.conf
3. service goferd start

Actual results:

tail /var/log/messages:

Aug 13 16:38:12 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [WARNING][Thread-2] qpid.messaging:527 - trying: localhost.localdomain:5672
Aug 13 16:38:12 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [WARNING][Thread-2] qpid.messaging:448 - recoverable error[attempt 2]: [Errno 111] Connection refused
Aug 13 16:38:11 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [WARNING][MainThread] qpid.messaging:450 - sleeping 1 seconds
Aug 13 16:38:11 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [ERROR][Actions] gofer.agent.action:131 - [Errno 111] Connection refused
Aug 13 16:38:11 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [ERROR][Actions] gofer.agent.action:131 - Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/gofer/agent/action.py", line 129, in __call__ self.target() File "/usr/lib64/gofer/plugins/pulpplugin.py", line 287, in update_profile profile.send() File "/usr/lib64/gofer/plugins/pulpplugin.py", line 437, in send http = bindings.profile.send(consumer_id, type_id, details) File "/usr/lib/python2.6/site-packages/pulp/bindings/consumer.py", line 187, in send return self.server.POST(path, data) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 99, in POST return self._request('POST', path, body=body, ensure_encoding=ensure_encoding) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 143, in _request response_code, response_body = self.server_wrapper.request(method, url, body) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 316, in request connection.request(method, url, body=body, headers=headers) File "/usr/lib64/python2.6/httplib.py", line 914, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request self.endheaders() File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders self._send_output() File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output self.send(msg) File "/usr/lib64/python2.6/httplib.py", line 739, in send self.connect() File "/usr/lib64/python2.6/site-packages/M2Crypto/httpslib.py", line 50, in connect self.sock.connect((self.host, self.port)) File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 181, in connect self.socket.connect(addr) File "<string>", line 1, in connect error: [Errno 111] Connection refused
Aug 13 16:38:12 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [WARNING][Thread-2] qpid.messaging:527 - trying: localhost.localdomain:5672
Aug 13 16:38:12 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [WARNING][Thread-2] qpid.messaging:448 - recoverable error[attempt 2]: [Errno 111] Connection refused

Expected results:

Aug 13 16:43:38 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.agent.plugin:521 - plugin [builtin] loaded using: /usr/share/gofer/plugins/builtin.py
Aug 13 16:43:38 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.agent.plugin:521 - plugin [pulpplugin] loaded using: /usr/lib64/gofer/plugins/pulpplugin.py
Aug 13 16:43:38 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] pulpplugin:87 - plugin configuration updated
Aug 13 16:43:38 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][Actions] builtin:38 - Hello:  [main] enabled=1  [messaging] url= uuid=admin
Aug 13 16:43:38 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.transport.qpid.broker:69 - connecting: {vanilla.intnet:5672}:|transport=TCP|host=vanilla.intnet|port=5672|cacert=None|clientcert=/etc/pki/pulp/consumer/consumer-cert.pem|userid=None|password=None|vhost=None
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.transport.qpid.broker:83 - {vanilla.intnet:5672} connected to AMQP
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][pulp.agent.consumer] gofer.transport.qpid.broker:69 - connecting: {vanilla.intnet:5672}:|transport=TCP|host=vanilla.intnet|port=5672|cacert=None|clientcert=/etc/pki/pulp/consumer/consumer-cert.pem|userid=None|password=None|vhost=None
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.agent.plugin:275 - plugin uuid="pulp.agent.consumer", attached
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.agent.main:136 - agent started.
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [ERROR][Actions] gofer.agent.action:131 - 
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [ERROR][Actions] gofer.agent.action:131 - Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/gofer/agent/action.py", line 129, in __call__ self.target() File "/usr/lib64/gofer/plugins/pulpplugin.py", line 287, in update_profile profile.send() File "/usr/lib64/gofer/plugins/pulpplugin.py", line 437, in send http = bindings.profile.send(consumer_id, type_id, details) File "/usr/lib/python2.6/site-packages/pulp/bindings/consumer.py", line 187, in send return self.server.POST(path, data) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 99, in POST return self._request('POST', path, body=body, ensure_encoding=ensure_encoding) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 143, in _request response_code, response_body = self.server_wrapper.request(method, url, body) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 324, in request raise exceptions.CertificateVerificationException() CertificateVerificationException
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][pulp.agent.consumer] gofer.transport.qpid.broker:83 - {vanilla.intnet:5672} connected to AMQP
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][worker-0] gofer.transport.qpid.broker:69 - connecting: {vanilla.intnet:5672}:|transport=TCP|host=vanilla.intnet|port=5672|cacert=None|clientcert=/etc/pki/pulp/consumer/consumer-cert.pem|userid=None|password=None|vhost=None
Aug 13 16:43:39 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][worker-0] gofer.transport.qpid.broker:83 - {vanilla.intnet:5672} connected to AMQP


Additional info:

Changing /etc/pulp/consumer/consumer.conf directly has unexpected effects as well.

Changing the host = localhost.localdomain directive to

host = vanilla.intnet #localhost.localdomain

results in this:

Aug 13 16:42:35 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.agent.plugin:521 - plugin [builtin] loaded using: /usr/share/gofer/plugins/builtin.py
Aug 13 16:42:35 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.agent.plugin:521 - plugin [pulpplugin] loaded using: /usr/lib64/gofer/plugins/pulpplugin.py
Aug 13 16:42:36 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] pulpplugin:87 - plugin configuration updated
Aug 13 16:42:36 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][Actions] builtin:38 - Hello:  [main] enabled=1  [messaging] url= uuid=admin
Aug 13 16:42:36 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [INFO][MainThread] gofer.transport.qpid.broker:69 - connecting: {vanilla.intnet #localhost.localdomain:5672}:|transport=TCP|host=vanilla.intnet #localhost.localdomain|port=5672|cacert=None|clientcert=/etc/pki/pulp/consumer/consumer-cert.pem|userid=None|password=None|vhost=None
Aug 13 16:42:36 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [WARNING][MainThread] qpid.messaging:448 - recoverable error[attempt 1]: [Errno -2] Name or service not known
Aug 13 16:42:36 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [WARNING][MainThread] qpid.messaging:450 - sleeping 1 seconds
Aug 13 16:42:36 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [ERROR][Actions] gofer.agent.action:131 - [Errno -2] Name or service not known
Aug 13 16:42:36 VM-e2ca25c3-874d-47ad-8b42-b876726d1fec goferd: [ERROR][Actions] gofer.agent.action:131 - Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/gofer/agent/action.py", line 129, in __call__ self.target() File "/usr/lib64/gofer/plugins/pulpplugin.py", line 287, in update_profile profile.send() File "/usr/lib64/gofer/plugins/pulpplugin.py", line 437, in send http = bindings.profile.send(consumer_id, type_id, details) File "/usr/lib/python2.6/site-packages/pulp/bindings/consumer.py", line 187, in send return self.server.POST(path, data) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 99, in POST return self._request('POST', path, body=body, ensure_encoding=ensure_encoding) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 143, in _request response_code, response_body = self.server_wrapper.request(method, url, body) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 316, in request connection.request(method, url, body=body, headers=headers) File "/usr/lib64/python2.6/httplib.py", line 914, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request self.endheaders() File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders self._send_output() File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output self.send(msg) File "/usr/lib64/python2.6/httplib.py", line 739, in send self.connect() File "/usr/lib64/python2.6/site-packages/M2Crypto/httpslib.py", line 50, in connect self.sock.connect((self.host, self.port)) File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 181, in connect self.socket.connect(addr) File "<string>", line 1, in connect gaierror: [Errno -2] Name or service not known

Comment 1 Jeff Ortel 2014-08-14 14:45:30 UTC
Thanks for the report.

Although, INI is an informal specification, it seems that only lines beginning with (;|#) are to be treated as comments.  In the case of the pulp agent (which is a plugin running within goferd) uses a pulp function to read the consumer.conf.  This function in pulp.common.config uses iniparse to parse the configuration file.  If you still believe that comments at the end of lines should be ignored, I would be more appropriate to file a bug against iniparse.

http://en.wikipedia.org/wiki/INI_file

As for ignoring the overrides in consumer/conf.d/* files -- This is also handled by the pulp function used to read consumer.conf and has already been fixed in 2.4.1.

Which exact version of pulp are you running?

Comment 2 Moritz Rogalli 2014-08-14 14:56:59 UTC
Alright!

Good to now, we use 2.4.0 as this is the latest stable version. We will just have to work around the bug until the 2.4.1 release then I guess.

Thank you for your quick reply!


Note You need to log in before you can comment on or make changes to this bug.