Bug 1129584

Summary: Repository sync for Pulp node is not working
Product: [Retired] Pulp Reporter: Dhaval <dhaval.joshi>
Component: nodesAssignee: pulp-bugs
Status: CLOSED INSUFFICIENT_DATA QA Contact: pulp-qe-list
Severity: medium Docs Contact:
Priority: unspecified    
Version: 2.4 BetaCC: bmbouter, dhaval.joshi, jortel, skarmark
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-27 17:52:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dhaval 2014-08-13 09:16:17 UTC
Description of problem:

pulp-admin -u admin -p admin node sync run --node-id <node name> gives errors


Version-Release number of selected component (if applicable):


How reproducible:

Configure Pulp on 2 Nodes
make one parent and other child follow ( http://pulp-user-guide.readthedocs.org/en/pulp-2.3/nodes.html )  ( with SSL option false )

run repository sync




Steps to Reproduce:
1.
2.
3.

Actual results:

pulp-admin -u admin -p admin node sync run --node-id <nodeid>

Error occurred during synchronization, check the child node logs for details

+----------------------------------------------------------------------+
                       Child Node Synchronization
+----------------------------------------------------------------------+

Repository:
  Action:          Pending
  Content Sources:
    Downloads:
  Id:              osbuild
  Units:
    Added:   0
    Removed: 0
    Updated: 0


+----------------------------------------------------------------------+
                 The following [1] errors were reported
+----------------------------------------------------------------------+

- 01: An unexpected error occurred.  repo_id=None




Expected results:


Additional info:

Messages on Parent Node :- 

[root@pulp-parent osbuild]# pulp-admin -u admin -p admin node sync run --node-id pulp-child
Aug 13 10:09:20 pulp-parent pulp: gofer.transport.qpid.broker:INFO: connecting: {localhost:5672}:|transport=TCP|host=localhost|port=5672|cacert=/etc/pki/qpid/ca/ca.crt|clientcert=/etc/pki/qpid/client/client.pem|userid=None|password=None|vhost=None
Aug 13 10:09:20 pulp-parent pulp: gofer.transport.qpid.broker:INFO: {localhost:5672} connected to AMQP
This command may be exited via ctrl+c without affecting the request.


Aug 13 10:09:20 pulp-parent pulp: pulp.server.agent.direct.services:INFO: Task RMI (succeeded)
Aug 13 10:09:20 pulp-parent pulp: pulp.server.agent.direct.services:INFO: Succeeded
Aug 13 10:09:20 pulp-parent pulp: pulp.server.agent.direct.services:INFO:   sn : f81f7cc5-1029-4fec-b16a-f849aaa50aee
Aug 13 10:09:20 pulp-parent pulp: pulp.server.agent.direct.services:INFO:   origin : 44719e99-1c09-4485-a459-2ae7fa0976d9
Aug 13 10:09:20 pulp-parent pulp: pulp.server.agent.direct.services:INFO:   user data : {'task_id': 'e002d68d-6c48-4aa5-862f-c66e11a348f1', 'consumer_id': 'pulp-child'}
Aug 13 10:09:20 pulp-parent pulp: pulp.server.agent.direct.services:INFO:   retval:
Aug 13 10:09:20 pulp-parent pulp: pulp.server.agent.direct.services:INFO: {'reboot': {'scheduled': False, 'details': {}}, 'details': {'node': {'details': {'errors': [{'error_id': 'exception', 'details': {'message': 'Peer certificate commonName does not match host, expected pulp-child, got pulp-parent', 'repo_id': None}}], 'repositories': [{'action': 'pending', 'sources': {}, 'repo_id': 'osbuild', 'units': {'removed': 0, 'added': 0, 'updated': 0}}]}, 'succeeded': False}}, 'succeeded': False, 'num_changes': 0}


Error Messages on CHILD NODE :-

Aug 13 10:09:20 pulp-child goferd: [INFO][worker-0] gofer.transport.qpid.broker:69 - connecting: {pulp-parent:5672}:|transport=TCP|host=pulp-parent|port=5672|cacert=None|clientcert=/etc/pki/pulp/consumer/consumer-cert.pem|userid=None|password=None|vhost=None
Aug 13 10:09:20 pulp-child goferd: [INFO][worker-0] gofer.transport.qpid.broker:83 - {pulp-parent:5672} connected to AMQP
Aug 13 10:09:20 pulp-child goferd: [INFO][worker-0] gofer.rmi.dispatcher:612 - call: Content.update() sn=f81f7cc5-1029-4fec-b16a-f849aaa50aee info={'task_id': 'e002d68d-6c48-4aa5-862f-c66e11a348f1', 'consumer_id': 'pulp-child'}
Aug 13 10:09:20 pulp-child goferd: [ERROR][worker-0] pulp_node.handlers.strategies:129 - synchronization failed
Aug 13 10:09:20 pulp-child goferd: [ERROR][worker-0] pulp_node.handlers.strategies:129 - Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pulp_node/handlers/strategies.py", line 116, in synchronize validator.validate(request.bindings) File "/usr/lib/python2.6/site-packages/pulp_node/handlers/validation.py", line 54, in validate self.report.errors.extend(self._validate_plugins(bindings)) File "/usr/lib/python2.6/site-packages/pulp_node/handlers/validation.py", line 70, in _validate_plugins child = ChildServer() File "/usr/lib/python2.6/site-packages/pulp_node/handlers/validation.py", line 88, in __init__ self.importers = self._importers() File "/usr/lib/python2.6/site-packages/pulp_node/handlers/validation.py", line 99, in _importers http = bindings.server_info.get_importers() File "/usr/lib/python2.6/site-packages/pulp/bindings/server_info.py", line 43, in get_importers return self.server.GET(path) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 93, in GET return self._request('GET', path, queries) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 143, in _request response_code, response_body = self.server_wrapper.request(method, url, body) File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 316, in request connection.request(method, url, body=body, headers=headers) File "/usr/lib64/python2.6/httplib.py", line 914, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request self.endheaders() File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders self._send_output() File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output self.send(msg) File "/usr/lib64/python2.6/httplib.py", line 739, in send self.connect() File "/usr/lib64/python2.6/site-packages/M2Crypto/httpslib.py", line 50, in connect self.sock.connect((self.host, self.port)) File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 188, in connect if not check(self.get_peer_cert(), self.addr[0]): F
Aug 13 10:09:20 pulp-child goferd: [ERROR][worker-0] pulp_node.handlers.handler:111 - An unexpected error occurred.  repo_id=None
Aug 13 10:09:20 pulp-child goferd: [INFO][worker-0] gofer.agent.rmi:149 - sn=f81f7cc5-1029-4fec-b16a-f849aaa50aee processed in: 170 (ms)

Comment 1 Jeff Ortel 2014-08-13 15:28:38 UTC
Looks like an SSL validation issue on the child node between the node handler in the agent (on the child) and the local pulp server.  Do you have 'verify_ssl: False' set in /etc/pulp/nodes.conf?  If not, please do so, restart goferd, and retry the sync.

Comment 2 Brian Bouterse 2014-08-27 17:52:04 UTC
This bug has been open for a while and marked as needs info. I'm going to close the issue, but please reopen it if you have more info in response to comment 1.

Comment 3 Dhaval 2014-09-15 11:08:41 UTC
please close the issue, I need to work further to re-produce it ..