Created attachment 902254 [details] journalctl of the whole test case including all pulp server logs Description of problem: Although wrong signature is detected in gofer.transport.consumer handler, related task is not terminated with error. Rather, the task is reported as pending. Version-Release number of selected component (if applicable): pulp-server-2.4.0-0.19.beta.fc20.noarch How reproducible: always pulp-automation, test case: tests.test_9_consumer_auth Steps to Reproduce: 1. ensure authentication=True 2. create rpm repo 3. register consumer with rsa primary's pub key 4. launch consumer agent with another rsa 5. bind consumer agent to the repo distributor Actual results: auth failure detected by gofer.transport.consumer isn't propagated to related async task Expected results: if auth failure is detected by gofer.transport.consumer then related async task is terminated with appropriate error attribute Additional info: # journalctl log entry gofer.transport.consumer:DEBUG: security.authentication, reason: None {"any": {"action": "bind", "consumer_id": "ConsumerAuthTest_consumer", "distributor_id": "zoo_distributor", "repo_id": "zoo", "task_id": "bb9c9824-33f7-4551-9a2a-d7e3d81ab4f6"}, "pam": null, "replyto": {"exchange": "pulp.agent.ConsumerAuthTest_consumer", "routing_key": "pulp.task"}, "routing": ["", "b0762bb4-a202-4358-984e-ab782fdef865"], "secret": "538f4df6bcb62b512ab4101a", "sn": "18dc9358-86c6-46d0-a8f8-1f70025aac0d", "status": "progress", "version": "0.5", "window": {}} # related pulp tasks status [root@ec2-79-125-97-56 ~]# pulp-admin -u admin -p admin tasks details --task-id bb9c9824-33f7-4551-9a2a-d7e3d81ab4f6 +----------------------------------------------------------------------+ Task Details +----------------------------------------------------------------------+ Operations: agent_bind Resources: ConsumerAuthTest_consumer (consumer), zoo (repository), zoo_distributor (repository_distributor) State: Waiting Start Time: Unstarted Finish Time: Incomplete Result: Incomplete Task Id: bb9c9824-33f7-4551-9a2a-d7e3d81ab4f6 Progress Report:
We need to make sure that both ends log the error properly.
Trying to reproduce: [jortel@localhost server]$ pulp-admin rpm consumer package install run --consumer-id=demo -n zsh Install task created with id [ 1b9d8499-7950-49c3-9a98-9f79ae015b02 ] This command may be exited via ctrl+c without affecting the request. [|] Waiting to begin... Install Failed [jortel@localhost server]$ pulp-admin tasks Usage: pulp-admin [SUB_SECTION, ..] COMMAND Description: list and cancel server-side tasks Available Commands: cancel - cancel one or more tasks details - displays more detailed information about a specific task list - lists tasks queued or running in the server [jortel@localhost server]$ pulp-admin tasks details --task-id=1b9d8499-7950-49c3-9a98-9f79ae015b02 +----------------------------------------------------------------------+ Task Details +----------------------------------------------------------------------+ Operations: unit_install Resources: demo (consumer) State: Failed Start Time: Unstarted Finish Time: 2014-06-10T19:26:09Z Result: N/A Task Id: 1b9d8499-7950-49c3-9a98-9f79ae015b02 Progress Report: [jortel@localhost server]$ pulp-admin repo list +----------------------------------------------------------------------+ Repositories +----------------------------------------------------------------------+ Id: open-stack Display Name: open-stack Description: None Content Unit Counts: Id: empty Display Name: empty Description: None Content Unit Counts: [jortel@localhost server]$ pulp-admin rpm consumer bind --repo-id=open-stack --consumer-id=demo This command may be exited via ctrl+c without affecting the request. [-] Waiting to begin... [-] Running... Task Failed [jortel@localhost server]$ pulp-admin tasks details --task-id='caa97b2f-7e89-4ec7-9f3b-0abef3b2db64' +----------------------------------------------------------------------+ Task Details +----------------------------------------------------------------------+ Operations: agent_bind Resources: demo (consumer), open-stack (repository), yum_distributor (repository_distributor) State: Failed Start Time: Unstarted Finish Time: 2014-06-10T19:28:05Z Result: N/A Task Id: caa97b2f-7e89-4ec7-9f3b-0abef3b2db64 Progress Report: ========================================== SERVER: log ========================================== Jun 10 14:24:34 localhost pulp: pulp.server.agent.direct.services:INFO: Task RMI (rejected) Jun 10 14:24:34 localhost pulp: pulp.server.agent.direct.services:INFO: Rejected Jun 10 14:24:34 localhost pulp: pulp.server.agent.direct.services:INFO: sn : 7f067078-d5a2-4c7c-b6ac-89e58dcc1b60 Jun 10 14:24:34 localhost pulp: pulp.server.agent.direct.services:INFO: origin : 69b3b11f-23ac-468d-ba16-d0c3bb8ced43 Jun 10 14:24:34 localhost pulp: pulp.server.agent.direct.services:INFO: user data : {'task_id': '1b9d8499-7950-49c3-9a98-9f79ae015b02', 'consumer_id': 'demo'} Jun 10 14:24:34 localhost pulp: pulp.server.agent.direct.services:INFO: rejected Jun 10 14:27:47 localhost pulp: gofer.transport.amqplib.broker:INFO: connecting: {localhost:5673}:|transport=AMQP|host=localhost|port=5673|cacert=/etc/pki/qpid/ca/ca.crt|clientcert=/etc/pki/qpid/client/client.pem|userid=guest|password=guest|vhost=/ Jun 10 14:27:47 localhost pulp: gofer.transport.amqplib.broker:INFO: {localhost:5673} connected to AMQP Jun 10 14:27:48 localhost goferd: [INFO][pulp.agent.demo] gofer.messaging.auth:138 - no start line Jun 10 14:28:01 localhost pulp: pulp.server.agent.direct.services:INFO: Task RMI (rejected) Jun 10 14:28:01 localhost pulp: pulp.server.agent.direct.services:INFO: Rejected Jun 10 14:28:01 localhost pulp: pulp.server.agent.direct.services:INFO: sn : ac8b7936-4664-408c-a80a-a3d80dc18d5a Jun 10 14:28:01 localhost pulp: pulp.server.agent.direct.services:INFO: origin : 69b3b11f-23ac-468d-ba16-d0c3bb8ced43 Jun 10 14:28:01 localhost pulp: pulp.server.agent.direct.services:INFO: user data : {'action': 'bind', 'repo_id': 'open-stack', 'distributor_id': 'yum_distributor', 'task_id': 'caa97b2f-7e89-4ec7-9f3b-0abef3b2db64', 'consumer_id': 'demo'} Jun 10 14:28:01 localhost pulp: pulp.server.agent.direct.services:INFO: rejected
Milan, I replaced the RSA key of my consumer with another key and tried to issue some requests. An install followed by a bind. In both cases, the rejection was logged on the server and the task state=Failed. Perhaps, I'm missing something but so far unable to reproduce. Any suggestions? -jeff
Created attachment 913827 [details] reproducer screen log Actually, I'm able to reproduce it. See the attachment for the screen log. The steps: - register a consumer - unregister a consumer - copy the keys from /etc/pki/pulp/consumer/ to a temporary directory - register a new consumer from the same node - change the configuration in /etc/pulp/consumer/consumer.conf:authentication.rsa_* to point to the temporary certificate directory in order to use previous consumer's rsa keys - restart goferd - pulp-consumer rpm bind --repo-id <an existing repo> - a task is created that will never finish
I'm sorry it took so long to respond, I was occupied by other tasks. Setting back to assigned based on Comment #4. Please, review. Thanks, milan
IMHO, this works as designed. The server will discard replies (messages) coming from an agent when the message signature fails validation. I don't think we want to be updating task state based on messages that fail validation.