Bug 632277

Summary: Fail more gracefully when qpidd isn't running
Product: [Retired] Pulp Reporter: Jay Dobies <jason.dobies>
Component: z_otherAssignee: Jeff Ortel <jortel>
Status: CLOSED CURRENTRELEASE QA Contact: wes hayutin <whayutin>
Severity: low Docs Contact:
Priority: low    
Version: unspecifiedCC: pthomas, skarmark, tsanders
Target Milestone: ---Keywords: Triaged
Target Release: Sprint 21   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-08-16 14:19:51 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 647488, 673216    

Description Jay Dobies 2010-09-09 14:58:40 UTC
So far this has bit 4 developers. If qpidd isn't running, everything pulp related just hangs. It manifested itself when running the unit tests (they just hung with no timeout) and when trying to access anything in the API.

We should handle this a little more cleanly, either with a reasonable timeout (I left the unit tests hanging for > 10 minutes) or an error message indicating the bus could not be accessed (and then potentially just skipping bus operations and continuing pulp operations.

Comment 1 Jeff Ortel 2010-10-28 16:05:41 UTC
This is really a problem with how the server logging is set up.  All of the qpid connection details are logged by python-qpid.  On the client this is very clear because the root logger is configured with the client.log appender.  The server log initialization needs to include this as well.

Comment 2 Jeff Ortel 2010-10-28 16:25:58 UTC
Fixed: 0981dd9396304ea0caf5dee834dc17f88c1180b4.

Configured logging for 'qpid' so the pulp.log will show more detail when trying to connect.

Comment 3 Jay Dobies 2010-10-29 16:58:20 UTC
Fixed in 0.77.

Comment 4 Jay Dobies 2010-11-03 19:35:16 UTC
Fixed in build 0.78.

Comment 5 Preethi Thomas 2010-11-10 14:42:26 UTC
fails_qa
The logging is happening in pulp.log and client.log

2010-11-10 08:48:31,908 [INFO][Thread-27] connection() @ endpoint.py:82 - {dcfb402e-987c-4916-bd9a-1801073795a1} connected to AMQP
2010-11-10 08:48:31,908 [WARNING][Thread-27] connect() @ driver.py:492 - trying: localhost:5672
2010-11-10 08:48:31,909 [WARNING][Thread-27] close_engine() @ driver.py:432 - recoverable error[attempt 1257]: [Errno 111] Connection refused
2010-11-10 08:48:31,909 [WARNING][Thread-27] close_engine() @ driver.py:434 - sleeping 120 seconds


But the pulp operation that we are trying done does not time out.
[root@preethi ~]# service qpidd status
qpidd is stopped


[root@preethi ~]#  pulp-admin -u admin -p redhat package install -n zsh --consumerid=preethi.usersys.redhat.com
Created task id: 348ffee8-ecd1-11df-bef1-002564a85a58
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^C

Comment 6 Jeff Ortel 2011-03-11 21:19:43 UTC
Checks consumer availability as follows:

[jortel@localhost pulp]$ pulp-admin package install --consumerid=jortel -n zsh

The following consumers appear to be unavailable.
You may continue with the understanding that your request
will not complete until all consumers have processed the request.

Unavailable Consumers
------------------------------
jortel

Continue? [y/n]:y

Wait? [y/n]:y
Created task id: df813459-4c23-11e0-91b6-0019d21a40be
Task is scheduled for: 2011-03-11 15:09
Waiting: [\] 


Enhanced the waiting graphics.  Unavailable consumer message and prompts only displayed when the agent heartbeat is overdue.

Comment 7 Jeff Ortel 2011-03-11 21:59:40 UTC
The default heartbeat is 10 seconds.  So, when testing, you need to kill goferd or qpidd and wait for 10 seconds before trying the package install.  Also, I updated the 'consumer info' command to show the agent status.

Comment 8 Jeff Ortel 2011-03-12 00:17:27 UTC
Build: 0.151

Comment 9 Jeff Ortel 2011-03-12 00:18:01 UTC
Build: 0.151

Comment 10 Preethi Thomas 2011-05-06 17:23:01 UTC
verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.173-1.fc14.noarch
[root@preethi ~]# 

[root@preethi ~]# pulp-admin package install --n screen --consumerid=10.16.79.198

The following consumers appear to be unavailable.
You may continue with the understanding that your request
will not complete until all consumers have processed the request.

Unavailable Consumers
------------------------------
10.16.79.198

Continue? [y/n]:y

Wait? [y/n]:y
Created task id: ce0bd607-7806-11e0-9179-002564a85a58
Task is scheduled for: 2011-05-06 13:32
Waiting: [\] 
[[['screen-4.1.0-0.3.20101110git066b098.fc15.x86_64'], None]] installed on 10.16.79.198

Comment 11 Preethi Thomas 2011-08-16 14:19:51 UTC
Closing with Community Release 15

pulp-0.0.223-4.