Bug 632277 - Fail more gracefully when qpidd isn't running
Summary: Fail more gracefully when qpidd isn't running
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: z_other
Version: unspecified
Hardware: All
OS: Linux
low
low
Target Milestone: ---
: Sprint 21
Assignee: Jeff Ortel
QA Contact: wes hayutin
URL:
Whiteboard:
Depends On:
Blocks: verified-to-close 673216
TreeView+ depends on / blocked
 
Reported: 2010-09-09 14:58 UTC by Jay Dobies
Modified: 2011-08-16 14:19 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2011-08-16 14:19:51 UTC
Embargoed:


Attachments (Terms of Use)

Description Jay Dobies 2010-09-09 14:58:40 UTC
So far this has bit 4 developers. If qpidd isn't running, everything pulp related just hangs. It manifested itself when running the unit tests (they just hung with no timeout) and when trying to access anything in the API.

We should handle this a little more cleanly, either with a reasonable timeout (I left the unit tests hanging for > 10 minutes) or an error message indicating the bus could not be accessed (and then potentially just skipping bus operations and continuing pulp operations.

Comment 1 Jeff Ortel 2010-10-28 16:05:41 UTC
This is really a problem with how the server logging is set up.  All of the qpid connection details are logged by python-qpid.  On the client this is very clear because the root logger is configured with the client.log appender.  The server log initialization needs to include this as well.

Comment 2 Jeff Ortel 2010-10-28 16:25:58 UTC
Fixed: 0981dd9396304ea0caf5dee834dc17f88c1180b4.

Configured logging for 'qpid' so the pulp.log will show more detail when trying to connect.

Comment 3 Jay Dobies 2010-10-29 16:58:20 UTC
Fixed in 0.77.

Comment 4 Jay Dobies 2010-11-03 19:35:16 UTC
Fixed in build 0.78.

Comment 5 Preethi Thomas 2010-11-10 14:42:26 UTC
fails_qa
The logging is happening in pulp.log and client.log

2010-11-10 08:48:31,908 [INFO][Thread-27] connection() @ endpoint.py:82 - {dcfb402e-987c-4916-bd9a-1801073795a1} connected to AMQP
2010-11-10 08:48:31,908 [WARNING][Thread-27] connect() @ driver.py:492 - trying: localhost:5672
2010-11-10 08:48:31,909 [WARNING][Thread-27] close_engine() @ driver.py:432 - recoverable error[attempt 1257]: [Errno 111] Connection refused
2010-11-10 08:48:31,909 [WARNING][Thread-27] close_engine() @ driver.py:434 - sleeping 120 seconds


But the pulp operation that we are trying done does not time out.
[root@preethi ~]# service qpidd status
qpidd is stopped


[root@preethi ~]#  pulp-admin -u admin -p redhat package install -n zsh --consumerid=preethi.usersys.redhat.com
Created task id: 348ffee8-ecd1-11df-bef1-002564a85a58
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^C

Comment 6 Jeff Ortel 2011-03-11 21:19:43 UTC
Checks consumer availability as follows:

[jortel@localhost pulp]$ pulp-admin package install --consumerid=jortel -n zsh

The following consumers appear to be unavailable.
You may continue with the understanding that your request
will not complete until all consumers have processed the request.

Unavailable Consumers
------------------------------
jortel

Continue? [y/n]:y

Wait? [y/n]:y
Created task id: df813459-4c23-11e0-91b6-0019d21a40be
Task is scheduled for: 2011-03-11 15:09
Waiting: [\] 


Enhanced the waiting graphics.  Unavailable consumer message and prompts only displayed when the agent heartbeat is overdue.

Comment 7 Jeff Ortel 2011-03-11 21:59:40 UTC
The default heartbeat is 10 seconds.  So, when testing, you need to kill goferd or qpidd and wait for 10 seconds before trying the package install.  Also, I updated the 'consumer info' command to show the agent status.

Comment 8 Jeff Ortel 2011-03-12 00:17:27 UTC
Build: 0.151

Comment 9 Jeff Ortel 2011-03-12 00:18:01 UTC
Build: 0.151

Comment 10 Preethi Thomas 2011-05-06 17:23:01 UTC
verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.173-1.fc14.noarch
[root@preethi ~]# 

[root@preethi ~]# pulp-admin package install --n screen --consumerid=10.16.79.198

The following consumers appear to be unavailable.
You may continue with the understanding that your request
will not complete until all consumers have processed the request.

Unavailable Consumers
------------------------------
10.16.79.198

Continue? [y/n]:y

Wait? [y/n]:y
Created task id: ce0bd607-7806-11e0-9179-002564a85a58
Task is scheduled for: 2011-05-06 13:32
Waiting: [\] 
[[['screen-4.1.0-0.3.20101110git066b098.fc15.x86_64'], None]] installed on 10.16.79.198

Comment 11 Preethi Thomas 2011-08-16 14:19:51 UTC
Closing with Community Release 15

pulp-0.0.223-4.


Note You need to log in before you can comment on or make changes to this bug.