Description of problem:
This was originally reported upstream but has recently been seen downstream by a number of customers with connectivity issues in their env.
Nova waits indefinitely on ceph client hangs due to network problems
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Deploy environment.
2. Later block access to the rbd cluster.
n-cpu will wait indefinitely while attempting to connect.
n-cpu should fail and mark the host as DOWN until connectivity is re-established.