Bug 1379130 - Unexpected client exception when glance server is not reachable
Summary: Unexpected client exception when glance server is not reachable
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: ---
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ovirt-4.1.1
: 4.1.1
Assignee: Benny Zlotnik
QA Contact: Elad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-25 09:48 UTC by Yaniv Kaul
Modified: 2017-04-21 09:36 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-04-21 09:36:27 UTC
oVirt Team: Storage
Embargoed:
rule-engine: ovirt-4.1+
rule-engine: planning_ack+
rule-engine: devel_ack+
ratamir: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 70969 0 master MERGED core: catch exception coming from the OpenStack client 2020-02-03 06:33:21 UTC
oVirt gerrit 71707 0 ovirt-engine-4.1 MERGED core: catch exception coming from the OpenStack client 2020-02-03 06:33:22 UTC

Description Yaniv Kaul 2016-09-25 09:48:03 UTC
Description of problem:
When setting glance.ovirt.org address to 127.0.0.1 and trying to reach it and list images from it, I'm getting the following stack:
  File "/home/ykaul/ovirt-system-tests/basic_suite_master/test-scenarios/002_bootstrap.py", line 392, in list_glance_images
    all_images = glance_provider.images.list()
  File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 26117, in list
    headers={}
  File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 46, in get
    return self.request(method='GET', url=url, headers=headers, cls=cls)
  File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request
    persistent_auth=self.__persistent_auth
  File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request
    persistent_auth)
  File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request
    raise errors.RequestError(response_code, response_reason, response_body)
RequestError: 
status: 400
reason: Bad Request
detail: unexpected client exception


I'd expect that exception to be caught and handled by the SDK and return an empty list, since this is not a server error - we never reached the server.



Version-Release number of selected component (if applicable):
ovirt-engine-sdk-python-3.6.9.1-1.fc24.noarch

How reproducible:
Always

Comment 1 Yaniv Kaul 2016-10-25 09:46:29 UTC
Adding AutomationBlocker flag, as it affects CI (see http://jenkins.ovirt.org/job/ovirt_master_system-tests/692/testReport/junit/(root)/002_bootstrap/list_glance_images/ for example).

Comment 2 Juan Hernández 2016-10-25 10:10:35 UTC
The relevant error message from engine.log is the following:

Caused by: java.net.UnknownHostException: glance.ovirt.org: Name or service not known
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) [rt.jar:1.8.0_111]
	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) [rt.jar:1.8.0_111]
	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) [rt.jar:1.8.0_111]
	at java.net.InetAddress.getAllByName0(InetAddress.java:1276) [rt.jar:1.8.0_111]
	at java.net.InetAddress.getAllByName(InetAddress.java:1192) [rt.jar:1.8.0_111]
	at java.net.InetAddress.getAllByName(InetAddress.java:1126) [rt.jar:1.8.0_111]

This happens when the API queries the backend, also from server.log:

2016-10-22 11:56:56,704 ERROR [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default task-7) [] Exception: java.lang.RuntimeException: Unexpected client exception
	at com.woorea.openstack.connector.RESTEasyConnector.request(RESTEasyConnector.java:107) [resteasy-connector.jar:]
	at com.woorea.openstack.base.client.OpenStackClient.request(OpenStackClient.java:51) [openstack-client.jar:]
	at com.woorea.openstack.base.client.OpenStackClient.execute(OpenStackClient.java:66) [openstack-client.jar:]
	at com.woorea.openstack.base.client.OpenStackRequest.execute(OpenStackRequest.java:98) [openstack-client.jar:]
	at org.ovirt.engine.core.bll.provider.storage.OpenStackImageProviderProxy.getAllImagesAsRepoImages(OpenStackImageProviderProxy.java:148) [bll.jar:]
	at org.ovirt.engine.core.bll.storage.domain.IsoDomainListSynchronizer.lambda$refreshImageDomain$1(IsoDomainListSynchronizer.java:237) [bll.jar:]
	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [utils.jar:]
	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:103) [utils.jar:]
	at org.ovirt.engine.core.bll.storage.domain.IsoDomainListSynchronizer.refreshImageDomain(IsoDomainListSynchronizer.java:232) [bll.jar:]
	at org.ovirt.engine.core.bll.storage.domain.IsoDomainListSynchronizer.refreshRepos(IsoDomainListSynchronizer.java:208) [bll.jar:]
	at org.ovirt.engine.core.bll.storage.domain.IsoDomainListSynchronizer.getUserRequestForStorageDomainRepoFileList(IsoDomainListSynchronizer.java:192) [bll.jar:]
	at org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQueryBase.getUserRequestForStorageDomainRepoFileList(GetImagesListQueryBase.java:42) [bll.jar:]
	at org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQueryBase.executeQueryCommand(GetImagesListQueryBase.java:27) [bll.jar:]
	at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:103) [bll.jar:]
	at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:]

The backend should probably catch the exception and generate a meaningful error message.

There is nothing to change. I am moving to the API component for further investigation, but most probably the changes will be done in the backend.

Comment 3 Juan Hernández 2016-10-25 10:26:49 UTC
Apparently the "execute" method of the OpenStack client throws unchecked exceptions, like UnknownHostException in this case, and we don't handle them correctly. I'd suggest to wrap calls to that "execute" method with code that handles them correctly, at least the one that causes this particular error. That needs to be changed in the backend, so I'm moving to the backend component and to the storage team.

Comment 4 Elad 2017-02-15 11:32:25 UTC
Got the following error message when tried to retrieve the images list of an unreachable Glance repository (127.0.0.1):

2017-02-15 13:28:05,719+02 ERROR [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default task-21) [dc2c439d-8ee1-4dcd-91a4-b2eafab1cfc0] Failed to retrieve image list: Connection refused (Connection refused)

Used:
rhevm-4.1.1-0.1.el7.noarch


Note You need to log in before you can comment on or make changes to this bug.