Hide Forgot
Created attachment 1230833 [details] engine.log and server.log Description of problem: When refreshing the default Glance repository images list ('ovirt-image-repository'), the operation fails with NullPointerException. The returned list is empty and no error appears to the admin. Version-Release number of selected component (if applicable): ovirt-engine-4.0.6-0.1.el7ev.noarch How reproducible: Happened to me with the default 'ovirt-image-repository'. With another Glance provider, the operation succeeds Steps to Reproduce: 1. Via Webadmin, navigate to the Images subtab under the 'ovirt-image-repository' Actual results: No images appear. No error in the Webadmin. In engine.log: 2016-12-12 17:47:26,101 ERROR [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default task-14) [] Query 'GetImagesListQuery' failed: null 2016-12-12 17:47:26,102 ERROR [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default task-14) [] Exception: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) [rt.jar:1.8.0_102] at com.woorea.openstack.keystone.utils.KeystoneTokenProvider.getAccessByTenant(KeystoneTokenProvider.java:28) [keystone-client.jar:] at com.woorea.openstack.keystone.utils.KeystoneTokenProvider$1.getToken(KeystoneTokenProvider.java:47) [keystone-client.jar:] at com.woorea.openstack.base.client.OpenStackClient.request(OpenStackClient.java:47) [openstack-client.jar:] at com.woorea.openstack.base.client.OpenStackClient.execute(OpenStackClient.java:66) [openstack-client.jar:] at com.woorea.openstack.base.client.OpenStackRequest.execute(OpenStackRequest.java:98) [openstack-client.jar:] at org.ovirt.engine.core.bll.provider.storage.OpenStackImageProviderProxy.getAllImagesAsRepoImages(OpenStackImageProviderProxy.java:152) [bll.jar:] at org.ovirt.engine.core.bll.storage.domain.IsoDomainListSyncronizer.lambda$refreshImageDomain$1(IsoDomainListSyncronizer.java:237) [bll.jar:] Expected results: Images should get refreshed successfully with no exception. Additional info: engine.log and server.log
Elad, going through the stack trace it seems like a null tenant name was provided, can you check that the field is populated?
Indeed, the tenant name is null in my setup (and I think it's the default for the default ovirt repository).
The code in KeystoneTokenProvider calls: hashTenantAccess.put(tenantName, access); Which in ConcurrentHashMap tries to hash the null key which results in an NPE: int h = spread(key.hashCode()); I really doubt this code has worked before with a null tenant, are you sure it did?
Can't tell for sure and we don't have an upstream setup which is lower than 4.0
Daniel, any idea?
Targeting to 4.1 for now pending on the needinfo on Daniel
Hi Elad, Is it 100% reproducible on latest build? (can't reproduce it on dev env) Any network connection error perhaps?
In latest build [1], the authentication is not required for the default ovirt Glance repository. It cannot be set to have authentication without tenant as it is configured in 4.0.6 where I encountered the bug. It seems that the issue here is that in 4.0.6 there is a different set of configuration for the default ovirt Glance repository where the tenant and authentication URL are not set while the authentication is required. This set of configuration exposes the exception we get. In latest built (4.1), the authentication is not required for the default ovirt Glance repository and once set required, the tenant and authentication URL are mandatory. [1] ovirt-engine-4.1.0-0.2.master.20161210231201.git26a385e.el7.centos.noarch
(In reply to Elad from comment #8) > In latest build [1], the authentication is not required for the default > ovirt Glance repository. It cannot be set to have authentication without > tenant as it is configured in 4.0.6 where I encountered the bug. > > It seems that the issue here is that in 4.0.6 there is a different set of > configuration for the default ovirt Glance repository where the tenant and > authentication URL are not set while the authentication is required. This > set of configuration exposes the exception we get. > In latest built (4.1), the authentication is not required for the default > ovirt Glance repository and once set required, the tenant and authentication > URL are mandatory. > > > > [1] ovirt-engine-4.1.0-0.2.master.20161210231201.git26a385e.el7.centos.noarch So IIUC, the issue can't be reproduced in 4.1, only 4.0.z?
> So IIUC, the issue can't be reproduced in 4.1, only 4.0.z? Yes
(In reply to Elad from comment #10) > > So IIUC, the issue can't be reproduced in 4.1, only 4.0.z? > Yes Thanks Elad! Moving to MODIFIED to verify on latest build.
Created attachment 1235616 [details] engine.log The operation seems to succeed but in engine.log I still see: 2016-12-28 13:46:26,991+02 ERROR [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default task-19) [41b193cf-286e-46d2-b0b4-a4679e4a6365] Query 'GetImagesListQuery' failed: null 2016-12-28 13:46:26,991+02 ERROR [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default task-19) [41b193cf-286e-46d2-b0b4-a4679e4a6365] Exception: java.lang.NullPointerException attached
Daniel, Can you please take a look?
(In reply to Raz Tamir from comment #13) > Daniel, > Can you please take a look? The exception originated from the woorea keystone project. Is it the latest 4.1 build? All images are displayed correctly? Is it the default Glance repo configuration?
This is the latest - 4.1.0-0.3.beta2.el7 and there are no images in ovirt-image-repository so there's probably an issue
(In reply to Raz Tamir from comment #15) > This is the latest - 4.1.0-0.3.beta2.el7 and there are no images in > ovirt-image-repository so there's probably an issue Looks similar to the issue Elad described. @Elad - is it the same issue from 4.0.6? Is the default configuration working correctly for you in 4.1? On my local env it works well, so I need to understand what's the reproduction scenario..
It's the same. It reproduced while you have the default ovirt Glance repository configured with authentication but without tenant. This set of configuration is impossible to be set by the user, so in case it wasn't pre-configured by engine automatically during setup, there is no way of reproducing it.
(In reply to Elad from comment #18) > It's the same. > It reproduced while you have the default ovirt Glance repository configured > with authentication but without tenant. > This set of configuration is impossible to be set by the user, so in case it > wasn't pre-configured by engine automatically during setup, there is no way > of reproducing it. OK, so IIUC, it shouldn't happen in a clean 4.1 setup? So it won't affect users as the default Glance repository doesn't require authentication.
Daniel, can you say for sure that in upcoming builds we won't have the authentication as required with no tenant (that reveals the bug) as out of the box configuration?
(In reply to Elad from comment #20) > Daniel, can you say for sure that in upcoming builds we won't have the > authentication as required with no tenant (that reveals the bug) as out of > the box configuration? I've tested it on the latest build (4.1.0-0.0.master.20170103090346) - seems to work as expected. So should work fine with the upcoming build on a clean environment (IIUC the exact scenario of course...)
The default oVirt Glance repository is configured without authentication, images list refresh works as expected. Verified using: ovirt-engine-4.1.0-0.4.master.20170117182028.git4678f8c.el7.centos.noarch
The bug is reproduced on latest downstream (rhevm-4.1.1.3-0.1.el7.noarch). Authentication is configured as true for an out of the box setup. 2017-03-12 14:20:27,256+02 ERROR [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default task-26) [0e96ee6f-6695-465a-a405-4c2a6d62b629] Exception: java.lang.NullPointerException at java.util.Objects.requireNonNull(Objects.java:203) [rt.jar:1.8.0_121] at java.util.Optional.<init>(Optional.java:96) [rt.jar:1.8.0_121] at java.util.Optional.of(Optional.java:108) [rt.jar:1.8.0_121] at org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQueryBase.executeQueryCommand(GetImagesListQueryBase.java:31) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:110) [bll.jar:] Daniel, should I open a new bug or re-open this one (if it's possible as it's CLOSED CURRENTRELEASE)?
(In reply to Elad from comment #23) > The bug is reproduced on latest downstream (rhevm-4.1.1.3-0.1.el7.noarch). > Authentication is configured as true for an out of the box setup. Is it a clean or upgraded env? Was it installed differently from previous build (as it was verified previously..)? > > 2017-03-12 14:20:27,256+02 ERROR > [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default > task-26) [0e96ee6f-6695-465a-a405-4c2a6d62b629] Exception: > java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > [rt.jar:1.8.0_121] > at java.util.Optional.<init>(Optional.java:96) [rt.jar:1.8.0_121] > at java.util.Optional.of(Optional.java:108) [rt.jar:1.8.0_121] > at > org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQueryBase. > executeQueryCommand(GetImagesListQueryBase.java:31) [bll.jar:] > at > org.ovirt.engine.core.bll.QueriesCommandBase. > executeCommand(QueriesCommandBase.java:110) [bll.jar:] > > > Daniel, should I open a new bug or re-open this one (if it's possible as > it's CLOSED CURRENTRELEASE)?
(In reply to Daniel Erez from comment #24) > Is it a clean or upgraded env? Was it installed differently from previous > build (as it was verified previously..)? Installed downstream 4.1.1.2 and upgraded to downstream 4.1.1.3. This one is downstream and I verified using upstream
Created attachment 1262243 [details] providers DB table
Created attachment 1262428 [details] logs-13.3.17 2017-03-13 14:26:23,176+02 ERROR [org.ovirt.engine.core.bll.storage.repoimage.GetImagesListQuery] (default task-12) [8d56fb5e-fa1d-4ff0-8296-1c68322f54a8] Exception: java.lang.NullPointerException at java.util.Objects.requireNonNull(Objects.java:203) [rt.jar:1.8.0_121] at java.util.Optional.<init>(Optional.java:96) [rt.jar:1.8.0_121] at java.util.Optional.of(Optional.java:108) [rt.jar:1.8.0_121]
Created attachment 1262436 [details] logs-13.3.17(1)
Created attachment 1262438 [details] logs-13.3.17(2) This one includes all engine.log files, setup logs and server.log
setup/ovirt-engine-setup-20170301164636-txlqg8.log: setup started at 2017-03-01 16:46:36 and finished at 2017-03-01 17:00:35 . Compatiengine.log-20170307.gz: 2017-03-01 17:12:13,033+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-21) [openstackimageproviders_create_30bde] EVENT_ID: PROVIDER_ADDED(205), Correlation ID: openstackimageproviders_create_30bde, Call Stack: null, Custom Event ID: -1, Message: Provider rhevm-qe-infra-glance was added. (User: admin@internal-authz) 2017-03-01 17:12:13,588+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-24) [openstackimageproviders_create_82b57] EVENT_ID: PROVIDER_ADDED(205), Correlation ID: openstackimageproviders_create_82b57, Call Stack: null, Custom Event ID: -1, Message: Provider ovirt-image-repository was added. (User: admin@internal-authz) So they were not created by engine-setup.