Bug 1177890 - External Keystone Connection Fails to Juno-based OpenStack
Summary: External Keystone Connection Fails to Juno-based OpenStack
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: rhevm-dependencies
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ovirt-3.6.0-rc
: 3.6.0
Assignee: Moti Asayag
QA Contact: Ondra Machacek
URL:
Whiteboard:
Depends On:
Blocks: 1181664 1185619
TreeView+ depends on / blocked
 
Reported: 2014-12-31 05:41 UTC by david.costakos
Modified: 2016-03-09 19:47 UTC (History)
15 users (show)

Fixed In Version: ovirt-engine-backend-3.6.0-0.0.master.20150412172306.git55ba764
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1181664 1185619 (view as bug list)
Environment:
Last Closed: 2016-03-09 19:47:33 UTC
oVirt Team: Infra
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:0398 0 normal SHIPPED_LIVE rhevm-dependencies enhancement update 2016-03-09 23:49:44 UTC
oVirt gerrit 36837 0 master MERGED engine: Bump openstack-java-sdk version to 3.0.6 2020-07-16 14:49:24 UTC
oVirt gerrit 37270 0 ovirt-engine-3.5 MERGED engine: Bump openstack-java-sdk version to 3.0.6 2020-07-16 14:49:24 UTC

Description david.costakos 2014-12-31 05:41:47 UTC
Description of problem:
External Provider connections in 3.5.0-0.20.el6ev fail at testing keystone authentication due to unknown/unfiltered keystone token attribute 'audit_ids'.


Version-Release number of selected component (if applicable):
3.5.0-0.20.el6ev


How reproducible:
always


Steps to Reproduce:
1. Connect to Keystone on a Juno-based RHEL OSP installation
2. Attempt to validate connection, it fails each time.
3.

Actual results:
Get log entries like this:
2014-12-30 21:27:45,534 ERROR [org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand] (ajp-/127.0.0.1:8702-12) [3e1a39a5] Command org.ovirt.engine.core.bll.provider.TestProvider
ConnectivityCommand throw Vdc Bll exception. With error message VdcBLLException: (Failed with error PROVIDER_FAILURE and code 5050)
2014-12-30 21:27:45,541 DEBUG [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) Before releasing a lock EngineLock [exclusiveLocks= key: 4a23ff16-a07e-49a3-ae80-f84892b20059 value: VDS_INIT
, sharedLocks= ]
2014-12-30 21:27:45,541 DEBUG [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) The exclusive lock for key 4a23ff16-a07e-49a3-ae80-f84892b20059VDS_INIT is released and lock is removed from map
2014-12-30 21:27:45,535 DEBUG [org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand] (ajp-/127.0.0.1:8702-12) [3e1a39a5] Command org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand throw Vdc Bll exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: (Failed with error PROVIDER_FAILURE and code 5050)
        at org.ovirt.engine.core.bll.provider.OpenStackImageProviderProxy.testConnection(OpenStackImageProviderProxy.java:94) [bll.jar:]
        at org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand.executeCommand(TestProviderConnectivityCommand.java:36) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1168) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1307) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1932) [bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:]
        at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1331) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:]
        at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:]
        at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:]
        at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:369) [bll.jar:]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_71]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_71]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_71]
        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_71]
        at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.2.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.2.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.2.Final-redhat-2]
        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.2.Final-redhat-2]
        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.2.Final-redhat-2]
        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.2.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:]
        at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_71]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_71]
        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_71]
        at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) [jboss-as-ee.jar:7.4.2.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.2.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.2.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:73) [jboss-as-weld.jar:7.4.2.Final-redhat-2]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1]
        at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1
... snip ...
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:490)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926)
        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71]
Caused by: org.jboss.resteasy.spi.ReaderException: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "audit_ids" (Class com.woorea.openstack.keystone.model.Token), not marked as ignorable
 at [Source: org.jboss.resteasy.core.interception.MessageBodyReaderContextImpl$InputStreamWrapper@5ac343ac; line: 1, column: 276] (through reference chain: com.woorea.openstack.keystone.model.Access["token"]->com.woorea.openstack.keystone.model.Token["audit_ids"])
        at org.jboss.resteasy.client.core.BaseClientResponse.readFrom(BaseClientResponse.java:469) [resteasy-jaxrs.jar:]
        at org.jboss.resteasy.client.core.BaseClientResponse.getEntity(BaseClientResponse.java:377) [resteasy-jaxrs.jar:]
        at org.jboss.resteasy.client.core.BaseClientResponse.getEntity(BaseClientResponse.java:350) [resteasy-jaxrs.jar:]
        at org.jboss.resteasy.client.core.BaseClientResponse.getEntity(BaseClientResponse.java:344) [resteasy-jaxrs.jar:]
        at com.woorea.openstack.connector.RESTEasyResponse.getEntity(RESTEasyResponse.java:25) [resteasy-connector.jar:]
        at com.woorea.openstack.base.client.OpenStackClient.execute(OpenStackClient.java:67) [openstack-client.jar:]
        at com.woorea.openstack.base.client.OpenStackRequest.execute(OpenStackRequest.java:98) [openstack-client.jar:]
        at com.woorea.openstack.keystone.utils.KeystoneTokenProvider.getAccessByTenant(KeystoneTokenProvider.java:30) [keystone-client.jar:]
        at com.woorea.openstack.keystone.utils.KeystoneTokenProvider$1.getToken(KeystoneTokenProvider.java:47) [keystone-client.jar:]
        at com.woorea.openstack.base.client.OpenStackClient.request(OpenStackClient.java:47) [openstack-client.jar:]
        at com.woorea.openstack.base.client.OpenStackClient.execute(OpenStackClient.java:66) [openstack-client.jar:]
        at org.ovirt.engine.core.bll.provider.OpenStackImageProviderProxy.testConnection(OpenStackImageProviderProxy.java:92) [bll.jar:]
        ... 123 more
Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "audit_ids" (Class com.woorea.openstack.keystone.model.Token), not marked as ignorable
 at [Source: org.jboss.resteasy.core.interception.MessageBodyReaderContextImpl$InputStreamWrapper@5ac343ac; line: 1, column: 276] (through reference chain: com.woorea.openstack.keystone.model.Access["token"]->com.woorea.openstack.keystone.model.Token["audit_ids"])
        at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.ObjectMapper._unwrapAndDeserialize(ObjectMapper.java:2802) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2702) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315) [jackson-mapper-asl.jar:1.9.9.redhat-3]
        at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
        at org.jboss.resteasy.core.interception.MessageBodyReaderContextImpl.proceed(MessageBodyReaderContextImpl.java:106) [resteasy-jaxrs.jar:]
        at org.jboss.resteasy.client.core.BaseClientResponse.readFrom(BaseClientResponse.java:433) [resteasy-jaxrs.jar:]
        ... 134 more




Expected results:
successful external provider connection to a Juno-based OpenStack installation


Additional info:
I don't have much time left to keep my test environment around in a non-functioning state.  But I can currently give someone short-term access to my env for testing, it's in Boston in a lab.

Comment 1 Oved Ourfali 2014-12-31 06:19:28 UTC
Seems like it is a known issue, addressed by the following patch of the openstack-java-sdk:
https://github.com/woorea/openstack-java-sdk/pull/166

Until it is merged, and there is a release with that, we can't address the issue in the engine side.

All I can recommend is to work with icehouse and not juno until this issue is resolved in the openstack-java-sdk.

Comment 5 Federico Simoncelli 2015-01-07 13:30:24 UTC
The upstream pull request has been merged and openstack-java-sdk-3.0.6 should reach the mirrors within 2 business day.

Comment 16 Moti Asayag 2015-01-13 22:07:21 UTC
(In reply to Federico Simoncelli from comment #13)
> (In reply to Oved Ourfali from comment #12)
> > fsimonce - what should be the version to depend on now?
> 
> I've mentioned it in comment #5 openstack-java-sdk-3.0.6

I've verified locally by [1] that openstack-java-sdk-3.0.6 fixed authentication issue against keystone. 

Federico, could you produce rpms in koji updates-testing ?  


[1] http://gerrit.ovirt.org/#/c/36837/

Comment 17 Alon Bar-Lev 2015-01-14 08:39:54 UTC
please dup to 3.5 so I can build rhevm-dependencies with this.

Comment 20 Federico Simoncelli 2015-01-15 23:11:58 UTC
(In reply to Moti Asayag from comment #16)
> (In reply to Federico Simoncelli from comment #13)
> > (In reply to Oved Ourfali from comment #12)
> > > fsimonce - what should be the version to depend on now?
> > 
> > I've mentioned it in comment #5 openstack-java-sdk-3.0.6
> 
> I've verified locally by [1] that openstack-java-sdk-3.0.6 fixed
> authentication issue against keystone. 
> 
> Federico, could you produce rpms in koji updates-testing ?  

The official builds are available here:

https://admin.fedoraproject.org/updates/openstack-java-sdk

please add karma to promote to updates.

Comment 25 Ondra Machacek 2015-05-12 08:02:54 UTC
Works ok with ovirt-engine-backend-3.6.0-0.0.master.20150412172301.git55ba764.el7.centos.noarch

Comment 27 errata-xmlrpc 2016-03-09 19:47:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0398.html


Note You need to log in before you can comment on or make changes to this bug.