Bug 534213 (RHQ-1030)

Summary:

plugin config editing affects multiple resources, last one edited wins

Product:

[Other] RHQ Project

Reporter:

John Mazzitelli <mazz>

Component:

Plugin Container

Assignee:

RHQ Project Maintainer <rhq-maint>

Status:

CLOSED NOTABUG

QA Contact:

Severity:

medium

Docs Contact:

Priority:

high

Version:

1.1

CC:

fmatar

Target Milestone:

---

Target Release:

---

Hardware:

All

OS:

All

URL:

http://jira.rhq-project.org/browse/RHQ-1030

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

Type:

---

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
233028-bug.swf	none

Description John Mazzitelli 2008-10-25 19:46:00 UTC

This has the potential to being very bad.  It looks like the plugin container is sharing plugin configurations across multiple resources.

See the attached flash demo to see the bug and how to replicate it.

This shows a Jopr installation because its illustrating the bug with JBossAS resources, but I did not file this bug in the Jopr JIRA because I think this may be a problem with the plugin container as a whole (it looks like its sharing plugin configurations across resources).

In short: I have two resources (JBossAS servers) with two different plugin configurations (the security credentials to connect to them are both different).  Only one resource can connect, the other cannot.  The last resource whose plugin configuraiton was edited/save goes GREEN, the other RED .

See the demo, it makes it more clear what the bug is.

Comment 1 John Mazzitelli 2008-10-26 18:14:29 UTC

Looks like an EMS issue. Put a breakpoint on the first line of JBossASServerComponent.getAvailability.

As it gets the availability for the default1 (127.0.0.1), the EMS connection has this connection settings:

EmsConnection [JBoss]
	Initial Context = org.jnp.interfaces.NamingContextFactory
	JNDI Name = jmx/rmi/RMIAdaptor
	Server URL = jnp://127.0.0.1:1099
	Principle = admin1
	Credentials = admin1
	 Advanced Property [java.naming.factory.initial] = org.jboss.security.jndi.JndiLoginInitialContextFactory
	 Advanced Property [java.naming.security.principal] = admin1
	 Advanced Property [jnp.sotimeout] = 15000
	 Advanced Property [jnp.timeout] = 30000
	 Advanced Property [jnp.disableDiscovery] = true
	 Advanced Property [mc4j.ems.DefaultDomainSearch] = jboss
	 Advanced Property [java.naming.provider.url] = jnp://127.0.0.1:1099
	 Advanced Property [java.naming.factory.url.pkgs] = org.jboss.naming:org.jnp.interfaces
	 Advanced Property [java.naming.security.credentials] = admin1
	 Control Property [mc4j.ems.CopyJarsToTemp] = true
	 Control Property [mc4j.ems.JarTempDir] = C:\mazz\source\rhq\trunk\modules\enterprise\agent\target\rhq-agent-1.2.0-SNAPSHOT\data\tmp

This connection allows for this to be successful:

            bean.getAttribute("Started").refresh();
            return AvailabilityType.UP;

Next, when the default2 server's avail is checked (192.168.0.7), there is the EMS connection settings:

EmsConnection [JBoss]
	Initial Context = org.jnp.interfaces.NamingContextFactory
	JNDI Name = jmx/rmi/RMIAdaptor
	Server URL = jnp://192.168.0.7:1099
	Principle = admin2
	Credentials = admin2
	 Advanced Property [java.naming.factory.initial] = org.jboss.security.jndi.JndiLoginInitialContextFactory
	 Advanced Property [java.naming.security.principal] = admin2
	 Advanced Property [jnp.sotimeout] = 15000
	 Advanced Property [jnp.timeout] = 30000
	 Advanced Property [jnp.disableDiscovery] = true
	 Advanced Property [mc4j.ems.DefaultDomainSearch] = jboss
	 Advanced Property [java.naming.provider.url] = jnp://192.168.0.7:1099
	 Advanced Property [java.naming.factory.url.pkgs] = org.jboss.naming:org.jnp.interfaces
	 Advanced Property [java.naming.security.credentials] = admin2
	 Control Property [mc4j.ems.CopyJarsToTemp] = true
	 Control Property [mc4j.ems.JarTempDir] = C:\mazz\source\rhq\trunk\modules\enterprise\agent\target\rhq-agent-1.2.0-SNAPSHOT\data\tmp

This connection however causes this to throw an exception:

            bean.getAttribute("Started").refresh();

The exception is (note: why is EMS using admin1 principal? you can see the connection's settings has admin2, from above):

org.mc4j.ems.connection.EmsException: Could not load attribute value Connection failure Failed to authenticate principal=admin1, securityDomain=jmx-console
        at org.mc4j.ems.impl.jmx.connection.bean.attribute.DAttribute.refresh(DAttribute.java:235)
        at org.rhq.plugins.jbossas.JBossASServerComponent.getAvailability(JBossASServerComponent.java:292)
        at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:450)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:619)
Caused by: org.mc4j.ems.connection.EmsConnectException: Connection failure Failed to authenticate principal=admin1, securityDomain=jmx-console
        at org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy.invoke(GenericMBeanServerProxy.java:144)
        at $Proxy44.getAttribute(Unknown Source)
        at org.mc4j.ems.impl.jmx.connection.bean.attribute.DAttribute.refresh(DAttribute.java:199)
        ... 10 more
Caused by: java.lang.SecurityException: Failed to authenticate principal=admin1, securityDomain=jmx-console
        at org.jboss.jmx.connector.invoker.AuthenticationInterceptor.invoke(AuthenticationInterceptor.java:97)
        at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
        at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
        at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
        at org.jboss.invocation.jrmp.server.JRMPProxyFactory.invoke(JRMPProxyFactory.java:179)
        at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
        at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
        at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
        at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
        at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
        at org.jboss.invocation.jrmp.server.JRMPInvoker$MBeanServerAction.invoke(JRMPInvoker.java:818)
        at org.jboss.invocation.jrmp.server.JRMPInvoker.invoke(JRMPInvoker.java:419)
        at sun.reflect.GeneratedMethodAccessor79.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
        at sun.rmi.transport.Transport$1.run(Transport.java:159)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:619)
        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:255)
        at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:233)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:142)
        at org.jboss.invocation.jrmp.server.JRMPInvoker_Stub.invoke(Unknown Source)
        at org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxy.invoke(JRMPInvokerProxy.java:133)
        at org.jboss.invocation.InvokerInterceptor.invokeInvoker(InvokerInterceptor.java:365)
        at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:197)
        at org.jboss.jmx.connector.invoker.client.InvokerAdaptorClientInterceptor.invoke(InvokerAdaptorClientInterceptor.java:66)
        at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:70)
        at org.jboss.proxy.ClientMethodInterceptor.invoke(ClientMethodInterceptor.java:74)
        at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:100)
        at $Proxy43.getAttribute(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBean
ServerProxy.invoke(GenericMBeanServerProxy.java:115)
        ... 12 more

Comment 2 John Mazzitelli 2008-10-26 18:23:10 UTC

renaming the subject since it looks like plugin configurations *are* correct, however, the underlying EMS library is somehow getting confused.  This may be a problem with EMS or the JMX plugin, I am not sure.

Comment 3 John Mazzitelli 2008-10-27 19:05:10 UTC

greg found the issue - its inside the JBoss JNP code.  Will need to talk to the JBossAS devs to find out how to fix.

In the meantime, the only workaround is to make sure all of your credentials for all of your JBossAS isntances running on the same box are the same.  This doesn't mean ALL user/passwords are the same across ALL boxes - only has to be same for those instances running on the same box.

Comment 4 John Mazzitelli 2008-10-27 19:06:56 UTC

Here's code to replciate the problem, to prove there is nothing specific to RHQ or EMS that causes the problem.  Note this code only uses javax.* classes - but you need the jboss client jar in the classpath when you run this. Set up two JBossAS servers, each bound to 127.0.0.1 and .2, each with different user/passwords and run this test:

package org.jboss.on.server.other;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.management.MBeanServer;
import javax.management.MBeanServerConnection;
import java.util.Properties;

public class MultiJBossConnectTest {
    public static void main(String[] args) throws Exception {
        MBeanServerConnection con1  = (MBeanServerConnection)connect("admin","admin2", "jnp://127.0.0.1");
        System.out.println("con1 done: " + con1.getMBeanCount());
        MBeanServerConnection con2 = (MBeanServerConnection) connect("admin", "admin3", "jnp://127.0.0.2");
        System.out.println("con2 done: " + con2.getMBeanCount());
        System.out.println("con1 done: " + con1.getMBeanCount());
    }

    private static Object connect(String user, String pass, String url) throws ClassNotFoundException, NamingException {
        Properties props = new Properties();
        Class.forName("org.jboss.security.jndi.JndiLoginInitialContextFactory");
        props.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.security.jndi.JndiLoginInitialContextFactory");
        props.put(Context.SECURITY_PRINCIPAL, user);
        props.put(Context.SECURITY_CREDENTIALS, pass);
        props.put(Context.PROVIDER_URL, url);
        props.put(Context.URL_PKG_PREFIXES, "org.jboss.naming:org.jnp.interfaces" );
        props.put("jnp.disableDiscovery", "True");
        InitialContext context = new InitialContext(props);
        Object rmiAdaptor = context.lookup("jmx/rmi/RMIAdaptor");
        return rmiAdaptor;
    }
}

Comment 5 Greg Hinkle 2008-10-28 17:11:59 UTC

This is not an RHQ bug. The config content is correctly being passed into the plugin.... in this case the naming context is storing stuff to thread local uncontrollably. Problem moved to JOPR-9 (https://jira.jboss.org/jira/browse/JOPR-9).

Comment 6 Red Hat Bugzilla 2009-11-10 20:22:14 UTC

This bug was previously known as http://jira.rhq-project.org/browse/RHQ-1030
Imported an attachment (id=368482)
This bug is duplicated by RHQ-945

Comment 7 David Lawrence 2009-11-11 17:30:41 UTC

*** Bug 536613 has been marked as a duplicate of this bug. ***