Created attachment 709544 [details] rhq-server.properties Description of problem: I have successfully configured the RHQ server on IPv6 but the agent is not able to poll the server so the registration process fails and the agent gets stuck on "The agent will now wait until it has registered with the server..." Version-Release number of selected component (if applicable): Version: 4.7.0-SNAPSHOT Build Number: 2174d4c How reproducible: Always Steps to Reproduce: 1. configure the server to use IPv6: a) change the rhq-server.sh to use following java opts -Djava.net.preferIPv4Stack=false -Djava.net.preferIPv6Addresses=true for starting the server b) use previous java opts for the rhq-installer.sh as well c) prepare the rhq-server.properties (look at attached example) d) change the rhq-server.sh - line 471 should look like this: export RHQ_SERVER_HOME; ${RHQ_SERVER_HOME}/bin/rhq-installer.sh --reconfig --host=::1 & (i'm not sure if this is necessary) e) run the server - ./rhq-server.sh start f) run the installer - ./rhq-installer.sh --host=::1 2. configure the agent to use IPv6: a)change the rhq-agent.sh to use following java opts -Djava.net.preferIPv4Stack=false -Djava.net.preferIPv6Addresses=true b) enable debug mode c) run the agent with a clean config - ./rhq-agent.sh -L d) use IPv6 Actual results: the agent gets stuck on "The agent will now wait until it has registered with the server..." And the agent log ends with: 2013-03-13 06:21:53,358 DEBUG [RHQ Server Polling Thread] (jboss.remoting.transport.http.HTTPClientInvoker)- Unable to retrieve response message java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384) at java.net.Socket.connect(Socket.java:546) at sun.net.NetworkClient.doConnect(NetworkClient.java:173) at sun.net.www.http.HttpClient.openServer(HttpClient.java:409) at sun.net.www.http.HttpClient.openServer(HttpClient.java:530) at sun.net.www.http.HttpClient.<init>(HttpClient.java:240) at sun.net.www.http.HttpClient.New(HttpClient.java:321) at sun.net.www.http.HttpClient.New(HttpClient.java:338) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1139) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:397) at java.net.HttpURLConnection.getResponseMessage(HttpURLConnection.java:463) at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:340) at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:137) at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122) at org.jboss.remoting.Client.invoke(Client.java:1634) at org.jboss.remoting.Client.invoke(Client.java:548) at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.rawSend(JBossRemotingRemoteCommunicator.java:514) at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.sendWithoutCallbacks(JBossRemotingRemoteCommunicator.java:456) at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.sendWithoutInitializeCallback(JBossRemotingRemoteCommunicator.java:475) at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.send(JBossRemotingRemoteCommunicator.java:496) at org.rhq.enterprise.communications.command.client.AbstractCommandClient.invoke(AbstractCommandClient.java:143) at org.rhq.enterprise.communications.command.client.ClientCommandSender.send(ClientCommandSender.java:1091) at org.rhq.enterprise.communications.command.client.ServerPollingThread.run(ServerPollingThread.java:115) 2013-03-13 06:21:53,363 DEBUG [RHQ Server Polling Thread] (enterprise.communications.command.client.ServerPollingThread)- {ServerPollingThread.server-poll-failure}Failed to successfully poll the server. This is normally due to the server not being up yet. You can usually ignore this message since it will be tried again later, however, you should ensure this failure was not really caused by a misconfiguration. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. Connection refused. -> java.net.ConnectException:Connection refused Expected results: Agent is registered with the server. Additional info: The cause could be that rhq-agent/lib/jboss-remoting-2.2.2.SP8.jar doesn't support the IPv6 (this should be confirmed by someone) I noticed in the agent's log following: 2013-03-13 06:21:53,229 DEBUG [main] (org.rhq.enterprise.communications.ServiceContainer)- {ServiceContainer.connector-uri}Connector is using locator URI of [InvokerLocator [socket://10.16.23.190/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000]] there is a IPv4 (10.16.23.190) but ~/.java/.userPrefs/rhq-agent/default/prefs.xml contains only IPv6. We suspect that jboss-remoting library changed IPv6 to IPv4
Created attachment 709545 [details] agent log
That was the first thing I thought of - does JBoss/Remoting 2 support IPv6? I don't know. The thing that I dn't get is that log message with the IPv4 address. Somehow, someway, the agent's address is getting reset to that. Can you attach the FULL agent log (with debug enabled) from start to finish? In otherwords, start with a clean agent, have debug enabled (RHQ_AGENT_DEBUG=true), and do that again, then attach agent.log. Also, attach the output of "getconfig" to that BZ as well
opps... duh... didn't notice you already attached agent.log :) can you post the "getconfig" output, too?
attaching the agent's config: shutdown> getconfig rhq.agent.agent-update.enabled=true rhq.agent.client.command-preprocessors=org.rhq.enterprise.agent.SecurityTokenCommandPreprocessor:org.rhq.enterprise.agent.ExternalizableStrategyCommandPreprocessor rhq.agent.client.command-spool-file.compressed=true rhq.agent.client.command-spool-file.name=command-spool.dat rhq.agent.client.command-spool-file.params=10000000:75 rhq.agent.client.command-timeout-msecs=600000 rhq.agent.client.max-concurrent=5 rhq.agent.client.max-retries=10 rhq.agent.client.queue-size=50000 rhq.agent.client.queue-throttling=200:2000 rhq.agent.client.retry-interval-msecs=15000 rhq.agent.client.send-throttling=100:1000 rhq.agent.client.server-polling-interval-msecs=60000 rhq.agent.configuration-schema-version=6 rhq.agent.configuration-setup-flag=true rhq.agent.data-directory=data rhq.agent.disable-native-system=false rhq.agent.name=last-rhq-fbrychta-5m.bc.jonqe.lab.eng.bos.redhat.com rhq.agent.plugins.directory=plugins rhq.agent.primary-server-switchover-check-interval-msecs=3600000 rhq.agent.register-with-server-at-startup=true rhq.agent.server-auto-detection=false rhq.agent.server.alias=rhqserver rhq.agent.server.bind-address=2620:52:0:1017:21a:4aff:fe10:17d9 rhq.agent.server.bind-port=7080 rhq.agent.server.transport=servlet rhq.agent.server.transport-params=/jboss-remoting-servlet-invoker/ServerInvokerServlet rhq.agent.test-failover-list-at-startup=true rhq.agent.update-plugins-at-startup=true rhq.agent.vm-health-check.interval-msecs=5000 rhq.agent.vm-health-check.low-heap-mem-threshold=0.90 rhq.agent.vm-health-check.low-nonheap-mem-threshold=0.90 rhq.agent.wait-for-server-at-startup-msecs=60000 rhq.communications.command-service-directory.allow-dynamic-discovery=true rhq.communications.command-services=org.rhq.enterprise.communications.command.impl.echo.server.EchoCommandService, org.rhq.enterprise.communications.command.impl.identify.server.IdentifyCommandService rhq.communications.configuration-schema-version=1 rhq.communications.connector.bind-address=2620:52:0:1017:21a:4aff:fe10:17d9 rhq.communications.connector.bind-port=16163 rhq.communications.connector.rhqtype=agent rhq.communications.connector.transport=socket rhq.communications.data-directory=data rhq.communications.global-concurrency-limit=-1 rhq.communications.multicast-detector.bind-address=0.0.0.0 rhq.communications.multicast-detector.default-time-delay=5000 rhq.communications.multicast-detector.enabled=false rhq.communications.multicast-detector.heartbeat-time-delay=1000 rhq.communications.multicast-detector.multicast-address=224.16.16.16 rhq.communications.multicast-detector.port=16162 rhq.communications.remote-pojos=org.rhq.enterprise.communications.PingImpl:org.rhq.enterprise.communications.Ping rhq.communications.remote-stream-max-idle-time-msecs=300000
I tried it again with all IPs surrounded by brackets (in the agent and server config) and it works. The problem with brackets is probably related to https://issues.jboss.org/browse/JBREM-1164 and brackets in configuration should not be required as soon as the jboss-remoting will be upgraded. (thanks mazz)
this might make a good documentation issue. adding sunny to notify list