When multiple Apache HTTPD instances are running on a single host that share the same server root (ServerRoot) but different configuration files (httpd.conf), the Apache plug-in will only discover one of the instances. This means that the plug-in is limited to the following use-cases:

  1) you can only run a single Apache instance on a single host
  2) you can configure multiple vhosts in a single httpd.conf for a single Apache instance on a single host
  3) you can install multiple Apache servers using a different ServerRoot and 1) or 2) applies

As the Apache installation is intended to be shared across a host, the plug-in should support the following use-case:

  4) you can install a single Apache instance and start each instance with a different httpd.conf configuration file

It appears this limitation is introduced by the resource key given to an httpd instance upon discovery. I believe the plug-in is using the ServerRoot as the resource key meaning that multiple servers which share the same ServerRoot will not be used/seen. Instead, the plug-in should be using the configuration file and its path as part of the resource key (much like the JBoss AS plug-in does).



Steps to reproduce this issue:

 Red Hat Enterprise Linux Server release 5.4 (Tikanga)
 Linux 2.6.18-164.32.1.el5 #1 SMP Tue Dec 14 11:18:35 EST 2010 x86_64 x86_64 x86_64 GNU/Linux
  
 # Install Sun JRE 1.6 RPM (x86_64):
   cd /tmp && wget -O "jre-6-linux-x64-rpm.bin" http://javadl.sun.com/webapps/download/AutoDL?BundleId=43874  && chmod +x jre-6-linux-x64-rpm.bin && ./jre-6-linux-x64-rpm.bin
   java -version
   java version "1.6.0_23"
   Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
   Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)
 
 # Install Apache HTTPD (2.2.3) via yum
   yum install httpd

 # Configure two httpd instances from same installation
   mkdir /etc/httpd/conf/instance1
   mkdir /etc/httpd/conf/instance2
   mv /etc/httpd/conf/httpd.conf /etc/httpd/conf/instance1
   cp /etc/httpd/conf/instance1/httpd.conf /etc/httpd/conf/instance2
   # Use two different PID files
   sed -i 's/PidFile run\/httpd.pid/PidFile run\/httpd-instance1.pid/g' /etc/httpd/conf/instance1/httpd.conf
   sed -i 's/PidFile run\/httpd.pid/PidFile run\/httpd-instance2.pid/g' /etc/httpd/conf/instance2/httpd.conf
    # Start instance2 on port 81
   sed -i 's/Listen 80/Listen 81/g' /etc/httpd/conf/instance2/httpd.conf
   # Setup init.d script and chkconfig
   cp /etc/init.d/httpd /etc/init.d/httpd-instance1
   cp /etc/init.d/httpd /etc/init.d/httpd-instance2
   sed -i 's/# httpd .* Startup script for the Apache HTTP Server/# httpd-instance1        Startup script for the Apache HTTP Server/g' /etc/init.d/httpd-instance1
   sed -i 's/# config: \/etc\/httpd\/conf\/httpd.conf/# config: \/etc\/httpd\/conf\/instance1\/httpd.conf/g' /etc/init.d/httpd-instance1
   sed -i 's/# pidfile: \/var\/run\/httpd.pid/# pidfile: \/var\/run\/httpd-instance1.pid/g' /etc/init.d/httpd-instance1
   sed -i 's/	CONFFILE=\/etc\/httpd\/conf\/httpd.conf/	CONFFILE=\/etc\/httpd\/conf\/instance1\/httpd.conf/g' /etc/init.d/httpd-instance1
   sed -i 's/HTTPD_LANG=${HTTPD_LANG-"C"}/HTTPD_LANG=${HTTPD_LANG-"C"}\nOPTIONS="-f conf\/instance1\/httpd.conf"/g' /etc/init.d/httpd-instance1
   sed -i 's/# httpd .* Startup script for the Apache HTTP Server/# httpd-instance2        Startup script for the Apache HTTP Server/g' /etc/init.d/httpd-instance2
   sed -i 's/# config: \/etc\/httpd\/conf\/httpd.conf/# config: \/etc\/httpd\/conf\/instance2\/httpd.conf/g' /etc/init.d/httpd-instance2
   sed -i 's/# pidfile: \/var\/run\/httpd.pid/# pidfile: \/var\/run\/httpd-instance2.pid/g' /etc/init.d/httpd-instance2
   sed -i 's/	CONFFILE=\/etc\/httpd\/conf\/httpd.conf/	CONFFILE=\/etc\/httpd\/conf\/instance2\/httpd.conf/g' /etc/init.d/httpd-instance2
   sed -i 's/HTTPD_LANG=${HTTPD_LANG-"C"}/HTTPD_LANG=${HTTPD_LANG-"C"}\nOPTIONS="-f conf\/instance2\/httpd.conf"/g' /etc/init.d/httpd-instance2
   # Install init.d scripts
   chkconfig --add httpd-instance1
   chkconfig --add httpd-instance2
   # Start them up
   service httpd-instance1 start
   service httpd-instance2 start
 # Install JON Agent from JON Server: http://myjonserver.host.com:7080/agentupdate/download
   mkdir -p /opt/jboss/jon
   cd /tmp && wget -O 'agent-install.jar' http://myjonserver.host.com:7080/agentupdate/download && cd /opt/jboss/jon && java -jar /tmp/agent-install.jar -i
 # Configure and Start JON Agent in DEBUG mode
   cd /opt/jboss/jon/rhq-agent/bin && RHQ_AGENT_DEBUG=true ./rhq-agent.sh
   RHQ 3.0.0.GA [8fb6059] (Wed Jul 28 15:17:44 EDT 2010)
   Answer the following questions to setup this RHQ Agent instance.
   - After each prompt, a default value will appear in square brackets.
     If you press the ENTER key without providing any value,
     the new preference value will be set to that default value.
   - If you wish to rely on the system internal default value and
     not define any preference value, enter '!*'.
   - If you wish to stop before finishing all the questions but still
     retain those preferences you already set, enter '!+'.
   - If you wish to cancel before finishing all the questions and revert
     all preferences back to their original values, enter '!-'.
   - If you need help for a particular preference, enter '!?'.

   Agent Name [myagent.name.host.com] : 
   Agent Hostname or IP Address [!*] : 
   Agent Port [16163] : 
   RHQ Server Hostname or IP Address [192.168.1.25] : myjonserver.host.com
   RHQ Server Port [7080] : 
   The setup has been completed for the preferences at node [/rhq-agent/default].
   sending> ping
   Pinging...
   Ping results: [Ack: PING]
 # From the JON UI, go to the Auto Discovery Queue
   Notice that only one instance is showing as NEW - Apache HTTP Server 192.168.1.25:81	/etc/httpd	Apache Web Server	1/28/11, 2:50:41 AM, EST	NEW
 # Shutdown the agent
   sending> exit
 # Review the Agent log and it is evident that both instances were discovered by only one was imported:
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.plugin.PluginComponentFactory)- Creating discovery component [org.rhq.plugins.apache.ApacheServerDiscoveryComponent] for resource type [Apache HTTP Server]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.plugin.PluginComponentFactory)- Loading class [org.rhq.plugins.apache.ApacheServerDiscoveryComponent] via classloader [PluginClassLoader@1d45d135[parent=PluginClassLoader@3f3cbbbf[parent=PluginClassLoader@326cbb2d[parent=org.rhq.core.pc.plugin.RootPluginClassLoader@36a06e8d,embedded-dir=[<>],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-platform-plugin-3.0.0.GA.jar]],embedded-dir=[file:/opt/jboss/jon/rhq-agent/data/tmp/rhq-augeas-plugin-3.0.0.GA.jar2811380490156212991.classloader/],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-augeas-plugin-3.0.0.GA.jar,augeas-0.0.2.jar2136125507623609348.tmp,jna-3.2.2.jar6081819787812569779.tmp]],embedded-dir=[file:/opt/jboss/jon/rhq-agent/data/tmp/rhq-apache-plugin-3.0.0.GA.jar6629516923495090867.classloader/],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-apache-plugin-3.0.0.GA.jar,snmp4j-1.8.2.jar6881846582328466494.tmp]]]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.plugin.PluginComponentFactory)- Loaded class [class org.rhq.plugins.apache.ApacheServerDiscoveryComponent] from classloader [PluginClassLoader@1d45d135[parent=PluginClassLoader@3f3cbbbf[parent=PluginClassLoader@326cbb2d[parent=org.rhq.core.pc.plugin.RootPluginClassLoader@36a06e8d,embedded-dir=[<>],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-platform-plugin-3.0.0.GA.jar]],embedded-dir=[file:/opt/jboss/jon/rhq-agent/data/tmp/rhq-augeas-plugin-3.0.0.GA.jar2811380490156212991.classloader/],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-augeas-plugin-3.0.0.GA.jar,augeas-0.0.2.jar2136125507623609348.tmp,jna-3.2.2.jar6081819787812569779.tmp]],embedded-dir=[file:/opt/jboss/jon/rhq-agent/data/tmp/rhq-apache-plugin-3.0.0.GA.jar6629516923495090867.classloader/],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-apache-plugin-3.0.0.GA.jar,snmp4j-1.8.2.jar6881846582328466494.tmp]]]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.plugin.PluginComponentFactory)- Created discovery component [org.rhq.plugins.apache.ApacheServerDiscoveryComponent] for resource type [Apache HTTP Server]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Executing process scans for server type ResourceType[id=0, category=Server, name=Apache HTTP Server, plugin=Apache]...
   INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=httpd.*,process|basename|nomatch|parent=httpd.*], name=[HttpdOnUnix]], discovered-process=[process: pid=[3102], name=[/usr/sbin/httpd], ppid=[1]]
   INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=httpd.*,process|basename|nomatch|parent=httpd.*], name=[HttpdOnUnix]], discovered-process=[process: pid=[3079], name=[/usr/sbin/httpd], ppid=[1]]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Executing discovery for [Apache HTTP Server] Resources...
   DEBUG [ResourceDiscoveryComponent.invoker.daemon-2] (org.rhq.plugins.apache.ApacheServerDiscoveryComponent)- Apache executable path: /usr/sbin/httpd
   DEBUG [ResourceDiscoveryComponent.invoker.daemon-2] (org.rhq.plugins.apache.ApacheServerDiscoveryComponent)- Apache Server resource details created: key=/etc/httpd,name=192.168.1.25:81,type=Apache HTTP Server,version=2.2.3,description=Apache Web Server
   DEBUG [ResourceDiscoveryComponent.invoker.daemon-2] (org.rhq.plugins.apache.ApacheServerDiscoveryComponent)- Apache executable path: /usr/sbin/httpd
   DEBUG [ResourceDiscoveryComponent.invoker.daemon-2] (org.rhq.plugins.apache.ApacheServerDiscoveryComponent)- Apache Server resource details created: key=/etc/httpd,name=192.168.1.25:80,type=Apache HTTP Server,version=2.2.3,description=Apache Web Server
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Discovery for [Apache HTTP Server] resources completed in [758] ms
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Detected server Resource[id=0, type=Apache HTTP Server, key=/etc/httpd, name=192.168.1.25:81, parent=<null>, version=2.2.3]
   INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Detected new Server [Resource[id=0, type=Apache HTTP Server, key=/etc/httpd, name=192.168.1.25:81, parent=<null>, version=2.2.3]] - adding to local inventory...
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.plugin.PluginComponentFactory)- Creating discovery component [org.rhq.plugins.apache.ApacheServerDiscoveryComponent] for resource type [Apache HTTP Server]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.plugin.PluginComponentFactory)- Loading class [org.rhq.plugins.apache.ApacheServerDiscoveryComponent] via classloader [PluginClassLoader@1d45d135[parent=PluginClassLoader@3f3cbbbf[parent=PluginClassLoader@326cbb2d[parent=org.rhq.core.pc.plugin.RootPluginClassLoader@36a06e8d,embedded-dir=[<>],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-platform-plugin-3.0.0.GA.jar]],embedded-dir=[file:/opt/jboss/jon/rhq-agent/data/tmp/rhq-augeas-plugin-3.0.0.GA.jar2811380490156212991.classloader/],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-augeas-plugin-3.0.0.GA.jar,augeas-0.0.2.jar2136125507623609348.tmp,jna-3.2.2.jar6081819787812569779.tmp]],embedded-dir=[file:/opt/jboss/jon/rhq-agent/data/tmp/rhq-apache-plugin-3.0.0.GA.jar6629516923495090867.classloader/],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-apache-plugin-3.0.0.GA.jar,snmp4j-1.8.2.jar6881846582328466494.tmp]]]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.plugin.PluginComponentFactory)- Loaded class [class org.rhq.plugins.apache.ApacheServerDiscoveryComponent] from classloader [PluginClassLoader@1d45d135[parent=PluginClassLoader@3f3cbbbf[parent=PluginClassLoader@326cbb2d[parent=org.rhq.core.pc.plugin.RootPluginClassLoader@36a06e8d,embedded-dir=[<>],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-platform-plugin-3.0.0.GA.jar]],embedded-dir=[file:/opt/jboss/jon/rhq-agent/data/tmp/rhq-augeas-plugin-3.0.0.GA.jar2811380490156212991.classloader/],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-augeas-plugin-3.0.0.GA.jar,augeas-0.0.2.jar2136125507623609348.tmp,jna-3.2.2.jar6081819787812569779.tmp]],embedded-dir=[file:/opt/jboss/jon/rhq-agent/data/tmp/rhq-apache-plugin-3.0.0.GA.jar6629516923495090867.classloader/],urls=[file:/opt/jboss/jon/rhq-agent/plugins/rhq-apache-plugin-3.0.0.GA.jar,snmp4j-1.8.2.jar6881846582328466494.tmp]]]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.plugin.PluginComponentFactory)- Created discovery component [org.rhq.plugins.apache.ApacheServerDiscoveryComponent] for resource type [Apache HTTP Server]
   DEBUG [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Server discovery scan took [1206] ms.