Description of problem: When scheduling config file deployments for more than 400 Systems using "Deploy all configuration files to selected subscribed systems" or "Deploy all configuration files to all subscribed systems", results in an ISE. Version-Release number of selected component (if applicable): Spacewalk 2.1 with PostgreSQL 9.2 How reproducible: Can be reproduced by creating some hundred duplicate system entries in Spacewalk, it does not matter for this issue, that the config file deployments will never be done. Steps to Reproduce: 3. create or use an existing config channel, and have about 10 config files in it. 2. create 400 System Profiles in Spacewalk by running on a connected system: for i in $(seq 1 400); do rhnreg_ks --force --activationkey=<myactivationkey> ; done Probably a good idea to add the config channel for testing to the activationkey beeing used 2. in WebUI go to Configuration -> Configuration Channels, select the config channel for your tests, then click "Deploy all configuration files to all subscribed systems" or "Deploy all configuration files to selected subscribed systems" and select the systems 3. Click "Confirma & Deploy to selected Systems" Actual results: ISE after 210s, /var/log/httpd/ssl_error_log: [Tue Apr 15 09:35:07 2014] [error] ajp_read_header: ajp_ilink_receive failed [Tue Apr 15 09:35:07 2014] [error] (70007)The timeout specified has expired: proxy: read response failed from [::1]:8009 (localhost) The 210s obviously result from ProxyTimeout 210 in /etc/httpd/conf.d/zz-spacewalk-www.conf Expected results: No ISE, faster scheduling of config deployment actions Additional info: Increasing ProxyTimeout can resolve the ISE, however it took 373s in my tests to complete. I'd consider increasing ProxyTimeout a workaround. It also seems that the available resources on the Spacewalk server are not utilized, the load stayed low while running the tests. Only one CPU is at 100% running tomcat. Also noted that any other Browser tab, eg. clicking on Overview or whatever, is stuck until tomcat finished adding the config deployment actions. I have run the tests with 100, 200, 300 and 400 Systems for comparison, logged all SQL Queries and created pgfouine reports that I'm going to attach. With rising numbers of systems, the performance degrades: # systems ms sys per s 100 23334 4,286 200 91130 2,195 300 200909 1,493 400 373237 1,072
Created attachment 886463 [details] Screenshot with ISE
Created attachment 886465 [details] Screenshot 100 Systems
Created attachment 886466 [details] Screenshot 200 Systems
Created attachment 886468 [details] Screenshot 300 Systems
Created attachment 886473 [details] Screenshot 400 Systems with ProxyTimeout 600
Created attachment 886475 [details] pgfouine report 100 systems
Created attachment 886478 [details] pgfouine report 200 systems
Created attachment 886479 [details] pgfouine report 300 systems
Created attachment 886480 [details] pgfouine report 400 systems
Created attachment 886483 [details] pgfouine report 400 systems with ProxyTimeout 600 When the ISE occurs, tomcat continues to schedule the config deployment actions, they show up under Schedule in the WebUI.
Spacewalk 2.8 (and older) has already reached it's End Of Life. Thank you for reporting this issue and we are sorry that we were not able to fix it before end of life. If you would still like to see this bug fixed and are able to reproduce it against current version of Spacewalk 2.9, you are encouraged change the 'version' and re-open it.