Bug 766084 - spacewalk-setup Hangs Restarting Services
Summary: spacewalk-setup Hangs Restarting Services
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Spacewalk
Classification: Community
Component: Installation
Version: 1.4
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Jan Pazdziora
QA Contact: Red Hat Satellite QA List
URL:
Whiteboard:
Depends On:
Blocks: space17
TreeView+ depends on / blocked
 
Reported: 2011-12-10 00:44 UTC by John Devereaux
Modified: 2012-01-18 09:38 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-01-18 09:38:16 UTC
Embargoed:


Attachments (Terms of Use)
logs pertaining to spacewalk 1.4 setup issue on FC14 x86_64 (229.28 KB, application/x-zip-compressed)
2011-12-10 00:45 UTC, John Devereaux
no flags Details

Description John Devereaux 2011-12-10 00:44:13 UTC
Description of problem:

Service Restarting Hangs after finishing up 'spacewalk-setup --disconnect'
Service Temporarily Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Apache Server at 141.144.112.204 Port 443

Version-Release number of selected component (if applicable):

spacewalk-setup-jabberd-1.3.2-1.fc14.noarch
spacewalk-setup-1.4.9-1.fc14.noarch
Fedora release 14 (Laughlin) (x86_64)


How reproducible:


Steps to Reproduce:
1. Follow directions for Spacewalk 1.4 installation with the use of Oracle XE      dB. 
2. Execute 'spacewalk-setup --disconnect'
3. Experienced httpd / Tomcat / jabberd issues
  
Actual results:

Service Temporarily Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Apache Server at 141.144.112.204 Port 443

Expected results:


Additional info:
During waiting for spacewalk-setup to complete, inspected jabberd processes and during a terminal session seen the following error: 


Starting spacewalk services...
Initializing jabberd processes ...
Starting router:                                           [  OK  ]
Starting sm:                                               [  OK  ]
Starting c2s:                                              [  OK  ]
Starting s2s:                                              [  OK  ]
Starting osa-dispatcher: RHN 10147 2011/12/09 19:23:50 -04:00: ('Server did not return a <features /> stanza',)
RHN 10147 2011/12/09 19:23:50 -04:00: ('Traceback (most recent call last):\n  File "/usr/share/rhn/osad/jabber_lib.py", line 253, in setup_connection\n    c = self._get_jabber_client(js)\n  File "/usr/share/rhn/osad/jabber_lib.py", line 310, in _get_jabber_client\n    c.connect()\n  File "/usr/share/rhn/osad/jabber_lib.py", line 583, in connect\n    raise SSLDisabledError\nSSLDisabledError\n',)


[root@spacewalk-FC14-x86-64 ~]# /usr/sbin/spacewalk-service status
router (pid 10983) is running...
sm (pid 10990) is running...
c2s (pid 10997) is running...
s2s (pid 11004) is running...
osa-dispatcher (pid  11027) is running...
tomcat6 (pid 11091) is running...                          [  OK  ]
httpd (pid  12512) is running...
rhn-search is running (10739).
cobblerd (pid 10782) is running...
RHN Taskomatic is not running.
[root@spacewalk-FC14-x86-64 ~]#

Comment 1 John Devereaux 2011-12-10 00:45:31 UTC
Created attachment 544812 [details]
logs pertaining to spacewalk 1.4 setup issue on FC14 x86_64

Comment 2 John Devereaux 2011-12-10 01:01:07 UTC
1. Reboot FC14 Server after executing /usr/sbin/spacewalk-service stop
2. after reboot, executed /usr/sbin/spacewalk-service restart:


[root@spacewalk-FC14-x86-64 ~]# /usr/sbin/spacewalk-service restart
Shutting down spacewalk services...
Stopping RHN Taskomatic...
RHN Taskomatic was not running.
Stopping cobbler daemon:                                   [  OK  ]
Stopping rhn-search...
Stopped rhn-search.
Stopping MonitoringScout ...
[ OK ]
Stopping Monitoring ...
[ OK ]
Stopping httpd:                                            [  OK  ]
Stopping tomcat6:                                          [  OK  ]
Shutting down osa-dispatcher:                              [  OK  ]
Terminating jabberd processes ...
Stopping router:                                           [  OK  ]
Stopping sm:                                               [  OK  ]
Stopping c2s:                                              [  OK  ]
Stopping s2s:                                              [  OK  ]
Done.
Starting spacewalk services...
Initializing jabberd processes ...
Starting router:                                           [  OK  ]
Starting sm:                                               [  OK  ]
Starting c2s:                                              [  OK  ]
Starting s2s:                                              [  OK  ]
Starting osa-dispatcher: RHN 2824 2011/12/09 19:59:22 -04:00: ('Server did not return a <features /> stanza',)
RHN 2824 2011/12/09 19:59:22 -04:00: ('Traceback (most recent call last):\n  File "/usr/share/rhn/osad/jabber_lib.py", line 253, in setup_connection\n    c = self._get_jabber_client(js)\n  File "/usr/share/rhn/osad/jabber_lib.py", line 310, in _get_jabber_client\n    c.connect()\n  File "/usr/share/rhn/osad/jabber_lib.py", line 583, in connect\n    raise SSLDisabledError\nSSLDisabledError\n',)
                                                           [  OK  ]
Starting tomcat6:                                          [  OK  ]
Waiting for tomcat to be ready ...

Seems to be an issue with tomcast ... cmd line just hangs here...

Comment 3 Šimon Lukašík 2011-12-10 10:34:51 UTC
Please do not attempt to describe multiple issues in one ticket.

Let's track the jabberd issue in bug 759406.

As for tomcat issue: Fedora 14 is end of life. Neither Spacewalk 1.4 is
the latest version. Would You mind to reproduce this on a newer one?
Let's say with Spacewalk 1.5 on Fedora 15?

Comment 4 John Devereaux 2011-12-10 15:56:16 UTC
understood, i will rebuild today and update the bug report

Comment 5 John Devereaux 2011-12-11 02:21:54 UTC

[root@spacewalk-FC14-x86-64 ~]# /usr/sbin/spacewalk-service status
jabberd.service - SYSV: jabberd is the next generation of the jabberd server
          Loaded: loaded (/etc/rc.d/init.d/jabberd)
          Active: active (running) since Sat, 10 Dec 2011 21:02:11 -0500; 15min ago
         Process: 1160 ExecStart=/etc/rc.d/init.d/jabberd start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/jabberd.service
                  â 1170 /usr/bin/router -c /etc/jabberd//router.xml
                  â 1177 /usr/bin/sm -c /etc/jabberd//sm.xml
                  â 1184 /usr/bin/c2s -c /etc/jabberd//c2s.xml
                  â 1191 /usr/bin/s2s -c /etc/jabberd//s2s.xml
jabberd.service - SYSV: jabberd is the next generation of the jabberd server
          Loaded: loaded (/etc/rc.d/init.d/jabberd)
          Active: active (running) since Sat, 10 Dec 2011 21:02:11 -0500; 15min ago
         Process: 1160 ExecStart=/etc/rc.d/init.d/jabberd start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/jabberd.service
                  â 1170 /usr/bin/router -c /etc/jabberd//router.xml
                  â 1177 /usr/bin/sm -c /etc/jabberd//sm.xml
                  â 1184 /usr/bin/c2s -c /etc/jabberd//c2s.xml
                  â 1191 /usr/bin/s2s -c /etc/jabberd//s2s.xml
jabberd.service - SYSV: jabberd is the next generation of the jabberd server
          Loaded: loaded (/etc/rc.d/init.d/jabberd)
          Active: active (running) since Sat, 10 Dec 2011 21:02:11 -0500; 15min ago
         Process: 1160 ExecStart=/etc/rc.d/init.d/jabberd start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/jabberd.service
                  â 1170 /usr/bin/router -c /etc/jabberd//router.xml
                  â 1177 /usr/bin/sm -c /etc/jabberd//sm.xml
                  â 1184 /usr/bin/c2s -c /etc/jabberd//c2s.xml
                  â 1191 /usr/bin/s2s -c /etc/jabberd//s2s.xml
jabberd.service - SYSV: jabberd is the next generation of the jabberd server
          Loaded: loaded (/etc/rc.d/init.d/jabberd)
          Active: active (running) since Sat, 10 Dec 2011 21:02:11 -0500; 15min ago
         Process: 1160 ExecStart=/etc/rc.d/init.d/jabberd start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/jabberd.service
                  â 1170 /usr/bin/router -c /etc/jabberd//router.xml
                  â 1177 /usr/bin/sm -c /etc/jabberd//sm.xml
                  â 1184 /usr/bin/c2s -c /etc/jabberd//c2s.xml
                  â 1191 /usr/bin/s2s -c /etc/jabberd//s2s.xml
osa-dispatcher.service - LSB: This shell script starts the osa-dispatcher daemon
          Loaded: loaded (/etc/rc.d/init.d/osa-dispatcher)
          Active: active (running) since Sat, 10 Dec 2011 21:02:17 -0500; 15min ago
         Process: 1193 ExecStart=/etc/rc.d/init.d/osa-dispatcher start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/osa-dispatcher.service
                  â 1294 /usr/bin/python /usr/sbin/osa-dispatcher --pid-f...
tomcat6 (pid 1152) is running...                           [  OK  ]
httpd.service - LSB: start and stop Apache HTTP Server
          Loaded: loaded (/etc/rc.d/init.d/httpd)
          Active: active (running) since Sat, 10 Dec 2011 21:02:17 -0500; 15min ago
         Process: 1038 ExecStart=/etc/rc.d/init.d/httpd start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/httpd.service
                  â 1292 /usr/sbin/httpd
                  â 1302 /usr/bin/perl /usr/share/rhn/satidmap.pl
                  â 1336 /usr/sbin/httpd
                  â 1337 /usr/sbin/httpd
                  â 1338 /usr/sbin/httpd
                  â 1339 /usr/sbin/httpd
                  â 1340 /usr/sbin/httpd
                  â 1344 /usr/sbin/httpd
                  â 1345 /usr/sbin/httpd
                  â 1349 /usr/sbin/httpd
rhn-search is running (1347).
cobblerd.service - LSB: daemon for libvirt virtualization API
          Loaded: loaded (/etc/rc.d/init.d/cobblerd)
          Active: active (running) since Sat, 10 Dec 2011 21:02:23 -0500; 14min ago
         Process: 1301 ExecStart=/etc/rc.d/init.d/cobblerd start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/cobblerd.service
                  â 1515 /usr/bin/python /usr/bin/cobblerd --daemonize
RHN Taskomatic is not running.

----------------------------

restarted the spacewalk-service and I keep getting the following error:

HTTP Status 404 -

type Status report

message

description The requested resource () is not available.

Apache Tomcat/6.0.32

Comment 6 John Devereaux 2011-12-11 02:34:27 UTC
To add I am using postgresql installation method to test to compare against Oracle dB installation.

Comment 7 John Devereaux 2011-12-11 14:56:06 UTC
Please note: I am enroute to Oracle HQ until my red-eye on Thursday. I will resume testing on Friday mid-morning.

Comment 8 Jan Pazdziora 2011-12-12 08:20:50 UTC
(In reply to comment #2)
>                                                            [  OK  ]
> Starting tomcat6:                                          [  OK  ]
> Waiting for tomcat to be ready ...
> 
> Seems to be an issue with tomcast ... cmd line just hangs here...

http://www.redhat.com/archives/spacewalk-list/2011-September/msg00164.html

What is in the catalina.out? What does lsof report on that tomcat process (what ports does tomcat listen on)? How about Spacewalk 1.5?

Comment 9 Jan Pazdziora 2011-12-12 08:21:44 UTC
(In reply to comment #5)
> 
> [root@spacewalk-FC14-x86-64 ~]# /usr/sbin/spacewalk-service status
> jabberd.service - SYSV: jabberd is the next generation of the jabberd server
>           Loaded: loaded (/etc/rc.d/init.d/jabberd)
>           Active: active (running) since Sat, 10 Dec 2011 21:02:11 -0500; 15min
> ago
>          Process: 1160 ExecStart=/etc/rc.d/init.d/jabberd start (code=exited,
> status=0/SUCCESS)
>           CGroup: name=systemd:/system/jabberd.service
>                   â 1170 /usr/bin/router -c /etc/jabberd//router.xml
>                   â 1177 /usr/bin/sm -c /etc/jabberd//sm.xml

How / why do you have systemd on Fedora 14?

Comment 10 Jan Pazdziora 2012-01-18 09:38:16 UTC
We seem to have communication lost here. In any case, Fedora 14 is no longer supported platform for Spacewalk 1.6. Please reopen if the issue can be reproduced on latest Fedora version.


Note You need to log in before you can comment on or make changes to this bug.