Bug 729755 - Indefinite looping in rhts-compat service preventing the boot from completion
Summary: Indefinite looping in rhts-compat service preventing the boot from completion
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Beaker
Classification: Retired
Component: beah
Version: 0.5
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Nick Coghlan
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-08-10 18:19 UTC by Gurhan Ozen
Modified: 2012-10-29 05:04 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-10-29 05:04:48 UTC
Embargoed:


Attachments (Terms of Use)

Description Gurhan Ozen 2011-08-10 18:19:37 UTC
Description of problem:
While playing around with the virtual guests in beaker, I have run into an issue where I would never be able to get the login prompt in the serial console of the guests because of an indefinite loops at rhts-compat service with the following message:
"ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory"

booting in single user mode and turning off rhts-compat service made things work better.

Below is the complete log from the moment it starts:

Running RHTS-Compatibility Service...
rhts-compat: Waiting for lock.
lockfile: Try praying, giving up on "/var/run/beah/rm.lock"
NOTE: This process runs in foreground.
Use 'service rhts-compat stop' from another terminal to stop it.
It will not stop immediately but only after running tasks are processed.
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
2011-08-10 14:00:40,210 backend linfo: INFO BackendFactory: Started to connect.
2011-08-10 14:00:40,213 backend linfo: INFO BackendFactory: Connection failed. Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectError'>: An error occurred while connecting: 2: No such file or directory.
]
2011-08-10 14:00:40,435 beah start_server: INFO ################################
2011-08-10 14:00:40,436 beah start_server: INFO #   Starting a Controller...   #
2011-08-10 14:00:40,436 beah start_server: INFO ################################
2011-08-10 14:00:40,436 beah start_server: INFO Controller: BackendListener listening on :12432
2011-08-10 14:00:40,447 backend linfo: INFO BackendFactory: Started to connect.
2011-08-10 14:00:40,448 beah start_server: INFO Controller: BackendListener listening on /var/beah/backend12432.socket
2011-08-10 14:00:40,449 beah start_server: INFO Controller: TaskListener listening on 127.0.0.1:12434
2011-08-10 14:00:40,449 beah start_server: INFO Controller: TaskListener listening on /var/beah/task12434.socket
2011-08-10 14:00:40,453 backend linfo: INFO BackendFactory: Connected.  Address: <twisted.internet.address.UNIXAddress object at 0x1881f90>
2011-08-10 14:00:40,453 beah buildProtocol: INFO BackendListener: Connected.  Address: UNIXSocket('')
2011-08-10 14:00:40,454 backend linfo: INFO BackendFactory: Resetting reconnection delay
2011-08-10 14:00:40,454 backend set_controller: INFO Connection to controller lost.
2011-08-10 14:00:40,458 backend set_controller: INFO Connected to controller.
2011-08-10 14:00:40,839 backend nothing_to_do: INFO The recipe has finished.
2011-08-10 14:00:43,040 backend linfo: INFO BackendFactory: Started to connect.
2011-08-10 14:00:43,040 backend linfo: INFO BackendFactory: Connected.  Address: <twisted.internet.address.UNIXAddress object at 0x2af2950>
2011-08-10 14:00:43,041 backend linfo: INFO BackendFactory: Resetting reconnection delay
2011-08-10 14:00:43,041 beah buildProtocol: INFO BackendListener: Connected.  Address: UNIXSocket('')
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory
ls: cannot access /var/run/beah/rhts-compat/launchers: No such file or directory


Version-Release number of selected component (if applicable):
# rpm -qa | egrep "beaker|beah"
beakerlib-1.3-7.el6.noarch
beakerlib-redhat-1-5.el6.noarch
beah-0.6.31-1.el6eso.noarch


How reproducible:
Very?

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Marian Csontos 2011-08-11 04:13:49 UTC
This is a "feature": rhts was working like that and this seemed to be the best way to ensure compatibility with existing tests. We would like to do away with rhts-compat.

For now, rhts-compat can be turned off when necessary:

  https://engineering.redhat.com/trac/rhts/wiki/TestMakefile#RhtsOptions

Comment 2 Marian Csontos 2011-08-11 04:19:39 UTC
Was going to suggest I could stop the rhts-compat service when the recipe is done. This made no sense, as when recipe is finished, machines get recycled. But looks like this is not true for virtual workflows as is the case.

Workaround: Using a short reservesys at the end of a recipe will stop the service.

Comment 3 Gurhan Ozen 2011-08-11 16:33:05 UTC
Marian, thanks for the workaround. I am reassigning because i have a question.

However I have a question about the workaround. Does it work for task only? I have added it on /distribution/virt/install test, but the guests still have it on. Is it possible to make it job-wise or recipe/recipeset wise?

Thanks!

Comment 4 Marian Csontos 2011-08-15 06:00:14 UTC
No, this has to be added task by task.

There is a trick to define system wide parameters:

Add following two lines to /etc/profile.d/task-defaults-rhts.sh:

  export RHTS_OPTION_COMPATIBLE=
  export RHTS_OPTION_COMPAT_SERVICE=

This has to be done on each machine. These values would be overridden by task's parameters if defined.

Comment 5 Nick Coghlan 2012-10-17 04:36:32 UTC
Bulk reassignment of issues as Bill has moved to another team.

Comment 6 Dan Callaghan 2012-10-29 05:04:48 UTC
I don't think there is anything else we need to do here.


Note You need to log in before you can comment on or make changes to this bug.