Bug 969962 - When the hostname and resolv.conf on the machine change, rhts-sync-set -s DONE in rhts-test-runner.sh blocks indefinitely
When the hostname and resolv.conf on the machine change, rhts-sync-set -s DON...
Status: NEW
Product: Beaker
Classification: Community
Component: beah (Show other bugs)
0.13
Unspecified Unspecified
unspecified Severity unspecified (vote)
: ---
: ---
Assigned To: beaker-dev-list
tools-bugs
: Triaged
: 1045010 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-03 03:45 EDT by Jan Pazdziora
Modified: 2016-05-26 09:17 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jan Pazdziora 2013-06-03 03:45:58 EDT
Description of problem:

I have multihost tests that test DNS servers. Part of that test is to setup new DNS server, have forwarders point to the original nameserver from resolve.conf so that the global resolution still works, defining my own domain (example.com) and changing the hostname of the machine to be withing that example.com domain.

My tests then start to block at the point where the whole test (within a job) is run and the job should move to the next test. I've identified the

   rhts-sync-set -s DONE

call in /usr/bin/rhts-test-runner.sh to be the culprit since it uses the current $HOSTNAME which it identified by running hostname command, and that value points to .example.com, and Marian says that something does not identify correctly that there is a new record in resolv.conf which would be able to resolve that hostname.

Version-Release number of selected component (if applicable):

rhts-test-env-4.55-1.el6eng.noarch

How reproducible:

Deterministic.

Steps to Reproduce:
1. Have two-host job, on one machine create DNS server, have it use resolv.conf's nameserver(s) as forwarders, create two records in the DNS server, point both hosts to that DNS server (edit resolv.conf), change hostnames on that server.
2. Have additional two test after this one in the job.

Actual results:

The job will never move from the first additional test to the second one. Running ps on that machine will show

   rhts-sync-set -s DONE

stuck and blocking.

Expected results:

It should work.

Additional info:

This is mostly about rhts-test-env which I'm not sure belongs to the product Beaker, component test harness. Please realign if there's better place for this bugzilla.
Comment 1 Jan Pazdziora 2013-06-03 03:50:07 EDT
Please consider patches

--- /usr/bin/rhts-environment.sh.orig	2013-04-05 02:55:07.000000000 -0400
+++ /usr/bin/rhts-environment.sh	2013-05-30 13:22:20.481051448 -0400
@@ -30,6 +30,13 @@
    HOSTNAME=$(hostname)
 fi 
 
+if [ -f /etc/original-hostname ]; then
+	export ORIGINAL_HOSTNAME=$( cat /etc/original-hostname )
+else
+	export ORIGINAL_HOSTNAME=$HOSTNAME
+	echo $ORIGINAL_HOSTNAME > /etc/original-hostname
+fi
+
 if [ -z "$ARCH" ]; then
         ARCH=$(uname -i)
 fi
--- /usr/bin/rhts-sync-set.orig	2013-04-05 02:55:07.000000000 -0400
+++ /usr/bin/rhts-sync-set	2013-05-30 13:24:16.242686978 -0400
@@ -80,7 +80,9 @@
         recipesetid = os.environ['RECIPESETID']
     if ('TESTORDER' in os.environ.keys()):
         testorder = os.environ['TESTORDER']
-    if ('HOSTNAME' in os.environ.keys()):
+    if ('ORIGINAL_HOSTNAME' in os.environ.keys()):
+        hostname = os.environ['ORIGINAL_HOSTNAME']
+    elif ('HOSTNAME' in os.environ.keys()):
         hostname = os.environ['HOSTNAME']
 
     args = sys.argv[1:]

which "fixates" the original HOSTNAME in the ORIGINAL_HOSTNAME variable and uses that variable as the default hostname in rhts-sync-set.

Note that this also makes it much easier to check if the machine is part of some role (SERVERS, CLIENTS, whatever you put to the role attribute) by using $ORIGINAL_HOSTNAME while using $HOSTNAME or $(hostname) no longer works.
Comment 2 Jan Pazdziora 2013-06-03 03:51:04 EDT
I might have needed the patch

--- /usr/lib/python2.6/site-packages/beah/tasks/rhts_xmlrpc.py.orig	2013-04-05 02:43:26.000000000 -0400
+++ /usr/lib/python2.6/site-packages/beah/tasks/rhts_xmlrpc.py	2013-05-31 05:28:09.676218239 -0400
@@ -288,7 +288,7 @@
 
     def __init__(self, main):
         self.main = main
-        self.localhosts = (main.env.get('HOSTNAME', ''), os.environ.get('HOSTNAME', ''))
+        self.localhosts = (main.env.get('HOSTNAME', ''), os.environ.get('HOSTNAME', ''), os.environ.get('ORIGINAL_HOSTNAME', ''))
 
     def trhostname(self, hostname):
         if hostname in self.localhosts:

in the past as well but I'm not 100 % sure about that -- I assume developers fluent in the beah code should know for sure.
Comment 4 Dan Callaghan 2013-08-21 19:54:13 EDT
Thanks for the ideas/patches Jan. I think they sound good in theory.

Many tasks are using $HOSTNAME but it's problematic because (a) it's bash-specific, (b) it's set once to whatever the hostname was at the time the shell started, (c) it's *not* the fully-qualified domain name, unless the DHCP config has caused the hostname to be set to a FQDN (which is not always the case).

A better option is $(hostname -f) but as you point out in this bug, very often what the task really wants is the hostname from Beaker's point of view, not whatever glibc thinks the current DNS FQDN is.

I wonder if $BEAKER_HOSTNAME might be a better name than $ORIGINAL_HOSTNAME. We need to consider how it fits in with the other environment variables defined by Beaker.
Comment 5 Jan Pazdziora 2013-08-29 03:21:43 EDT
(In reply to Dan Callaghan from comment #4)
> 
> I wonder if $BEAKER_HOSTNAME might be a better name than $ORIGINAL_HOSTNAME.
> We need to consider how it fits in with the other environment variables
> defined by Beaker.

Sure, BEAKER_HOSTNAME will do just as well.
Comment 6 Dan Callaghan 2013-12-22 18:32:31 EST
*** Bug 1045010 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.