Bug 969962

Summary: When the hostname and resolv.conf on the machine change, rhts-sync-set -s DONE in rhts-test-runner.sh blocks indefinitely
Product: [Retired] Beaker Reporter: Jan Pazdziora <jpazdziora>
Component: beahAssignee: beaker-dev-list
Status: CLOSED WONTFIX QA Contact: tools-bugs <tools-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.13CC: jpazdziora, qwan, sgoveas, tklohna, tools-bugs
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-03 08:52:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Pazdziora 2013-06-03 07:45:58 UTC
Description of problem:

I have multihost tests that test DNS servers. Part of that test is to setup new DNS server, have forwarders point to the original nameserver from resolve.conf so that the global resolution still works, defining my own domain (example.com) and changing the hostname of the machine to be withing that example.com domain.

My tests then start to block at the point where the whole test (within a job) is run and the job should move to the next test. I've identified the

   rhts-sync-set -s DONE

call in /usr/bin/rhts-test-runner.sh to be the culprit since it uses the current $HOSTNAME which it identified by running hostname command, and that value points to .example.com, and Marian says that something does not identify correctly that there is a new record in resolv.conf which would be able to resolve that hostname.

Version-Release number of selected component (if applicable):

rhts-test-env-4.55-1.el6eng.noarch

How reproducible:

Deterministic.

Steps to Reproduce:
1. Have two-host job, on one machine create DNS server, have it use resolv.conf's nameserver(s) as forwarders, create two records in the DNS server, point both hosts to that DNS server (edit resolv.conf), change hostnames on that server.
2. Have additional two test after this one in the job.

Actual results:

The job will never move from the first additional test to the second one. Running ps on that machine will show

   rhts-sync-set -s DONE

stuck and blocking.

Expected results:

It should work.

Additional info:

This is mostly about rhts-test-env which I'm not sure belongs to the product Beaker, component test harness. Please realign if there's better place for this bugzilla.

Comment 1 Jan Pazdziora 2013-06-03 07:50:07 UTC
Please consider patches

--- /usr/bin/rhts-environment.sh.orig	2013-04-05 02:55:07.000000000 -0400
+++ /usr/bin/rhts-environment.sh	2013-05-30 13:22:20.481051448 -0400
@@ -30,6 +30,13 @@
    HOSTNAME=$(hostname)
 fi 
 
+if [ -f /etc/original-hostname ]; then
+	export ORIGINAL_HOSTNAME=$( cat /etc/original-hostname )
+else
+	export ORIGINAL_HOSTNAME=$HOSTNAME
+	echo $ORIGINAL_HOSTNAME > /etc/original-hostname
+fi
+
 if [ -z "$ARCH" ]; then
         ARCH=$(uname -i)
 fi
--- /usr/bin/rhts-sync-set.orig	2013-04-05 02:55:07.000000000 -0400
+++ /usr/bin/rhts-sync-set	2013-05-30 13:24:16.242686978 -0400
@@ -80,7 +80,9 @@
         recipesetid = os.environ['RECIPESETID']
     if ('TESTORDER' in os.environ.keys()):
         testorder = os.environ['TESTORDER']
-    if ('HOSTNAME' in os.environ.keys()):
+    if ('ORIGINAL_HOSTNAME' in os.environ.keys()):
+        hostname = os.environ['ORIGINAL_HOSTNAME']
+    elif ('HOSTNAME' in os.environ.keys()):
         hostname = os.environ['HOSTNAME']
 
     args = sys.argv[1:]

which "fixates" the original HOSTNAME in the ORIGINAL_HOSTNAME variable and uses that variable as the default hostname in rhts-sync-set.

Note that this also makes it much easier to check if the machine is part of some role (SERVERS, CLIENTS, whatever you put to the role attribute) by using $ORIGINAL_HOSTNAME while using $HOSTNAME or $(hostname) no longer works.

Comment 2 Jan Pazdziora 2013-06-03 07:51:04 UTC
I might have needed the patch

--- /usr/lib/python2.6/site-packages/beah/tasks/rhts_xmlrpc.py.orig	2013-04-05 02:43:26.000000000 -0400
+++ /usr/lib/python2.6/site-packages/beah/tasks/rhts_xmlrpc.py	2013-05-31 05:28:09.676218239 -0400
@@ -288,7 +288,7 @@
 
     def __init__(self, main):
         self.main = main
-        self.localhosts = (main.env.get('HOSTNAME', ''), os.environ.get('HOSTNAME', ''))
+        self.localhosts = (main.env.get('HOSTNAME', ''), os.environ.get('HOSTNAME', ''), os.environ.get('ORIGINAL_HOSTNAME', ''))
 
     def trhostname(self, hostname):
         if hostname in self.localhosts:

in the past as well but I'm not 100 % sure about that -- I assume developers fluent in the beah code should know for sure.

Comment 4 Dan Callaghan 2013-08-21 23:54:13 UTC
Thanks for the ideas/patches Jan. I think they sound good in theory.

Many tasks are using $HOSTNAME but it's problematic because (a) it's bash-specific, (b) it's set once to whatever the hostname was at the time the shell started, (c) it's *not* the fully-qualified domain name, unless the DHCP config has caused the hostname to be set to a FQDN (which is not always the case).

A better option is $(hostname -f) but as you point out in this bug, very often what the task really wants is the hostname from Beaker's point of view, not whatever glibc thinks the current DNS FQDN is.

I wonder if $BEAKER_HOSTNAME might be a better name than $ORIGINAL_HOSTNAME. We need to consider how it fits in with the other environment variables defined by Beaker.

Comment 5 Jan Pazdziora 2013-08-29 07:21:43 UTC
(In reply to Dan Callaghan from comment #4)
> 
> I wonder if $BEAKER_HOSTNAME might be a better name than $ORIGINAL_HOSTNAME.
> We need to consider how it fits in with the other environment variables
> defined by Beaker.

Sure, BEAKER_HOSTNAME will do just as well.

Comment 6 Dan Callaghan 2013-12-22 23:32:31 UTC
*** Bug 1045010 has been marked as a duplicate of this bug. ***

Comment 7 Tomas Klohna 🔧 2019-04-03 08:52:03 UTC
Closing the issue due to better harness available. Please use Restraint for your testing purposes. Feel free to reopen if you have the same issue with Restraint as well.