Bug 1353831

Summary: ipa-server-install fails in container because of hostnamectl set-hostname
Product: Red Hat Enterprise Linux 7 Reporter: Jan Pazdziora <jpazdziora>
Component: ipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED ERRATA QA Contact: Kaleem <ksiddiqu>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: jcholast, jpazdziora, mbasti, pvoborni, rcritten
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-4.4.0-6.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 05:56:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Pazdziora 2016-07-08 06:50:36 UTC
Description of problem:

When running https://github.com/adelton/docker-freeipa with RHEL 7.3 nightly, the ipa-server-install fails because in unprivileged container, hostnamectl set-hostname fails.

Version-Release number of selected component (if applicable):

ipa-server-4.4.0-1.el7.x86_64

How reproducible:

Deterministic.

Steps to Reproduce:
1. Build container image based on RHEL 7.3 nightly repo / upgraded to this content.
2. Run the container: docker run -t --name freeipa-server-container -h ipa.example.test -v /dev/urandom:/dev/random:ro -v /opt/ipa-data-rhel-7.3-nightly:/data -v /sys/fs/cgroup:/sys/fs/cgroup:ro  --cap-add=SYS_TIME  freeipa-server

Actual results:

systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization docker.
Detected architecture x86-64.
Set hostname to <ipa.example.test>.
Thu Jul  7 16:14:35 EDT 2016 /usr/sbin/ipa-server-configure-first 
systemd-tmpfiles-setup.service: main process exited, code=exited, status=1/FAILURE
Job container-ipa.target/start failed with result 'dependency'.
Unit systemd-tmpfiles-setup.service entered failed state.
systemd-tmpfiles-setup.service failed.
ipa         : ERROR    Failed to set this machine hostname to ipa.example.test (Command '/bin/hostnamectl set-hostname ipa.example.test' returned non-zero exit status 1).
ipa.ipapython.install.cli.install_tool(Server): ERROR    Command '/bin/hostnamectl set-hostname ipa.example.test' returned non-zero exit status 1
ipa.ipapython.install.cli.install_tool(Server): ERROR    The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information
Checking DNS domain example.test, please wait ...

The log file for this installation can be found in /var/log/ipaserver-install.log
==============================================================================
This program will set up the IPA Server.

This includes:
  * Configure a stand-alone CA (dogtag) for certificate management
  * Configure the Network Time Daemon (ntpd)
  * Create and configure an instance of Directory Server
  * Create and configure a Kerberos Key Distribution Center (KDC)
  * Configure Apache (httpd)
  * Configure DNS (bind)

Warning: skipping DNS resolution of host ipa.example.test
Checking DNS domain example.test., please wait ...
Checking DNS forwarders, please wait ...

The IPA Master Server will be configured with:
Hostname:       ipa.example.test
IP address(es): 172.17.0.2
Domain name:    example.test
Realm name:     EXAMPLE.TEST

BIND DNS server will be configured to serve IPA domain with:
Forwarders:       10.16.36.29
Forward policy:   only
Reverse zone(s):  No reverse zone

FreeIPA server configuration failed.

Expected results:

No error.

Additional info:

The problem seems to come from change c5686295f14c955d34d9598ddb80b30cb9df663c.

Note that in containers, we rely on the hostname being set from "outside", in docker run case with -h option. The installer should not run hostnamectl when the hostname is already set to the required value.

Comment 4 Petr Vobornik 2016-07-12 16:17:47 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6071

Comment 7 Jan Pazdziora 2016-09-08 13:24:35 UTC
Things work when --hostname option is not passed.

However, when the container is run with

docker run -t --name freeipa-server-container -h ipa.example.test -v /dev/urandom:/dev/random:ro -v /opt/ipa-data-rhel-7.3-nightly:/data -v /sys/fs/cgroup:/sys/fs/cgroup:ro  --cap-add=SYS_TIME  freeipa-server --hostname ipa.example.test

and even when the --hostname parameter value matches the hostname set in the container by -h, I get

The IPA Master Server will be configured with:
Hostname:       ipa.example.test
IP address(es): 172.17.0.2
Domain name:    example.test
Realm name:     EXAMPLE.TEST

BIND DNS server will be configured to serve IPA domain with:
Forwarders:       10.10.160.1
Forward policy:   only
Reverse zone(s):  No reverse zone

ipa.ipapython.install.cli.install_tool(Server): ERROR    Command '/bin/hostnamectl set-hostname ipa.example.test' returned non-zero exit status 1
ipa.ipapython.install.cli.install_tool(Server): ERROR    The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information

with /var/log/ipaserver-install.log showing

2016-09-08T13:21:56Z DEBUG Checking DNS server: 10.10.160.1
2016-09-08T13:21:56Z DEBUG will use DNS forwarders: [CheckedIPAddress('10.10.160.1')]

2016-09-08T13:21:56Z DEBUG Backing up system configuration file '/etc/hostname'
2016-09-08T13:21:56Z DEBUG Saving Index File to '/var/lib/ipa/sysrestore/sysrestore.index'
2016-09-08T13:21:56Z DEBUG Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state'
2016-09-08T13:21:56Z DEBUG Saving StateFile to '/var/lib/ipa/sysrestore/sysrestore.state'
2016-09-08T13:21:56Z DEBUG Starting external process
2016-09-08T13:21:56Z DEBUG args=/bin/hostnamectl set-hostname ipa.example.test
2016-09-08T13:22:21Z DEBUG Process finished, return code=1
2016-09-08T13:22:21Z DEBUG stdout=
2016-09-08T13:22:21Z DEBUG stderr=Failed to open /dev/tty: No such device or address
Could not set property: Activation of org.freedesktop.hostname1 timed out

2016-09-08T13:22:21Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318, in run
    cfgr.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310, in run
    self.execute()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332, in execute
    for nothing in self._executor():
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 586, in _configure
    next(executor)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 449, in _handle_exception
    self.__parent._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 446, in _handle_exception
    super(ComponentBase, self)._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 63, in _install
    for nothing in self._installer(self.parent):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 1357, in main
    install(self)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 267, in decorated
    func(installer)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 711, in install
    tasks.set_hostname(host_name)
  File "/usr/lib/python2.7/site-packages/ipaplatform/redhat/tasks.py", line 473, in set_hostname
    ipautil.run([paths.BIN_HOSTNAMECTL, 'set-hostname', hostname])
  File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 494, in run
    raise CalledProcessError(p.returncode, arg_string, str(output))

2016-09-08T13:22:21Z DEBUG The ipa-server-install command failed, exception: CalledProcessError: Command '/bin/hostnamectl set-hostname ipa.example.test' returned non-zero exit status 1
2016-09-08T13:22:21Z ERROR Command '/bin/hostnamectl set-hostname ipa.example.test' returned non-zero exit status 1
2016-09-08T13:22:21Z ERROR The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information

Comment 8 Petr Vobornik 2016-09-08 14:36:22 UTC
According to the fix it is expected. I.e.
* Fix was to stop running hostnamectl every time. 
* It is expected that `hostnamectl set-hostname` is run when --hostname option is provided.

But man pages says something different:

--hostname=HOST_NAME
              The fully-qualified DNS name of this server. If the hostname does not match system hostname, the system hostname will be updated accordingly to prevent service failures.

I'm not sure if current behavior is a regression to behavior in RHEL 7.2

Comment 11 Martin Bašti 2016-09-12 08:03:39 UTC
Man page should be updated, when --hostname option is used installer will always use hostnamectl set-hostname

Comment 12 Jan Pazdziora 2016-09-12 08:05:54 UTC
This should be evaluated in the context of workaround needed to be carried in IdM container indefinitely then.

Comment 13 Petr Vobornik 2016-09-12 10:38:47 UTC
Jan what do you mean by the workaround?

Is there any reason to use --hostname option in a container? Or do you want a workaround for the case when somebody uses the option, e.g. by accident.

Comment 14 Jan Pazdziora 2016-09-13 04:33:26 UTC
(In reply to Petr Vobornik from comment #13)
> Jan what do you mean by the workaround?

Forcing hostnamectl a symlink to /bin/true to stop it from failing.

> Is there any reason to use --hostname option in a container? Or do you want
> a workaround for the case when somebody uses the option, e.g. by accident.

We need a way to name the container and using --hostname seemed like the best way to do that. So we parse options that are passed and use the --hostname value. We potentially could remove it from the list of options that are later passed to ipa-server-install but we got burned in the past by trying to be too clever and diverging the in-container behaviour from the on-host one, so I'd rather not do that. Users will likely be confused when they use the option and then will not see it set in the logs, for example.

Comment 15 Petr Vobornik 2016-09-13 13:41:08 UTC
I agree that removal of the option in process might confuse users.

Why not use separate, container specific option for the container name?

Comment 16 Petr Vobornik 2016-09-13 15:21:14 UTC
For the container name/--hostname issue I've file new bug 1375648. 

Let's continue with checking the fix described in comment 8

For the man page change, ticket https://fedorahosted.org/freeipa/ticket/6330 was opened.

Comment 17 Kaleem 2016-09-23 14:13:59 UTC
Verified SanityOnly on the basis of #comment16 and ipa-server-install with/without hostname parameter.

IPA version:
============
[root@dhcp207-129 ~]# rpm -q ipa-server
ipa-server-4.4.0-12.el7.x86_64
[root@dhcp207-129 ~]#

Comment 19 errata-xmlrpc 2016-11-04 05:56:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2404.html