Bug 1540641

Summary: httpd-configmap-generator shipped with podified cfme cannot be used for generating ipa ext auth config
Product: Red Hat CloudForms Management Engine Reporter: Ievgen Zapolskyi <izapolsk>
Component: DocumentationAssignee: Dayle Parker <dayleparker>
Status: CLOSED CURRENTRELEASE QA Contact: Ievgen Zapolskyi <izapolsk>
Severity: high Docs Contact: Steve Bream <sbream>
Priority: high    
Version: 5.9.0CC: adahms, cpelland, dayleparker, hhudgeon, izapolsk, jprause, jvlcek, lavenel, obarenbo, simaishi, ssainkar
Target Milestone: GA   
Target Release: 5.9.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
The hostnamectl tool, as used by the ipa-client-install command and in the appliance console, is limited by the 'HOST_NAME_MAX (64)' kernel variable. Because of this limitation in the underlying technology, Red Hat CloudForms cannot support fully qualified domain names longer than 64 characters.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-05 03:42:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core Target Upstream Version:
Embargoed:

Description Ievgen Zapolskyi 2018-01-31 15:25:46 UTC
Description of problem:
when httpd-configmap-generator is kicked off it fails on changing host name. Look at backtrace below
It doesn't matter whether it belongs to anyuid or privileged scc.
So, it seems pod requires some special right

Version-Release number of selected component (if applicable):
5.9.0.18

How reproducible:
100%

Steps to Reproduce:
Follow steps from https://github.com/ManageIQ/httpd_configmap_generator
under "Deploy the Httpd Configmap Generator Application" and "Generating a configmap for external authentication against IPA" sections

Actual results:
httpd-configmap-generator fails on attempt to set hostname 

Expected results:
configuration should be generated


Additional info:
[root@cfme-smicro-628-b09 cfme-59018-0126]# oc exec $CONFIGMAP_GENERATOR_POD -- bash -c 'httpd_configmap_generator ipa --host=*** --ipa-server=*** --ipa-domain=** --ipa-realm=*** --ipa-principal=admin --ipa-password=**** -f -d -o /tmp/external-ipa.yaml'
Running Command: /usr/bin/hostname
Running Command: /usr/bin/hostname --params ***
Command Error: /usr/bin/hostname exit code: 1
stdout:
stderr: hostname: you must be root to change the host name
/opt/httpd_configmap_generator/vendor/bundle/ruby/2.3.0/gems/awesome_spawn-1.4.1/lib/awesome_spawn.rb:105:in `run!': /usr/bin/hostname exit code: 1 (AwesomeSpawn::CommandResultError)
        from /opt/httpd_configmap_generator/lib/httpd_configmap_generator/base/command.rb:17:in `command_run!'
        from /opt/httpd_configmap_generator/lib/httpd_configmap_generator/base/network.rb:24:in `update_hostname'
        from /opt/httpd_configmap_generator/lib/httpd_configmap_generator/ipa.rb:52:in `configure'
        from /opt/httpd_configmap_generator/lib/httpd_configmap_generator/base.rb:83:in `run_configure'
        from /opt/rh/rh-ruby23/root/usr/bin/httpd_configmap_generator:68:in `run'
        from /opt/rh/rh-ruby23/root/usr/bin/httpd_configmap_generator:87:in `<main>'
command terminated with exit code 1

Comment 2 Ievgen Zapolskyi 2018-01-31 15:26:31 UTC
btw, I don't have such error in upstream docker container

Comment 3 Ievgen Zapolskyi 2018-01-31 15:50:36 UTC
I figured out the reason - hostname is too long.
echo cfme-59018-0126-project-xz3tto.ocp-sprout-01.cmqe.lab.eng.rdu2.redhat.com |wc -c  = 74
When I make it echo xz3tto.ocp-sprout-01.cmqe.lab.eng.rdu2.redhat.com |wc -c  = 50, hostname doesn't fail.

Can we handle this somehow ?

Comment 4 Ievgen Zapolskyi 2018-01-31 15:57:21 UTC
(In reply to Ievgen Zapolskyi from comment #3)
> I figured out the reason - hostname is too long.
> echo
> cfme-59018-0126-project-xz3tto.ocp-sprout-01.cmqe.lab.eng.rdu2.redhat.com
> |wc -c  = 74
> When I make it echo xz3tto.ocp-sprout-01.cmqe.lab.eng.rdu2.redhat.com |wc -c
> = 50, hostname doesn't fail.
> 
> Can we handle this somehow ?
sorry, this helps only with docker. same thing with pod

Comment 5 abellott 2018-02-02 15:51:46 UTC
The limitation comes from the Linux kernel's getconf HOST_NAME_MAX of 64 characters. Problem also exists with the appliance console and how the hostname is setup, so it's not just the configmap generator. JoeV and I are looking further into the issue.

Comment 6 abellott 2018-02-02 17:30:58 UTC
Brief:

Upon researching this further, JoeV and I found that the limitation of the hostname limitation is rooted in the underlying kernel/system commands. 

Therefore we feel this should not be a blocker.


Details:
 
The system hostnamectl tool, as used by the IPA client install command as well as the appliance console, is limited by the kernel's HOST_NAME_MAX (64). Due to these limitations of the underlying technologies, ManageIQ/CloudForms is unable to support FQDN hostnames longer than 64 characters.

Comment 8 Ievgen Zapolskyi 2018-02-05 10:09:59 UTC
I encountered that limitation in both docker and pod. 
So, I tried to make dns name shorter.
xz3tto.ocp-sprout-01.cmqe.lab.eng.rdu2.redhat.com |wc -c  = 50
The issue disappeared in docker. 
However, I still face it in pod. 
So, I don't think this is related to host name length.

Comment 9 Joe Vlcek 2018-02-05 14:31:34 UTC
(In reply to Ievgen Zapolskyi from comment #8)
> I encountered that limitation in both docker and pod. 
> So, I tried to make dns name shorter.
> xz3tto.ocp-sprout-01.cmqe.lab.eng.rdu2.redhat.com |wc -c  = 50
> The issue disappeared in docker. 
> However, I still face it in pod. 
> So, I don't think this is related to host name length.

Ievgen,

Please provide more information about the exact failure you are now
encountering in the pod. I suspect you may be encountering something
unrelated to the hostname length issue.

Thank you, JoeV

Comment 10 Ievgen Zapolskyi 2018-02-05 15:31:02 UTC
nope, I tried to call hostname with new host name in pod and got the same error.
I'll provide you project where you can reproduce and investigate this issue.

Comment 11 Joe Vlcek 2018-02-06 15:01:55 UTC
I looked into this with Ievgen. If he deploys a build of the httpd pod from upstream he does not encounter this failure.

We have decided to wait until there is a usable downstream CFME build and retest to see if the issue is not resolved in the latest downstream build.

Ievgen Please retest with the latest good downstream build once available.

Comment 12 Joe Vlcek 2018-02-06 15:01:55 UTC
I looked into this with Ievgen. If he deploys a build of the httpd pod from upstream he does not encounter this failure.

We have decided to wait until there is a usable downstream CFME build and retest to see if the issue is not resolved in the latest downstream build.

Ievgen Please retest with the latest good downstream build once available.

Comment 13 Ievgen Zapolskyi 2018-02-20 14:02:14 UTC
Joe, I see the same issue in 5.9.0.21

Comment 14 Joe Vlcek 2018-02-28 21:07:26 UTC
(In reply to Ievgen Zapolskyi from comment #13)
> Joe, I see the same issue in 5.9.0.21

Ievgen,

Just FYI.

I just wanted to let you know that I've been able to reproduce this issue, which
will be a big help in finding out what's wrong and how to fix it.

Stay tuned.

JoeV

Comment 15 Joe Vlcek 2018-02-28 21:34:53 UTC
(In reply to Ievgen Zapolskyi from comment #13)
> Joe, I see the same issue in 5.9.0.21

Ievgen,

Is the traceback the same with 5.9.0.21? Does the traceback still contain:

/opt/httpd_configmap_generator/lib/httpd_configmap_generator/base/network.rb:24:in `update_hostname'

JoeV

Comment 16 Joe Vlcek 2018-02-28 21:56:20 UTC
(In reply to Joe Vlcek from comment #14)
> (In reply to Ievgen Zapolskyi from comment #13)
> > Joe, I see the same issue in 5.9.0.21
> 
> Ievgen,
> 
> Just FYI.
> 
> I just wanted to let you know that I've been able to reproduce this issue,
> which
> will be a big help in finding out what's wrong and how to fix it.
> 
> Stay tuned.
> 
> JoeV

Sorry for the confusion Ievgen but I had an error in my httpd_configmap_generator
command that resulted in a different traceback from the one you are reporting.
Once I correct the error on my httpd_configmap_generator the configuration worked
fine for me.

JoeV

Comment 17 Ievgen Zapolskyi 2018-03-01 11:02:47 UTC
Joe, 
Please contact me when you are at work. I'll provide you environment with such failing pod.

Comment 18 Joe Vlcek 2018-03-06 22:04:03 UTC
Ievgen,

The configmap generator instructions regarding the Security Context Constraints (SCC) may very well be incorrect for Openshift.

They direct the user to configure the httpd-configmap-generator service account to be included in the "anyuid" SCC. This is not necessary.

Instead the httpd-configmap-generator service account is needs to be included in the "httpd-scc-sysadmin" SCC, as the instructions indicate for minishift.

Can you please try removing the httpd-configmap-generator service account from the the "anyuid" SCC and adding it to the httpd-scc-sysadmin" SCC, as follows?:

oc adm policy remove-scc-from-user anyuid system:serviceaccount:<your-namespace>:httpd-configmap-generator
oc adm policy remove-scc-from-user anyuid system:serviceaccount:<your-namespace>
oc adm policy add-scc-to-user httpd-scc-sysadmin system:serviceaccount:<your-namespace>:httpd-configmap-generator

Thank you for all your help and patience with this!

JooeV

Comment 19 Ievgen Zapolskyi 2018-03-07 17:30:22 UTC
Joe, 
 both hostname and http_configmap_generator don't encounter that issue with cluster-admin rights and sys_admin capability.
So, I think we can update upstream and downstream documentation with correct steps.

However, I ran into another issue which I had in docker before - https://github.com/ManageIQ/httpd_configmap_generator/issues/25.
I didn't manage to submit it before since downstream configmap generator didn't allow me to reach it.
Should I submit another BZ regarding it ?

Comment 20 Joe Vlcek 2018-03-08 12:12:15 UTC
(In reply to Ievgen Zapolskyi from comment #19)
> Joe, 
>  both hostname and http_configmap_generator don't encounter that issue with
> cluster-admin rights and sys_admin capability.
> So, I think we can update upstream and downstream documentation with correct
> steps.
> 
> However, I ran into another issue which I had in docker before -
> https://github.com/ManageIQ/httpd_configmap_generator/issues/25.
> I didn't manage to submit it before since downstream configmap generator
> didn't allow me to reach it.
> Should I submit another BZ regarding it ?

Thank you Ievgen.

I'll update the documentation with the correct SCC steps.

Regarding https://github.com/ManageIQ/httpd_configmap_generator/issues/25
Are you only seeing that on docker are are you seeing it with Openshift now too?

Please file a BZ and include the answer.

Thank you again for all the help!
JoeV

Comment 21 Ievgen Zapolskyi 2018-03-09 14:47:02 UTC
awesome, thank you, Joe!
I put in another ticket regarding that ipa issue - https://bugzilla.redhat.com/show_bug.cgi?id=1553790.

Comment 25 Dayle Parker 2018-05-30 02:07:06 UTC
After investigating further, it looks like this docs bug will include several changes downstream for the 4.6 documentation:

* Since there is no content on configuring external authentication in the downstream 'podified CloudForms' guide (Installing CloudForms on OpenShift Container Platform [1]), we will need to add the content in https://github.com/ManageIQ/manageiq-pods#configuring-external-authentication.

* There also is no content on automatically generating authentication configuration maps for the httpd pod in the downstream 'podified CloudForms' guide (Installing CloudForms on OpenShift Container Platform). We will need to add the relevant content from https://github.com/ManageIQ/httpd_configmap_generator.

I will seek SME review for accuracy and support status once the content is in.

[1] https://access.redhat.com/documentation/en-us/red_hat_cloudforms/4.6/html-single/installing_red_hat_cloudforms_on_openshift_container_platform/

Comment 30 Dayle Parker 2018-06-15 08:21:28 UTC
Hi Ievgen and Joe,

Previously, we didn't have any instructions to configure external authentication in the Installing CloudForms on OpenShift guide, so I've added and adapted the content from upstream in [1] and [2].

Can you please review the following (in the attached PDF in Comment 29) and let me know if any steps are missing or need corrections?

* CHAPTER 2. CONFIGURING EXTERNAL AUTHENTICATION TO CLOUDFORMS
* APPENDIX A. APPENDIX

A few questions:
* The docker and OpenShift directions look similar (in the upstream docs), so I've deleted the docker instructions for now. Should the gem install instructions remain in the CloudForms guide?

* I found the cfme-httpd-configmap-generator container image on registry.redhat.com, so I imagine this is the image to tell customers to download. Does that step look correct? And to import the image, should you first log in as an admin user to OCP?

* Will the templates be obtained by the customers in the image stream configured in section "1.1.3. Templates and Images" of this guide?

Thanks for your review and comments.
Cheers,
Dayle

[1]https://github.com/ManageIQ/httpd_configmap_generator 
[2] https://github.com/ManageIQ/manageiq-pods#configuring-external-authentication.

Comment 32 Joe Vlcek 2018-06-15 13:10:20 UTC
(In reply to Dayle Parker from comment #30)
> Hi Ievgen and Joe,
> 
> Previously, we didn't have any instructions to configure external
> authentication in the Installing CloudForms on OpenShift guide, so I've
> added and adapted the content from upstream in [1] and [2].
> 
> Can you please review the following (in the attached PDF in Comment 29) and
> let me know if any steps are missing or need corrections?
> 
> * CHAPTER 2. CONFIGURING EXTERNAL AUTHENTICATION TO CLOUDFORMS
> * APPENDIX A. APPENDIX
> 
> A few questions:
> * The docker and OpenShift directions look similar (in the upstream docs),
> so I've deleted the docker instructions for now. Should the gem install
> instructions remain in the CloudForms guide?
> 
> * I found the cfme-httpd-configmap-generator container image on
> registry.redhat.com, so I imagine this is the image to tell customers to
> download. Does that step look correct? And to import the image, should you
> first log in as an admin user to OCP?
> 
> * Will the templates be obtained by the customers in the image stream
> configured in section "1.1.3. Templates and Images" of this guide?
> 
> Thanks for your review and comments.
> Cheers,
> Dayle
> 
> [1]https://github.com/ManageIQ/httpd_configmap_generator 
> [2]
> https://github.com/ManageIQ/manageiq-pods#configuring-external-
> authentication.

Hey Dayle,

Sorry I let this slide. I'm reviewing it now.

JoeV

Comment 33 Joe Vlcek 2018-06-15 13:42:04 UTC
John P.

Do you know the answer to Dayle's questions regarding the official locations for and how to get the downstream cfme containers?


"* I found the cfme-httpd-configmap-generator container image on registry.redhat.com, so I imagine this is the image to tell customers to download. Does that step look correct? And to import the image, should you first log in as an admin user to OCP?"

JoeV

Comment 34 Joe Vlcek 2018-06-15 15:13:21 UTC
Satoe,

Can you please reply to Dayle's questions regarding how podified images and templates are delivered downstream?

Thank you. JoeV

Comment 38 Ievgen Zapolskyi 2018-06-19 12:36:17 UTC
Hello Dayle, 
I added a few comments to your PR.
Documentation looks good so far. 

We will probably need to run thru it in test environment and check that nothing is missing when you finish with documetation.

Comment 39 Dayle Parker 2018-06-19 16:40:37 UTC
Thank you Eugene and Joe for your reviews and comments (and the run-through, Joe), much appreciated :) I think I've applied all the suggested edits now, feel free to make any changes/comments as you wish. And Eugene, if you want to test the docs that would be great -- many thanks!

https://github.com/ManageIQ/manageiq_docs/pull/870/files#diff-6f1fc3056c0a123e62996415882d2356

Comment 41 Dayle Parker 2018-06-29 02:47:21 UTC
Many thanks for confirming, Satoe. I've added those template names to the doc now.

Comment 45 Dayle Parker 2018-07-05 03:42:11 UTC
This update is now published on the Customer Portal in the following guide:

https://access.redhat.com/documentation/en-us/red_hat_cloudforms/4.6/html-single/installing_red_hat_cloudforms_on_openshift_container_platform/

The update includes a new chapter, "2. Configuring External Authentication to CloudForms", plus reference material to help configure authentication in the Appendix (and a few other minor updates throughout the guide).

Thanks to JoeV, Ievgen, Satoe, and Suyog for your reviews and help along the way :)