Bug 974497 - On fresh all-on-one OpenShift Origin installation, oo-diagnostics reports errors
On fresh all-on-one OpenShift Origin installation, oo-diagnostics reports errors
Status: CLOSED WONTFIX
Product: OpenShift Origin
Classification: Red Hat
Component: Pod (Show other bugs)
2.x
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Luke Meyer
libra bugs
:
Depends On:
Blocks: 1007752
  Show dependency treegraph
 
Reported: 2013-06-14 05:41 EDT by Jan Pazdziora
Modified: 2017-05-31 14:22 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1007752 (view as bug list)
Environment:
Last Closed: 2017-05-31 14:22:11 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jan Pazdziora 2013-06-14 05:41:36 EDT
Description of problem:

I've installed OpenShift Origin nightly on Fedora 19 today. I've noticed it now includes oo-diagnostics. When running it, error listed below were reported.

Version-Release number of selected component (if applicable):

rubygem-openshift-origin-common-1.10.2-1.git.0.a17abd2.fc19.noarch

How reproducible:

Deterministic.

Steps to Reproduce:
1. Install OpenShift Origin using steps described at http://openshift.github.io/origin/file.install_origin_using_puppet.html with the patching script https://raw.github.com/openshift/puppet-openshift_origin/master/test/manifests/f19_patches.sh. Do the all-on-one installation.
2. Run oo-diagnostics -v -w 1

Actual results:

No errors.

Expected results:

[root@broker ~]# oo-diagnostics -v -w 1
INFO: loading list of installed packages
INFO: OpenShift broker installed.
INFO: OpenShift node installed.
INFO: running: prereq_dns_server_available
INFO: checking that the first server in /etc/resolv.conf responds
INFO: running: test_enterprise_rpms
INFO: skipping test_enterprise_rpms
INFO: running: test_selinux_policy_rpm
INFO: running: test_selinux_enabled
INFO: running: test_broker_cache_permissions
INFO: broker application cache permissions appear fine
INFO: running: test_node_profiles_districts_from_broker
INFO: checking node profiles via MCollective
INFO: profile for broker.example.com: small
WARN: test_node_profiles_districts_from_broker
        The following gear profile(s) are configured but not provided by node hosts:
          medium
        Attempts to create apps using these gear profiles will fail.
        Please fix the settings in /etc/openshift/broker.conf or add node hosts accordingly.

WARN: test_node_profiles_districts_from_broker
        No districts are defined. Districts should be used in any production installation.
        Please consult the Administration Guide.

INFO: skipping test_node_profiles_districts_from_broker
INFO: running: test_broker_accept_scripts
INFO: running oo-accept-broker
FAIL: run_script
oo-accept-broker had errors:
--BEGIN OUTPUT--
NOTICE: SELinux is Enforcing
NOTICE: SELinux is  Enforcing
FAIL: SELinux boolean httpd_unified is disabled -- run setsebool -P httpd_unified=on
Failed to issue method call: No such file or directory
FAIL: service iptables not enabled;
FAIL: service iptables not running
FAIL: Datastore Password has been left configured as the default 'mooo'
	-- please reconfigure and ensure the DB user's password matches.
FAIL: Datastore Password has been left configured as the default 'mooo'
	-- please reconfigure and ensure the DB user's password matches.
5 ERRORS

--END oo-accept-broker OUTPUT--
INFO: running oo-accept-systems -w 1.0
INFO: oo-accept-systems -w 1.0 ran without error:
--BEGIN OUTPUT--
PASS

--END oo-accept-systems -w 1.0 OUTPUT--
INFO: running: test_node_accept_scripts
INFO: running oo-accept-node
FAIL: run_script
oo-accept-node had errors:
--BEGIN OUTPUT--
FAIL: selinux boolean allow_polyinstantiation should be on
FAIL: service cgconfig not running
FAIL: Could not get SELinux context for mcollective
FAIL: Could not get SELinux context for oddjobd
FAIL: kernel.sem semaphores too low: 128 < 512
5 ERRORS

--END oo-accept-node OUTPUT--
INFO: running: test_broker_httpd_error_log
INFO: running: test_broker_passenger_ps
INFO: checking the broker application process tree
INFO: running: test_for_nonrpm_rubygems
INFO: skipping test_for_nonrpm_rubygems
INFO: running: test_for_multiple_gem_versions
INFO: checking for presence of gem-installed rubygems
INFO: running: test_node_httpd_error_log
INFO: running: test_node_mco_log
INFO: running: test_pam_openshift
INFO: running: test_services_enabled
INFO: checking that required services are running now
FAIL: test_services_enabled
      The following service(s) are not currently started:
        network, cgconfig
      These services are required for OpenShift functionality.

INFO: checking that required services are enabled at boot
INFO: running: test_node_quota_bug
INFO: skipping test_node_quota_bug
INFO: running: test_vhost_servernames
INFO: checking for vhost interference problems
WARN: test_vhost_servernames
        The VirtualHost defined by default in /etc/httpd/conf.d/ssl.conf is not needed
        and can cause spurious warnings. Please remove it by running this command:

          sed -i '/VirtualHost/,/VirtualHost/ d' /etc/httpd/conf.d/ssl.conf

INFO: running: test_altered_package_owned_configs
/usr/sbin/oo-diagnostics: No such file or directory - updatedb
sh: locate: command not found
INFO: running: test_broken_httpd_version
INFO: running: test_usergroups_enabled
INFO: running: test_mcollective_context
FAIL: test_mcollective_context
      Mcollectived is not running in the expected SELinux context, which
      may result in node execution failures. Please check that the correct
      context is set on /usr/sbin/mcollectived and that the correct SELinux
      policies are loaded.
        Expected: system_r:openshift_initrc_t:s0-s0:c0.c1023
        Found: unconfined_r:unconfined_t:s0-s0:c0.c1023
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

INFO: running: test_mcollective_bad_facts
INFO: running: test_auth_conf_files
ls: cannot access /var/www/openshift/console/httpd/conf.d/*auth*.conf: No such file or directory
INFO: running: test_broker_certificate
WARN: rescue in test_broker_certificate
There was an error verifying the Broker SSL cert: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
4 WARNINGS
4 ERRORS

Additional info:

I understand that in some cases, it's probably not oo-diagnostics which is at fault but rather the puppet stuff which produces configuration that does not match the expectation of oo-diagnostics.
Comment 1 Luke Meyer 2013-07-12 08:35:38 EDT
oo-diagnostics probably also needs some adjustment for Origin; I don't think it's been kept up to date.
Comment 2 Luke Meyer 2013-08-28 16:07:51 EDT
Some of these problems may be improved. I have a card in for working on this:
https://trello.com/c/PQjvrZDN/8-oo-accept-and-oo-diagnostics-on-origin

Note that while oo-diagnostics should run without ERROR/FAIL don't expect it to be totally without WARNINGs out of the box. There are some things the install won't do for you.
Comment 3 Peter Ruan 2013-08-29 02:59:42 EDT
Hi Luke,
  Here's the output from the latest origin image from kraman.

[root@broker-ba8f ~]# oo-diagnostics 
WARN: test_node_profiles_districts_from_broker
        The following gear profile(s) are configured but not provided by node hosts:
          medium
        Attempts to create apps using these gear profiles will fail.
        Please fix the settings in /etc/openshift/broker.conf or add node hosts accordingly.

WARN: test_node_profiles_districts_from_broker
        No districts are defined. Districts should be used in any production installation.
        Please consult the Administration Guide.

FAIL: run_script
oo-accept-broker had errors:
--BEGIN OUTPUT--
NOTICE: SELinux is Enforcing
NOTICE: SELinux is  Enforcing
FAIL: SELinux boolean httpd_unified is disabled -- run setsebool -P httpd_unified=on
FAIL: service iptables not enabled;
FAIL: service iptables not running
FAIL: Datastore Password has been left configured as the default 'mooo'
	-- please reconfigure and ensure the DB user's password matches.
FAIL: Datastore Password has been left configured as the default 'mooo'
	-- please reconfigure and ensure the DB user's password matches.
NOTICE: unknown dns class: OpenShift::AvahiPlugin
5 ERRORS

--END oo-accept-broker OUTPUT--
FAIL: run_script
oo-accept-node had errors:
--BEGIN OUTPUT--
FAIL: selinux boolean allow_polyinstantiation should be on
FAIL: Could not get SELinux context for mcollective
FAIL: Could not get SELinux context for oddjobd
3 ERRORS

--END oo-accept-node OUTPUT--
WARN: test_vhost_servernames
        The VirtualHost defined by default in /etc/httpd/conf.d/ssl.conf is not needed
        and can cause spurious warnings. Please remove it by running this command:

          sed -i '/VirtualHost/,/VirtualHost/ d' /etc/httpd/conf.d/ssl.conf

WARN: test_altered_package_owned_configs
           RPM package owned configuration files have been altered:
             /etc/yum.repos.d/jenkins.repo.rpmnew

           Ensure any package-owned configuration files which have been
           altered are accurate. This may require a manual merge of
           your previous alterations. Once you are comfortable with the merge,
           remove the reported .rpm* configuration file (or you will continue
           to see this warning each time you run the diagnostic test).

FAIL: test_mcollective_context
      Mcollectived is not running in the expected SELinux context, which
      may result in node execution failures. Please check that the correct
      context is set on /usr/sbin/mcollectived and that the correct SELinux
      policies are loaded.
        Expected: system_r:openshift_initrc_t:s0-s0:c0.c1023
        Found: unconfined_r:unconfined_t:s0-s0:c0.c1023
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

ls: cannot access /var/www/openshift/console/httpd/conf.d/*auth*.conf: No such file or directory
WARN: rescue in test_broker_certificate
There was an error verifying the Broker SSL cert: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
5 WARNINGS
3 ERRORS
Comment 4 Jan Pazdziora 2013-12-10 08:57:47 EST
Today on Fedora 19, the oo-diagnostics reports:

oo-diagnostics is /usr/sbin/oo-diagnostics
INFO: loading list of installed packages
INFO: OpenShift broker installed.
INFO: OpenShift node installed.
/usr/share/gems/gems/psych-2.0.0/lib/psych.rb:98: warning: already initialized constant Psych::VERSION
/usr/share/ruby/vendor_ruby/psych.rb:98: warning: previous definition of VERSION was here
/usr/share/gems/gems/psych-2.0.0/lib/psych.rb:101: warning: already initialized constant Psych::LIBYAML_VERSION
/usr/share/ruby/vendor_ruby/psych.rb:101: warning: previous definition of LIBYAML_VERSION was here
INFO: running: prereq_dns_server_available
INFO: checking that the first server in /etc/resolv.conf responds
INFO: running: test_enterprise_rpms
INFO: skipping test_enterprise_rpms
INFO: running: test_selinux_policy_rpm
INFO: running: test_selinux_enabled
INFO: running: test_broker_cache_permissions
INFO: broker application cache permissions appear fine
INFO: running: test_node_profiles_districts_from_broker
INFO: checking node profiles via MCollective
INFO: profile for broker.example.com: small
[33mWARN: test_node_profiles_districts_from_broker
        The following gear profile(s) are configured but not provided by node hosts:
          medium
        Attempts to create apps using these gear profiles will fail.
        Please fix the settings in /etc/openshift/broker.conf or add node hosts accordingly.
[0m
[33mWARN: test_node_profiles_districts_from_broker
        No districts are defined. Districts should be used in any production installation.
        Please consult the Administration Guide.
[0m
INFO: skipping test_node_profiles_districts_from_broker
INFO: running: test_broker_accept_scripts
INFO: running oo-accept-broker

MARK-LWD-LOOP -- 2013-12-10 04:57:15 --
[31mFAIL: run_script
oo-accept-broker had errors:
--BEGIN OUTPUT--
NOTICE: SELinux is Enforcing
NOTICE: SELinux is  Enforcing
FAIL: service iptables not running
1 ERRORS

--END oo-accept-broker OUTPUT--[0m
INFO: running oo-accept-systems -w 1.0
INFO: oo-accept-systems -w 1.0 ran without error:
--BEGIN OUTPUT--
PASS

--END oo-accept-systems -w 1.0 OUTPUT--
INFO: running: test_node_accept_scripts
INFO: running oo-accept-node
[31mFAIL: run_script
oo-accept-node had errors:
--BEGIN OUTPUT--
FAIL: service cgconfig not running
FAIL: Could not get SELinux context for mcollective
FAIL: Could not get SELinux context for oddjobd
3 ERRORS

--END oo-accept-node OUTPUT--[0m
INFO: running: test_broker_httpd_error_log
INFO: running: test_broker_passenger_ps
INFO: checking the broker application process tree
INFO: running: test_for_nonrpm_rubygems
INFO: skipping test_for_nonrpm_rubygems
INFO: running: test_for_multiple_gem_versions
INFO: checking for presence of gem-installed rubygems
INFO: running: test_node_httpd_error_log
INFO: running: test_node_containerization_plugin
INFO: running: test_node_mco_log
INFO: running: test_pam_openshift
INFO: running: test_services_enabled
INFO: checking that required services are running now
[31mFAIL: test_services_enabled
      The following service(s) are not currently started:
        network, openshift-iptables-port-proxy, cgconfig
      These services are required for OpenShift functionality.
[0m
INFO: checking that required services are enabled at boot
[31mFAIL: test_services_enabled
      The following service(s) are not started at boot time:
        network, cgconfig, cgred
      These services are required for OpenShift functionality.
      Please ensure that they start at boot.
[0m
INFO: running: test_node_quota_bug
INFO: skipping test_node_quota_bug
INFO: running: test_vhost_servernames
INFO: checking for vhost interference problems
[33mWARN: test_vhost_servernames
        The VirtualHost defined by default in /etc/httpd/conf.d/ssl.conf is not needed
        and can cause spurious warnings. Please remove it by running this command:

          sed -i '/VirtualHost/,/VirtualHost/ d' /etc/httpd/conf.d/ssl.conf
[0m
INFO: running: test_altered_package_owned_configs
[33mWARN: test_altered_package_owned_configs
          The mlocate package is not installed. mlocate is not a required runtime package; however,
          you may install mlocate to enable further diagnostics checking.
[0m
INFO: running: test_broken_httpd_version
INFO: running: test_usergroups_enabled
INFO: running: test_mcollective_context
INFO: running: test_mcollective_bad_facts
INFO: running: test_auth_conf_files
INFO: running: test_broker_certificate
[33mWARN: test_broker_certificate
Using a self-signed certificate for the broker[0m
grep: /etc/httpd/conf.d/openshift: Is a directory
[33mWARN: block (2 levels) in test_broker_certificate
            /etc/httpd/conf.d/000002_openshift_origin_broker_servername.conf 
            defines ServerName as localhost.  This does not match the certificate common name of 
            *.example.com.  
            This can cause errors when client tools try to connect to the broker.
[0m
INFO: running: test_abrt_addon_python
INFO: running: test_node_frontend_clash
INFO: running: test_yum_configuration
[33m6 WARNINGS[0m
[31m4 ERRORS[0m
Comment 6 Eric Paris 2017-05-31 14:22:11 EDT
We apologize, however, we do not plan to address this report at this time. The majority of our active development is for the v3 version of OpenShift. If you would like for Red Hat to reconsider this decision, please reach out to your support representative. We are very sorry for any inconvenience this may cause.

Note You need to log in before you can comment on or make changes to this bug.