Bug 1007752 - [Backport Request] On fresh all-on-one OpenShift Origin installation, oo-diagnostics reports errors
Summary: [Backport Request] On fresh all-on-one OpenShift Origin installation, oo-diag...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 1.2.0
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
: ---
Assignee: Jason DeTiberus
QA Contact: libra bugs
URL:
Whiteboard:
Depends On: 974497
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-13 09:16 UTC by Nikhil Mone
Modified: 2018-12-03 19:56 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 974497
Environment:
Last Closed: 2013-09-26 14:32:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Nikhil Mone 2013-09-13 09:16:28 UTC
+++ This bug was initially created as a clone of Bug #974497 +++

This is a backport request for OSE 1.2

Description of problem:

I've installed OpenShift Origin nightly on Fedora 19 today. I've noticed it now includes oo-diagnostics. When running it, error listed below were reported.

Version-Release number of selected component (if applicable):

rubygem-openshift-origin-common-1.10.2-1.git.0.a17abd2.fc19.noarch

How reproducible:

Deterministic.

Steps to Reproduce:
1. Install OpenShift Origin using steps described at http://openshift.github.io/origin/file.install_origin_using_puppet.html with the patching script https://raw.github.com/openshift/puppet-openshift_origin/master/test/manifests/f19_patches.sh. Do the all-on-one installation.
2. Run oo-diagnostics -v -w 1

Actual results:

No errors.

Expected results:

[root@broker ~]# oo-diagnostics -v -w 1
INFO: loading list of installed packages
INFO: OpenShift broker installed.
INFO: OpenShift node installed.
INFO: running: prereq_dns_server_available
INFO: checking that the first server in /etc/resolv.conf responds
INFO: running: test_enterprise_rpms
INFO: skipping test_enterprise_rpms
INFO: running: test_selinux_policy_rpm
INFO: running: test_selinux_enabled
INFO: running: test_broker_cache_permissions
INFO: broker application cache permissions appear fine
INFO: running: test_node_profiles_districts_from_broker
INFO: checking node profiles via MCollective
INFO: profile for broker.example.com: small
WARN: test_node_profiles_districts_from_broker
        The following gear profile(s) are configured but not provided by node hosts:
          medium
        Attempts to create apps using these gear profiles will fail.
        Please fix the settings in /etc/openshift/broker.conf or add node hosts accordingly.

WARN: test_node_profiles_districts_from_broker
        No districts are defined. Districts should be used in any production installation.
        Please consult the Administration Guide.

INFO: skipping test_node_profiles_districts_from_broker
INFO: running: test_broker_accept_scripts
INFO: running oo-accept-broker
FAIL: run_script
oo-accept-broker had errors:
--BEGIN OUTPUT--
NOTICE: SELinux is Enforcing
NOTICE: SELinux is  Enforcing
FAIL: SELinux boolean httpd_unified is disabled -- run setsebool -P httpd_unified=on
Failed to issue method call: No such file or directory
FAIL: service iptables not enabled;
FAIL: service iptables not running
FAIL: Datastore Password has been left configured as the default 'mooo'
	-- please reconfigure and ensure the DB user's password matches.
FAIL: Datastore Password has been left configured as the default 'mooo'
	-- please reconfigure and ensure the DB user's password matches.
5 ERRORS

--END oo-accept-broker OUTPUT--
INFO: running oo-accept-systems -w 1.0
INFO: oo-accept-systems -w 1.0 ran without error:
--BEGIN OUTPUT--
PASS

--END oo-accept-systems -w 1.0 OUTPUT--
INFO: running: test_node_accept_scripts
INFO: running oo-accept-node
FAIL: run_script
oo-accept-node had errors:
--BEGIN OUTPUT--
FAIL: selinux boolean allow_polyinstantiation should be on
FAIL: service cgconfig not running
FAIL: Could not get SELinux context for mcollective
FAIL: Could not get SELinux context for oddjobd
FAIL: kernel.sem semaphores too low: 128 < 512
5 ERRORS

--END oo-accept-node OUTPUT--
INFO: running: test_broker_httpd_error_log
INFO: running: test_broker_passenger_ps
INFO: checking the broker application process tree
INFO: running: test_for_nonrpm_rubygems
INFO: skipping test_for_nonrpm_rubygems
INFO: running: test_for_multiple_gem_versions
INFO: checking for presence of gem-installed rubygems
INFO: running: test_node_httpd_error_log
INFO: running: test_node_mco_log
INFO: running: test_pam_openshift
INFO: running: test_services_enabled
INFO: checking that required services are running now
FAIL: test_services_enabled
      The following service(s) are not currently started:
        network, cgconfig
      These services are required for OpenShift functionality.

INFO: checking that required services are enabled at boot
INFO: running: test_node_quota_bug
INFO: skipping test_node_quota_bug
INFO: running: test_vhost_servernames
INFO: checking for vhost interference problems
WARN: test_vhost_servernames
        The VirtualHost defined by default in /etc/httpd/conf.d/ssl.conf is not needed
        and can cause spurious warnings. Please remove it by running this command:

          sed -i '/VirtualHost/,/VirtualHost/ d' /etc/httpd/conf.d/ssl.conf

INFO: running: test_altered_package_owned_configs
/usr/sbin/oo-diagnostics: No such file or directory - updatedb
sh: locate: command not found
INFO: running: test_broken_httpd_version
INFO: running: test_usergroups_enabled
INFO: running: test_mcollective_context
FAIL: test_mcollective_context
      Mcollectived is not running in the expected SELinux context, which
      may result in node execution failures. Please check that the correct
      context is set on /usr/sbin/mcollectived and that the correct SELinux
      policies are loaded.
        Expected: system_r:openshift_initrc_t:s0-s0:c0.c1023
        Found: unconfined_r:unconfined_t:s0-s0:c0.c1023
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

INFO: running: test_mcollective_bad_facts
INFO: running: test_auth_conf_files
ls: cannot access /var/www/openshift/console/httpd/conf.d/*auth*.conf: No such file or directory
INFO: running: test_broker_certificate
WARN: rescue in test_broker_certificate
There was an error verifying the Broker SSL cert: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
4 WARNINGS
4 ERRORS

Additional info:

I understand that in some cases, it's probably not oo-diagnostics which is at fault but rather the puppet stuff which produces configuration that does not match the expectation of oo-diagnostics.

--- Additional comment from Luke Meyer on 2013-07-12 08:35:38 EDT ---

oo-diagnostics probably also needs some adjustment for Origin; I don't think it's been kept up to date.

--- Additional comment from Luke Meyer on 2013-08-28 16:07:51 EDT ---

Some of these problems may be improved. I have a card in for working on this:
https://trello.com/c/PQjvrZDN/8-oo-accept-and-oo-diagnostics-on-origin

Note that while oo-diagnostics should run without ERROR/FAIL don't expect it to be totally without WARNINGs out of the box. There are some things the install won't do for you.

--- Additional comment from Peter Ruan on 2013-08-29 02:59:42 EDT ---

Hi Luke,
  Here's the output from the latest origin image from kraman.

[root@broker-ba8f ~]# oo-diagnostics 
WARN: test_node_profiles_districts_from_broker
        The following gear profile(s) are configured but not provided by node hosts:
          medium
        Attempts to create apps using these gear profiles will fail.
        Please fix the settings in /etc/openshift/broker.conf or add node hosts accordingly.

WARN: test_node_profiles_districts_from_broker
        No districts are defined. Districts should be used in any production installation.
        Please consult the Administration Guide.

FAIL: run_script
oo-accept-broker had errors:
--BEGIN OUTPUT--
NOTICE: SELinux is Enforcing
NOTICE: SELinux is  Enforcing
FAIL: SELinux boolean httpd_unified is disabled -- run setsebool -P httpd_unified=on
FAIL: service iptables not enabled;
FAIL: service iptables not running
FAIL: Datastore Password has been left configured as the default 'mooo'
	-- please reconfigure and ensure the DB user's password matches.
FAIL: Datastore Password has been left configured as the default 'mooo'
	-- please reconfigure and ensure the DB user's password matches.
NOTICE: unknown dns class: OpenShift::AvahiPlugin
5 ERRORS

--END oo-accept-broker OUTPUT--
FAIL: run_script
oo-accept-node had errors:
--BEGIN OUTPUT--
FAIL: selinux boolean allow_polyinstantiation should be on
FAIL: Could not get SELinux context for mcollective
FAIL: Could not get SELinux context for oddjobd
3 ERRORS

--END oo-accept-node OUTPUT--
WARN: test_vhost_servernames
        The VirtualHost defined by default in /etc/httpd/conf.d/ssl.conf is not needed
        and can cause spurious warnings. Please remove it by running this command:

          sed -i '/VirtualHost/,/VirtualHost/ d' /etc/httpd/conf.d/ssl.conf

WARN: test_altered_package_owned_configs
           RPM package owned configuration files have been altered:
             /etc/yum.repos.d/jenkins.repo.rpmnew

           Ensure any package-owned configuration files which have been
           altered are accurate. This may require a manual merge of
           your previous alterations. Once you are comfortable with the merge,
           remove the reported .rpm* configuration file (or you will continue
           to see this warning each time you run the diagnostic test).

FAIL: test_mcollective_context
      Mcollectived is not running in the expected SELinux context, which
      may result in node execution failures. Please check that the correct
      context is set on /usr/sbin/mcollectived and that the correct SELinux
      policies are loaded.
        Expected: system_r:openshift_initrc_t:s0-s0:c0.c1023
        Found: unconfined_r:unconfined_t:s0-s0:c0.c1023
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

ls: cannot access /var/www/openshift/console/httpd/conf.d/*auth*.conf: No such file or directory
WARN: rescue in test_broker_certificate
There was an error verifying the Broker SSL cert: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
5 WARNINGS
3 ERRORS

Comment 2 Jason DeTiberus 2013-09-13 13:20:29 UTC
The specific error in question is the SSL cert check

Comment 3 manoj 2013-09-26 14:32:40 UTC
We will pick this up in OSE 2.0

Comment 4 Jan Pazdziora 2013-09-27 01:00:15 UTC
The problem is that bug 974497 hasn't been fixed. And if it is not fixed upstream, it won't get to 2.0. Wouldn't it be better to keep this bugzilla open and track it against 2.0 to make sure it's on the radar?


Note You need to log in before you can comment on or make changes to this bug.