Hide Forgot
Description of problem: Deployement of 3 controller 3 compute. The deployment fail on the first controller cause httpd is not able to start cause :80 is used by haproxy. Version-Release number of selected component (if applicable): rhosp-director-images-10.0-20161130.1.el7ost.noarch rhosp-director-images-ipa-10.0-20161130.1.el7ost.noarch openstack-tripleo-heat-templates-5.1.0-7.el7ost.noarch Log : Warning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications. Error: Could not start Service[httpd]: Execution of '/usr/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details. Error: /Stage[main]/Apache::Service/Service[httpd]/ensure: change from stopped to running failed: Could not start Service[httpd]: Execution of '/usr/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journ alctl -xe" for details. Warning: /Stage[main]/Keystone::Deps/Anchor[keystone::service::end]: Skipping because of failed dependencies Error: Could not prefetch keystone_tenant provider 'openstack': Execution of '/usr/bin/openstack project list --quiet --format csv --long' returned 1: Unable to establish connection to http://172.16.0.22:35357/v3/projects: HTTPConnectionPool(host='172.16.0.22', port=35357): Max retries exceeded with url: /v3/projec ts (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x3817c10>: Failed to establish a new connection: [Errno 111] Connection refused',)) (tried 38, for a total of 170 seconds) Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[service]: Skipping because of failed dependencies Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[admin]: Skipping because of failed dependencies Error: Could not prefetch keystone_role provider 'openstack': Execution of '/usr/bin/openstack role list --quiet --format csv' returned 1: Unable to establish connection to http://172.16.0.22:35357/v3/roles: HTTPConnectionPool(host='172.16.0.22', port=35357): Max retries exceeded with url: /v3/roles (Caused by NewC onnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x300c050>: Failed to establish a new connection: [Errno 111] Connection refused',)) (tried 38, for a total of 170 seconds) Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_role[admin]: Skipping because of failed dependencies Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_user[admin]: Skipping because of failed dependencies Warning: /Stage[main]/Keystone::Roles::Admin/Keystone_user_role[admin@admin]: Skipping because of failed dependencies Error: Could not prefetch keystone_service provider 'openstack': Execution of '/usr/bin/openstack service list --quiet --format csv --long' returned 1: Unable to establish connection to http://172.16.0.22:35357/v3/services: HTTPConnectionPool(host='172.16.0.22', port=35357): Max retries exceeded with url: /v3/servi ces (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x3bd9850>: Failed to establish a new connection: [Errno 111] Connection refused',)) (tried 38, for a total of 170 seconds) Warning: /Stage[main]/Keystone::Endpoint/Keystone::Resource::Service_identity[keystone]/Keystone_service[keystone::identity]: Skipping because of failed dependencies Error: Could not prefetch keystone_endpoint provider 'openstack': Execution of '/usr/bin/openstack endpoint list --quiet --format csv' returned 1: Unable to establish connection to http://172.16.0.22:35357/v3/endpoints: HTTPConnectionPool(host='172.16.0.22', port=35357): Max retries exceeded with url: /v3/endpoints (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x33228d0>: Failed to establish a new connection: [Errno 111] Connection refused',)) (tried 38, for a total of 170 seconds) Manual Test : [root@overcloud-controller-0 ~]# systemctl status httpd * httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/httpd.service.d `-openstack-dashboard.conf Active: failed (Result: exit-code) since Fri 2016-12-02 12:23:20 GMT; 10s ago Docs: man:httpd(8) man:apachectl(8) Process: 201487 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE) Process: 201484 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=1/FAILURE) Process: 200784 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py compress --force (code=exited, status=0/SUCCESS) Process: 200448 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear (code=exited, status=0/SUCCESS) Main PID: 201484 (code=exited, status=1/FAILURE) Dec 02 12:23:05 overcloud-controller-0.localdomain python[200448]: Copying '/usr/share/javascript/jquery_ui/themes/south-street/images/ui-bg_inset-soft_15_2b2922_1x100.png' Dec 02 12:23:05 overcloud-controller-0.localdomain python[200448]: Copying '/usr/share/javascript/jquery_ui/themes/south-street/images/ui-bg_highlight-soft_95_ffedad_1x100.png' Dec 02 12:23:05 overcloud-controller-0.localdomain python[200448]: Copying '/usr/share/javascript/jquery_ui/themes/south-street/images/ui-bg_highlight-hard_100_f5f3e5_1x100.png' Dec 02 12:23:05 overcloud-controller-0.localdomain python[200448]: Copying '/usr/share/javascript/jquery_ui/themes/south-street/images/ui-icons_847e71_256x240.png' Dec 02 12:23:05 overcloud-controller-0.localdomain python[200448]: Copying '/usr/share/javascript/jquery_ui/themes/south-street/images/ui-bg_highlight-hard_95_cccccc_1x100.png' Dec 02 12:23:20 overcloud-controller-0.localdomain systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE Dec 02 12:23:20 overcloud-controller-0.localdomain systemd[1]: httpd.service: control process exited, code=exited status=1 Dec 02 12:23:20 overcloud-controller-0.localdomain systemd[1]: Failed to start The Apache HTTP Server. Dec 02 12:23:20 overcloud-controller-0.localdomain systemd[1]: Unit httpd.service entered failed state. Dec 02 12:23:20 overcloud-controller-0.localdomain systemd[1]: httpd.service failed. [root@overcloud-controller-0 ~]# /usr/sbin/httpd $OPTIONS -DFOREGROUND (98)Address already in use: AH00073: make_sock: unable to listen for connections on address [::]:80 (98)Address already in use: AH00073: make_sock: unable to listen for connections on address 0.0.0.0:80 no listening sockets available, shutting down [root@overcloud-controller-0 ~]# systemctl stop haproxy [root@overcloud-controller-0 ~]# systemctl start [root@overcloud-controller-0 ~]# systemctl status httpd * httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/httpd.service.d `-openstack-dashboard.conf Active: active (running) since Fri 2016-12-02 12:28:56 GMT; 50s ago Docs: man:httpd(8) man:apachectl(8) Main PID: 220283 (httpd) Status: "Total requests: 0; Current requests/sec: 0; Current traffic: 0 B/sec" CGroup: /system.slice/httpd.service |-220283 /usr/sbin/httpd -DFOREGROUND |-220285 keystone-admin -DFOREGROUND |-220286 keystone-admin -DFOREGROUND |-220287 keystone-admin -DFOREGROUND |-220288 keystone-admin -DFOREGROUND |-220289 keystone-main -DFOREGROUND |-220290 keystone-main -DFOREGROUND |-220291 keystone-main -DFOREGROUND |-220292 keystone-main -DFOREGROUND |-220293 /usr/sbin/httpd -DFOREGROUND |-220294 /usr/sbin/httpd -DFOREGROUND |-220295 /usr/sbin/httpd -DFOREGROUND |-220296 /usr/sbin/httpd -DFOREGROUND |-220297 /usr/sbin/httpd -DFOREGROUND |-220298 /usr/sbin/httpd -DFOREGROUND |-220299 /usr/sbin/httpd -DFOREGROUND `-220300 /usr/sbin/httpd -DFOREGROUND
It's look the root cause : [root@overcloud-controller-0 ~]# cat /etc/httpd/conf.d/15-default.conf # ************************************ # Vhost template in module puppetlabs-apache # Managed by Puppet # ************************************ <VirtualHost *:80> ServerName default ServerAdmin root@localhost ## Vhost docroot DocumentRoot "/var/www/html" ## Directories, there should at least be a declaration for /var/www/html <Directory "/var/www/html"> Options Indexes FollowSymLinks MultiViews AllowOverride None Require all granted </Directory> ## Logging ErrorLog "/var/log/httpd/default_error.log" ServerSignature Off CustomLog "/var/log/httpd/access_log" combined ## Script alias directives ScriptAlias /cgi-bin "/var/www/cgi-bin" </VirtualHost>
Using this env is the root cause : /usr/share/openstack-tripleo-heat-templates/environments/updates/update-from-keystone-admin-internal-api.yaml Cause ApacheNetwork: internal_api is not set and in /puppet/services/apache.yaml, apache::ip: {get_param: [ServiceNetMap, ApacheNetwork]} can be set.