Bug 1523272

Add downstream review for easy testing (merge conflict resolved)

Switch to stable/pike review.

Oki, here is a descriptive workaround:

 On every controller ensure that all directories in /usr/share/openstack-puppet/modules/ have a corresponding symlink in /etc/puppet/modules/

For instance if panko is missing in /etc/puppet/modules you would need that command:

cd /etc/puppet/modules 
sudo ln -s /usr/share/openstack-puppet/modules/panko

verified on openstack-tripleo-heat-templates-7.0.9-6.el7ost.noarch

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0602

for Openstack 14 I had to do the following for --allinone:

systemctl disable NetworkManager
systemctl stop NetworkManager
systemctl disable firewalld
systemctl stop firewalld
setenforce 0

systemctl restart network
systemctl status network
########################################################################
subscription-manager list --available

subscription-manager attach --pool=

subscription-manager repos --enable=rhel-7-server-optional-rpms \
  --enable=rhel-7-server-extras-rpms --enable=rhel-7-server-rh-common-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-devtools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-tools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-devtools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-tools-rpms
yum repolist enabled #enable all

subscriptiion-manager repos --enable=

sudo yum -y install yum-plugin-priorities yum-utils

yum install openstack-selinux
yum install python2*
yum install *pyparsing*
yum install *chardet*
yum install -y mariadb-server-galera
yum install -y *openvswitch*
#yum install -y openstack.*   #saves time installing with packstack#yum install -y puppet hiera openssh-clients tar nc rubygem-json
yum install -y *rabbitmq*

gnocchi gave errors
rabittmq gave loopbac_users error
... etc

I had to do this again with yum install -y openstack* --skip-broken

now I have the installation passed the errors only timeout error

so I did the installation with

packstack --answer-file=/root/answer.txt --timeout=999999999 --debug ### and I'm monitoring the install with ps -ef|grep and tail -f var log file

sorry I forgot to say I added heat and sahara and https disabled demo and I'm not planning tribleo for the underdog :)

not to mention top to see the running processes of the installation

now it's configuring Project ID's ... after long installation process ... I did the following

sudo yum -y install yum-plugin-priorities yum-utils

yum install openstack-selinux --skip-broken
yum install python2* --skip-broken 
yum install *pyparsing* --skip-broken
yum install *urllib3* --skip-broken
yum install *chardet* --skip-broken
yum install *gnocchi* --skip-broken
yum install *puppet* --skip-broken
yum install -y mariadb-server-galera --skip-broken
yum install -y *openvswitch* --skip-broken
#yum install -y openstack.*   #saves time installing with packstack#yum install -y puppet hiera openssh-clients tar nc rubygem-json
yum install -y puppet hiera openssh-clients tar nc rubygem-json --skip-broken
yum install -y *rabbitmq* --skip-broken
yum install -y *ironic* --skip-broken
rpm -q --whatprovides rubygem-json   ###### rubygem-json-1.7.7-20.el7.x86_64


#do not update with ### yum update ### it will upgrade the RHEL 7 to 7.6 Maipo if you want to keep 7.0 on your VM
yum install -y openstack-packstack --skip-broken

packstack --answer-file=/root/answer.txt --timeout=99999999999999 --debug

monitoring with:

tail -f ####putty session
top     ####putty session
ps -ef|grep -i openstack
ps -ef|grep -i neutron
ps -ef|grep -i yum
ps -ef|grep -i swift
ps -ef|grep -i nova
ps -ef|grep -i cinder
ps -ef|grep -i gnocchi
ps -ef|grep -i puppet
ps -ef|grep -i glance

...etc on another putty session ...

so three putty sessions to monitor and the VM is running the installation ...

ps -ef|grep -i openstack
root      60906  31701  0 11:16 pts/2    00:00:00 tail -f /var/tmp/packstack/20190118-111616-VhYx_9/openstack-setup.log
root      69005  62364  0 11:26 ?        00:00:05 [openstack] <defunct>
root      69201  62364  0 11:27 ?        00:00:05 [openstack] <defunct>
root      70774  62364  0 11:29 ?        00:00:03 [openstack] <defunct>
root      74891  62364  0 11:34 ?        00:00:04 [openstack] <defunct>
root      85696  32095  0 11:52 pts/4    00:00:00 grep --color=auto -i openstack

it shows that it's configuring the api's and modules (swift , glance, nova, cinder ...etc)

it took lots of time on a 5G VM .... lots of time

Preparing Swift proxy entries                        [ DONE ]
Preparing Swift storage entries                      [ DONE ]
Preparing Gnocchi entries                            [ DONE ]
Preparing Redis entries                              [ DONE ]
Preparing Ceilometer entries                         [ DONE ]
Preparing Aodh entries                               [ DONE ]
Preparing Puppet manifests                           [ DONE ]
Copying Puppet modules and manifests                 [ DONE ]
Applying 192.168.43.110_controller.pp
192.168.43.110_controller.pp:                        [ DONE ]           
Applying 192.168.43.110_network.pp
192.168.43.110_network.pp:                           [ DONE ]        
Applying 192.168.43.110_compute.pp
Testing if puppet apply is finished: 192.168.43.110_compute.pp  [ \ ]


god something is going right now

*** Installation completed successfully ******

Additional information:
 * Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.
 * File /root/keystonerc_admin has been created on OpenStack client host 192.168.43.110. To use the command line tools you need to source the file.
 * NOTE : A certificate was generated to be used for ssl, You should change the ssl certificate configured in /etc/httpd/conf.d/ssl.conf on 192.168.43.110 to use a CA signed cert.
 * To access the OpenStack Dashboard browse to https://192.168.43.110/dashboard .
Please, find your login credentials stored in the keystonerc_admin in your home directory.
 * The installation log file is available at: /var/tmp/packstack/20190118-111616-VhYx_9/openstack-setup.log
 * The generated manifests are available at: /var/tmp/packstack/20190118-111616-VhYx_9/manifests
 * Note temporary directory /var/tmp/packstack/006147b431fc4ccc9e81d798e9b5b8d7 on host 192.168.43.110 was not deleted for debugging purposes.

the full process is as follows:

[root@localhost amer]# packstack --answer-file=/root/answer.txt --timeout=99999999999999 --debug
Welcome to the Packstack setup utility

The installation log file is available at: /var/tmp/packstack/20190118-111616-VhYx_9/openstack-setup.log

Installing:
Clean Up                                             [ DONE ]
Discovering ip protocol version                      [ DONE ]
Setting up ssh keys                                  [ DONE ]
Preparing servers                                    [ DONE ]
Pre installing Puppet and discovering hosts' details [ DONE ]
Preparing pre-install entries                        [ DONE ]
Setting up CACERT                                    [ DONE ]
Preparing AMQP entries                               [ DONE ]
Preparing MariaDB entries                            [ DONE ]
Fixing Keystone LDAP config parameters to be undef if empty[ DONE ]
Preparing Keystone entries                           [ DONE ]
Preparing Glance entries                             [ DONE ]
Checking if the Cinder server has a cinder-volumes vg[ DONE ]
Preparing Cinder entries                             [ DONE ]
Preparing Nova API entries                           [ DONE ]
Creating ssh keys for Nova migration                 [ DONE ]
Gathering ssh host keys for Nova migration           [ DONE ]
Preparing Nova Compute entries                       [ DONE ]
Preparing Nova Scheduler entries                     [ DONE ]
Preparing Nova VNC Proxy entries                     [ DONE ]
Preparing OpenStack Network-related Nova entries     [ DONE ]
Preparing Nova Common entries                        [ DONE ]
Preparing Neutron LBaaS Agent entries                [ DONE ]
Preparing Neutron API entries                        [ DONE ]
Preparing Neutron L3 entries                         [ DONE ]
Preparing Neutron L2 Agent entries                   [ DONE ]
Preparing Neutron DHCP Agent entries                 [ DONE ]
Preparing Neutron Metering Agent entries             [ DONE ]
Checking if NetworkManager is enabled and running    [ DONE ]
Preparing OpenStack Client entries                   [ DONE ]
Preparing Horizon entries                            [ DONE ]
Preparing Swift builder entries                      [ DONE ]
Preparing Swift proxy entries                        [ DONE ]
Preparing Swift storage entries                      [ DONE ]
Preparing Gnocchi entries                            [ DONE ]
Preparing Redis entries                              [ DONE ]
Preparing Ceilometer entries                         [ DONE ]
Preparing Aodh entries                               [ DONE ]
Preparing Puppet manifests                           [ DONE ]
Copying Puppet modules and manifests                 [ DONE ]
Applying 192.168.43.110_controller.pp
192.168.43.110_controller.pp:                        [ DONE ]           
Applying 192.168.43.110_network.pp
192.168.43.110_network.pp:                           [ DONE ]        
Applying 192.168.43.110_compute.pp
192.168.43.110_compute.pp:                           [ DONE ]        
Applying Puppet manifests                            [ DONE ]
Finalizing                                           [ DONE ]

 **** Installation completed successfully ******

Additional information:
 * Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.
 * File /root/keystonerc_admin has been created on OpenStack client host 192.168.43.110. To use the command line tools you need to source the file.
 * NOTE : A certificate was generated to be used for ssl, You should change the ssl certificate configured in /etc/httpd/conf.d/ssl.conf on 192.168.43.110 to use a CA signed cert.
 * To access the OpenStack Dashboard browse to https://192.168.43.110/dashboard .
Please, find your login credentials stored in the keystonerc_admin in your home directory.
 * The installation log file is available at: /var/tmp/packstack/20190118-111616-VhYx_9/openstack-setup.log
 * The generated manifests are available at: /var/tmp/packstack/20190118-111616-VhYx_9/manifests
 * Note temporary directory /var/tmp/packstack/006147b431fc4ccc9e81d798e9b5b8d7 on host 192.168.43.110 was not deleted for debugging purposes.
[root@localhost amer]#

I have installed openstack completely without Errors
build the External internal router using CLI and Horizon ... there is an option to create External Networks but inside the Project that you create not on amdin
I have waited because my VM is very slow ... then the router gave an interface with internal network GREEN at last ... the interface with Extrenal network is not it's (RED-DOWN)
then the openstack crashed (Horizon Dashboard) ... then my system became very very slow ...
then I executed some CLI commands to trouble shoot ... the Horizon httpd was down for the dashboard and I learned that I have to install ironic baremetal to clean node because all openstack CLI commannds didn't execute ...
then my laptop crashed again ... this is when I learned that software really crashes hardware and hits very bad :)
then I rebooted and executed

packstack --answer-file=/root/answer.txt --timeout=99999999999999999 --debug #### again I enabled the ironic install to y

then executed the following

export OS_USERNAME=admin
export OS_PASSWORD=server
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://localhost:5000/v3

neutron net-create External1 --provider:network_type flat --provider:physical_network br-ex --router:external=true --shared
neutron net-create External2 --provider:network_type flat --provider:physical_network eno16777736 --router:external=true --shared
neutron net-create External3 --provider-physical-network provider --provider:physical_network eno16777736 --router:external=true --shared
openstack subnet create --network provider \
  --allocation-pool start=192.168.43.2,end=192.168.43.240 \
  --dns-nameserver 192.168.43.1 --gateway 192.168.43.1 \
  --subnet-range 192.168.43.0/24 provider

mysql
create database neutron;
grant all privileges on neutron.* to 'neutron'@'localhost' identified by 'server';
grant all privileges on neutron.* to 'neutron'@'%' identified by 'server';
quit

export | grep OS_declare -x OS_AUTH_URL="https://192.168.64.128:5000/v3"

#source admin-openrc.sh
. /root/keystonerc_admin
openstack user create --domain default --password-prompt neutron
openstack role add --project service --user neutron admin
openstack service create --name neutron --description "OpenStack Networking" network
openstack endpoint create --region RegionOne network public http://controller:9696
openstack endpoint create --region RegionOne network internal http://controller:9696
openstack endpoint create --region RegionOne network admin http://controller:9696

systemctl enable neutron-server.service neutron-openvswitch-agent.service neutron-dhcp-agent.service neutron-metadata-agent.service neutron-ovs-cleanup.service
systemctl start neutron-server.service neutron-openvswitch-agent.service neutron-dhcp-agent.service neutron-metadata-agent.service neutron-ovs-cleanup.service
systemctl enable neutron-l3-agent.service
systemctl start neutron-l3-agent.service

which is good and bad ... it seems my last configuration was not deleted from the system it gave me the following:

[root@localhost ~]# cat keystonerc_admin
unset OS_SERVICE_TOKEN
    export OS_USERNAME=admin
    export OS_PASSWORD='server'
    export OS_AUTH_URL=http://192.168.43.110:5000/v3
    export PS1='[\u@\h \W(keystone_admin)]\$ '

export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_IDENTITY_API_VERSION=3
    [root@localhost ~]# . keystoner_admin
-bash: keystoner_admin: No such file or directory
[root@localhost ~]# opestack
bash: opestack: command not found...
[root@localhost ~]# openstack
^C
^CTraceback (most recent call last):
  File "/usr/bin/openstack", line 6, in <module>
    from openstackclient.shell import main
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 28, in <module>
    from openstackclient.common import clientmanager
  File "/usr/lib/python2.7/site-packages/openstackclient/common/clientmanager.py", line 191, in <module>
    'openstack.cli.base',
  File "/usr/lib/python2.7/site-packages/openstackclient/common/clientmanager.py", line 153, in get_plugin_modules
    for ep in pkg_resources.iter_entry_points(group):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 527, in iter_entry_points
    entries = dist.get_entry_map(group)
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2574, in get_entry_map
    self._get_metadata('entry_points.txt'), self
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2340, in parse_map
    for group, lines in data:
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2964, in split_sections
    for line in yield_lines(s):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2161, in yield_lines
    for ss in strs:
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2503, in _get_metadata
    for line in self.get_metadata_lines(name):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1429, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1421, in get_metadata
    return self._get(self._fn(self.egg_info,name))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1538, in _get
    return stream.read()
KeyboardInterrupt
[root@localhost ~]# openstack user create --domain default --password-prompt neutron
^CTraceback (most recent call last):
  File "/usr/bin/openstack", line 6, in <module>
    from openstackclient.shell import main
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 28, in <module>
    from openstackclient.common import clientmanager
  File "/usr/lib/python2.7/site-packages/openstackclient/common/clientmanager.py", line 191, in <module>
    'openstack.cli.base',
  File "/usr/lib/python2.7/site-packages/openstackclient/common/clientmanager.py", line 153, in get_plugin_modules
    for ep in pkg_resources.iter_entry_points(group):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 527, in iter_entry_points
    entries = dist.get_entry_map(group)
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2574, in get_entry_map
    self._get_metadata('entry_points.txt'), self
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2340, in parse_map
    for group, lines in data:
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2964, in split_sections
    for line in yield_lines(s):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2161, in yield_lines
    for ss in strs:
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2503, in _get_metadata
    for line in self.get_metadata_lines(name):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1429, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1421, in get_metadata
    return self._get(self._fn(self.egg_info,name))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1538, in _get
    return stream.read()
KeyboardInterrupt
[root@localhost ~]# neutron net-create External1 --provider:network_type flat --provider:physical_network br-ex --router:external=true --shared
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Auth plugin requires parameters which were not given: auth_url
[root@localhost ~]# neutron net-create External2 --provider:network_type flat --provider:physical_network eno16777736 --router:external=true --shared
^CTraceback (most recent call last):
  File "/usr/bin/neutron", line 6, in <module>
    from neutronclient.shell import main
  File "/usr/lib/python2.7/site-packages/neutronclient/shell.py", line 30, in <module>
    from keystoneauth1 import session
  File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 40, in <module>
    import osprofiler.web as osprofiler_web
  File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 20, in <module>
    from osprofiler import profiler
  File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 27, in <module>
    from osprofiler import notifier
  File "/usr/lib/python2.7/site-packages/osprofiler/notifier.py", line 16, in <module>
    from osprofiler.drivers import base
  File "/usr/lib/python2.7/site-packages/osprofiler/drivers/__init__.py", line 2, in <module>
    from osprofiler.drivers import elasticsearch_driver # noqa
  File "/usr/lib/python2.7/site-packages/osprofiler/drivers/elasticsearch_driver.py", line 18, in <module>
    from oslo_config import cfg
  File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 509, in <module>
    from oslo_config import iniparser
  File "/usr/lib/python2.7/site-packages/oslo_config/iniparser.py", line 16, in <module>
    class ParseError(Exception):
KeyboardInterrupt
[root@localhost ~]# export OS_USERNAME=admin
[root@localhost ~]# export OS_PASSWORD=server
[root@localhost ~]# export OS_TENANT_NAME=admin
[root@localhost ~]# export OS_AUTH_URL=http://localhost:5000/v3
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# neutron net-create External1 --provider:network_type flat --provider:physical_network br-ex --router:external=true --shared
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-06f8141b-a387-44b0-8e3b-1dbd65563232)
[root@localhost ~]# neutron net-create External2 --provider:network_type flat --provider:physical_network eno16777736 --router:external=true --shared
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-09fe0995-c646-4ff3-9dbf-df9593394698)
[root@localhost ~]# neutron net-create External3 --provider-physical-network provider --provider:physical_network eno16777736 --router:external=true --shared
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-21229781-fe7a-47ac-ac38-015cb2b79f23)
[root@localhost ~]# openstack subnet create --network provider \
>   --allocation-pool start=192.168.43.2,end=192.168.43.240 \
>   --dns-nameserver 192.168.43.1 --gateway 192.168.43.1 \
>   --subnet-range 192.168.43.0/24 provider
Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-404b8087-cafa-4a8d-b843-f337fe4a3901)
[root@localhost ~]#
[root@localhost ~]# mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 75028
Server version: 10.1.20-MariaDB MariaDB Server

Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> create database neutron;
ERROR 1007 (HY000): Can't create database 'neutron'; database exists
MariaDB [(none)]> grant all privileges on neutron.* to 'neutron'@'localhost' identified by 'server';
Query OK, 0 rows affected (0.90 sec)

MariaDB [(none)]> grant all privileges on neutron.* to 'neutron'@'%' identified by 'server';
Query OK, 0 rows affected (0.05 sec)

MariaDB [(none)]> quit
Bye
[root@localhost ~]#
[root@localhost ~]# export | grep OS_declare -x OS_AUTH_URL="https://192.168.64.128:5000/v3"
grep: OS_AUTH_URL=https://192.168.64.128:5000/v3: No such file or directory
[root@localhost ~]#
[root@localhost ~]# #source admin-openrc.sh
[root@localhost ~]# . /root/keystonerc_admin
[root@localhost ~(keystone_admin)]# openstack user create --domain default --password-prompt neutron

Message from syslogd@localhost at Jan 19 01:13:37 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#2 stuck for 26s! [neutron-server:108188]
User Password:
Repeat User Password:
Conflict occurred attempting to store user - Duplicate entry found with name neutron at domain ID default. (HTTP 409) (Request-ID: req-966b39eb-db15-42ad-ad1b-b225645fd1bb)
[root@localhost ~(keystone_admin)]# ^C
You have new mail in /var/spool/mail/root
[root@localhost ~(keystone_admin)]#
[root@localhost ~(keystone_admin)]# openstack role add --project service --user neutron admin
No project with a name or ID of 'service' exists.
[root@localhost ~(keystone_admin)]# openstack service create --name neutron --description "OpenStack Networking" network
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Networking |
| enabled | True |
| id | 7c09e68b5f4444aeb64d724ff65b0bdb |
| name | neutron |
| type | network |
+-------------+----------------------------------+
[root@localhost ~(keystone_admin)]# openstack endpoint create --region RegionOne network public http://controller:9696
Multiple service matches found for 'network', use an ID to be more specific.
[root@localhost ~(keystone_admin)]# openstack endpoint create --region RegionOne network internal http://controller:9696
Multiple service matches found for 'network', use an ID to be more specific.
[root@localhost ~(keystone_admin)]# openstack endpoint create --region RegionOne network admin http://controller:9696
Multiple service matches found for 'network', use an ID to be more specific.
[root@localhost ~(keystone_admin)]#

[root@localhost ~(keystone_admin)]# openstack baremetal node clean 97effffc1f96444d94143b352e37e2fb
usage: openstack baremetal node clean [-h] [--wait [<time-out>]] --clean-steps
                                      <clean-steps>
                                      <node>
openstack baremetal node clean: error: argument --clean-steps is required
[root@localhost ~(keystone_admin)]# openstack baremetal node clean ironic \
>     --clean-steps '[{"interface": "deploy", "step": "erase_devices"}]'
Error contacting Ironic server: Unable to establish connection to https://192.168.43.110:6385/v1/nodes/ironic/states/provision: HTTPSConnectionPool(host='192.168.43.110', port=6385): Max retries exceeded with url: /v1/nodes/ironic/states/provision (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1be6fde590>: Failed to establish a new connection: [Errno 111] Connection refused',)). Attempt 6 of 6
Unable to establish connection to https://192.168.43.110:6385/v1/nodes/ironic/states/provision: HTTPSConnectionPool(host='192.168.43.110', port=6385): Max retries exceeded with url: /v1/nodes/ironic/states/provision (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1be6fde590>: Failed to establish a new connection: [Errno 111] Connection refused',))

well it seems that the cleaner wouldn't clean and I dought that the service is there lets see:

[root@localhost ~(keystone_admin)]# sudo systemctl enable openstack-ironic-conductor
[root@localhost ~(keystone_admin)]# sudo systemctl restart openstack-ironic-conductor
[root@localhost ~(keystone_admin)]# sudo systemctl status openstack-ironic-conductor
● openstack-ironic-conductor.service - OpenStack Ironic Conductor service
   Loaded: loaded (/usr/lib/systemd/system/openstack-ironic-conductor.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2019-01-19 01:35:38 EST; 2s ago
 Main PID: 119599 (ironic-conducto)
    Tasks: 1
   CGroup: /system.slice/openstack-ironic-conductor.service
           └─119599 /usr/bin/python2 /usr/bin/ironic-conductor

Jan 19 01:35:38 localhost.localdomain systemd[1]: Started OpenStack Ironic Co...
Hint: Some lines were ellipsized, use -l to show in full.
[root@localhost ~(keystone_admin)]#

it's running and active then it's not installed ... shixxxxx ... the packstack failed (did it while I was sleeping)

anyway that's what I want to tell you now for now .. I might pull out the original VM and packstack again with ironic installation set to y and have a fresh start and execute the script:

export OS_USERNAME=admin
export OS_PASSWORD=server
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://localhost:5000/v3

neutron net-create External1 --provider:network_type flat --provider:physical_network br-ex --router:external=true --shared
neutron net-create External2 --provider:network_type flat --provider:physical_network eno16777736 --router:external=true --shared
neutron net-create External3 --provider-physical-network provider --provider:physical_network eno16777736 --router:external=true --shared
openstack subnet create --network provider \
  --allocation-pool start=192.168.43.2,end=192.168.43.240 \
  --dns-nameserver 192.168.43.1 --gateway 192.168.43.1 \
  --subnet-range 192.168.43.0/24 provider

mysql
create database neutron;
grant all privileges on neutron.* to 'neutron'@'localhost' identified by 'server';
grant all privileges on neutron.* to 'neutron'@'%' identified by 'server';
quit

export | grep OS_declare -x OS_AUTH_URL="https://192.168.64.128:5000/v3"

#source admin-openrc.sh
. /root/keystonerc_admin
openstack user create --domain default --password-prompt neutron
openstack role add --project service --user neutron admin
openstack service create --name neutron --description "OpenStack Networking" network
openstack endpoint create --region RegionOne network public http://controller:9696
openstack endpoint create --region RegionOne network internal http://controller:9696
openstack endpoint create --region RegionOne network admin http://controller:9696

systemctl enable neutron-server.service neutron-openvswitch-agent.service neutron-dhcp-agent.service neutron-metadata-agent.service neutron-ovs-cleanup.service
systemctl start neutron-server.service neutron-openvswitch-agent.service neutron-dhcp-agent.service neutron-metadata-agent.service neutron-ovs-cleanup.service
systemctl enable neutron-l3-agent.service
systemctl start neutron-l3-agent.service

which is my collection from the Bugs reports sites for Openstack ..

[root@localhost network-scripts]# systemctl status network -l
● network.service - LSB: Bring up/down networking
   Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 2019-01-19 03:47:01 EST; 21s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 86319 ExecStop=/etc/rc.d/init.d/network stop (code=exited, status=0/SUCCESS)
  Process: 86591 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=1/FAILURE)
    Tasks: 0

Jan 19 03:47:01 localhost.localdomain dhclient[86963]: Please report for this software via the Red Hat Bugzilla site:
Jan 19 03:47:01 localhost.localdomain dhclient[86963]:     http://bugzilla.redhat.com
Jan 19 03:47:01 localhost.localdomain dhclient[86963]: ution.
Jan 19 03:47:01 localhost.localdomain dhclient[86963]: exiting.
Jan 19 03:47:01 localhost.localdomain network[86591]: failed.
Jan 19 03:47:01 localhost.localdomain network[86591]: [FAILED]
Jan 19 03:47:01 localhost.localdomain systemd[1]: network.service: control process exited, code=exited status=1
Jan 19 03:47:01 localhost.localdomain systemd[1]: Failed to start LSB: Bring up/down networking.
Jan 19 03:47:01 localhost.localdomain systemd[1]: Unit network.service entered failed state.
Jan 19 03:47:01 localhost.localdomain systemd[1]: network.service failed.
[root@localhost network-scripts]#