Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1646641

Summary: The ironic-conductor image must have staging-ovirt RPMS if staging-ovirt driver needs to be supported. Otherwise the container is constantly restarting.
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: openstack-tripleo-commonAssignee: Harald Jensås <hjensas>
Status: CLOSED ERRATA QA Contact: Alexander Chuzhoy <sasha>
Severity: high Docs Contact:
Priority: high    
Version: 14.0 (Rocky)CC: bfournie, hjensas, mburns, racedoro, slinaber, tvignaud
Target Milestone: rcKeywords: Triaged
Target Release: 14.0 (Rocky)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-common-9.4.1-0.20181012010877.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-01-11 11:54:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alexander Chuzhoy 2018-11-05 19:33:39 UTC
'openstack baremetal driver list' returns an empty list.

Environment:
instack-undercloud-9.4.1-0.20180928005746.15cda5a.el7ost.noarch
python-ovirt-engine-sdk4-4.2.6-1.el7ev.x86_64
python2-ironicclient-2.5.0-0.20180810135843.fb94fb8.el7ost.noarch
python2-ironic-inspector-client-3.3.0-0.20180810080932.53bf4e8.el7ost.noarch
puppet-ironic-13.3.1-0.20181013115248.d85f830.el7ost.noarch


Steps to reproduce:
1.
Install python-ovirt-engine-sdk4

2.
Configure undercloud.conf with:
(undercloud) [stack@undercloud ~]$ grep -v "^$\|^#" undercloud.conf
[DEFAULT]
enabled_hardware_types = staging-ovirt
local_interface = eth1
local_ip = 192.168.0.1/24
undercloud_admin_host = 192.168.0.3
undercloud_public_host = 192.168.0.2
container_images_file = /home/stack/containers-prepare-parameter.yaml
undercloud_ntp_servers = clock.redhat.com
docker_insecure_registries = docker-registry.engineering.redhat.com,brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888
[ctlplane-subnet]
cidr = 192.168.0.0/24
dhcp_end = 192.168.0.24
dhcp_start = 192.168.0.5
gateway = 192.168.0.1
inspection_iprange = 192.168.0.100,192.168.0.120
masquerade = true

3.
deploy undercloud with 'openstack undercloud install'.


4. run 'openstack baremetal driver list'

Result:
empty list


Expected result:
staging-ovirt driver.


Note that having:
enabled_hardware_types = ipmi,redfish,ilo,idrac,staging-ovirt

Also results in empty driver list.

Comment 1 Alexander Chuzhoy 2018-11-05 23:53:22 UTC
Having:
enabled_hardware_types = ipmi,ilo,idrac

Works fine and 'openstack baremetal driver list' returns the specified 3 drivers.

Adding staging-ovirt:
enabled_hardware_types = ipmi,ilo,idrac,staging-ovirt

Results in empty list.

Comment 2 Alexander Chuzhoy 2018-11-06 15:54:15 UTC
The ironic_conductor container is constantly restarting:

2018-11-06 07:46:42.480 1 ERROR oslo_service.service [req-0aba91ab-718e-43a5-a26f-2d880cb9822c - - - - -] Error starting thread.: DriverNotFoundInEntrypoint: Could not find the following items in the 'ironic.hardware.types' entrypoint: staging-ovirt.
2018-11-06 07:46:42.480 1 ERROR oslo_service.service Traceback (most recent call last):
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 796, in run_service
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     service.start()
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/ironic/common/rpc_service.py", line 54, in start
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     self.manager.init_host(admin_context)
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/ironic/conductor/base_manager.py", line 127, in init_host
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     hardware_types = driver_factory.hardware_types()
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/ironic/common/driver_factory.py", line 253, in hardware_types
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     return _get_all_drivers(HardwareTypesFactory())
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/ironic/common/driver_factory.py", line 323, in __init__
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     self.__class__._init_extension_manager()
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 274, in inner
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     return f(*args, **kwargs)
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/ironic/common/driver_factory.py", line 393, in _init_extension_manager
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     on_missing_entrypoints_callback=missing_callback))
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/stevedore/named.py", line 85, in __init__
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     on_missing_entrypoints_callback(self._missing_names)
2018-11-06 07:46:42.480 1 ERROR oslo_service.service   File "/usr/lib/python2.7/site-packages/ironic/common/driver_factory.py", line 384, in missing_callback
2018-11-06 07:46:42.480 1 ERROR oslo_service.service     names=names, entrypoint=cls._entrypoint_name)
2018-11-06 07:46:42.480 1 ERROR oslo_service.service DriverNotFoundInEntrypoint: Could not find the following items in the 'ironic.hardware.types' entrypoint: staging-ovirt.
2018-11-06 07:46:42.480 1 ERROR oslo_service.service

Comment 3 Alexander Chuzhoy 2018-11-06 17:48:33 UTC
Was able to workaround the issue by rebuilding the docker image used by ironic_conductor to include:
 yum install -y python-ovirt-engine-sdk4 openstack-ironic-staging-drivers
After restarting the ironic_conductor container:
(undercloud) [stack@undercloud ~]$ openstack baremetal driver list                                                                                                                            
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.21.1) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
+---------------------+------------------+
| Supported driver(s) | Active host(s)   |
+---------------------+------------------+
| idrac               | undercloud.local |
| ilo                 | undercloud.local |
| ipmi                | undercloud.local |
| staging-ovirt       | undercloud.local |
+---------------------+------------------+

Comment 8 Alexander Chuzhoy 2018-11-21 22:33:29 UTC
FailedQA

Environment: openstack-tripleo-common-9.4.1-0.20181012010877.el7ost.noarch



(undercloud) [stack@undercloud ~]$ cat undercloud.conf 
[DEFAULT]
enabled_hardware_types = staging-ovirt
local_interface = eth1
local_ip = 192.168.0.1/24
undercloud_admin_host = 192.168.0.3
undercloud_public_host = 192.168.0.2
container_images_file = /home/stack/containers-prepare-parameter.yaml
undercloud_ntp_servers = clock.redhat.com
docker_insecure_registries = docker-registry.engineering.redhat.com,brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888
[ctlplane-subnet]
cidr = 192.168.0.0/24
dhcp_end = 192.168.0.24
dhcp_start = 192.168.0.5
gateway = 192.168.0.1
inspection_iprange = 192.168.0.100,192.168.0.120
masquerade = true


(undercloud) [stack@undercloud ~]$ sudo sudo docker ps|grep ironic_conductor
2fabeb13795c        192.168.0.1:8787/rhosp14/openstack-ironic-conductor:2018-11-20.1            "kolla_start"            22 minutes ago      Restarting (0) 2 seconds ago                       ironic_conductor
(undercloud) [stack@undercloud ~]$

Comment 9 Thierry Vignaud 2018-11-23 17:05:36 UTC
Containers definitions have been regenerated

Comment 11 Alexander Chuzhoy 2018-11-26 18:51:42 UTC
Verified:
openstack-tripleo-common-9.4.1-0.20181012010879.el7ost.noarch


The reported issue doesn't reproduce.
crudini --get undercloud.conf DEFAULT enabled_hardware_types
ipmi,staging-ovirt

openstack baremetal driver list
+---------------------+------------------+
| Supported driver(s) | Active host(s)   |
+---------------------+------------------+
| ipmi                | undercloud.local |
| staging-ovirt       | undercloud.local |
+---------------------+------------------+

Comment 14 errata-xmlrpc 2019-01-11 11:54:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045