Hide Forgot
Description of problem: When using OSP 8 director to upgrade an OSP 7 installation that was originally deployed as 7.1 or 7.2, the converge step of the upgrade will fail with the following puppet error: puppet apply 0d5c39da-44bf-439c-9c01-9ccb3b6ea52a.pp Could not retrieve fact='apache_version', resolution='<anonymous>': undefined method `[]' for nil:NilClass Could not retrieve fact='apache_version', resolution='<anonymous>': undefined method `[]' for nil:NilClass Error: Invalid resource type kmod::load at /var/lib/heat-config/heat-config-puppet/0d5c39da-44bf-439c-9c01-9ccb3b6ea52a.pp:19 on node overcloud-compute-0.localdomain Wrapped exception: Invalid resource type kmod::load Error: Invalid resource type kmod::load at /var/lib/heat-config/heat-config-puppet/0d5c39da-44bf-439c-9c01-9ccb3b6ea52a.pp:19 on node overcloud-compute-0.localdomain This occurs on controller, compute, and ceph nodes in my deployment. This is similar to these issues: https://ask.openstack.org/en/question/88671/heat-stack-failed-with-invalid-resource-type-kmodload/ https://bugzilla.redhat.com/show_bug.cgi?id=1310651 Basically the issue is that new puppet modules have been added to the image, but are not linked correctly in /etc/puppet. You can see this by doing the following on one of the nodes: ls /usr/share/openstack-puppet/modules | while read line do if [ ! -L /etc/puppet/modules/$line ]; then echo "$line not a link" fi done You'll see several modules not linked although kmod is the one being complained about. I linked all remaining modules as follows: ls /usr/share/openstack-puppet/modules | while read line do if [ ! -L /etc/puppet/modules/$line ]; then echo "Adding link /etc/puppet/modules/$line" ln -s /usr/share/openstack-puppet/modules/$line /etc/puppet/modules/$line fi done After this the converge deploy ran fine without the invalid kmod message. Version-Release number of selected component (if applicable): RHEL OSP 8 / Director 8 GA How reproducible: I've only tried once in my environment, but believe this will impact anyone originally starting with a deployment less than 7.3 Steps to Reproduce: 1. Deploy RHEL OSP 7.1 or 7.2 2. Update to latest 7.x packages 3. Follow documented process to upgrade from latest 7.x to 8.0 GA