Description of problem: neutron_sriov_agent_config(container-puppet-neutron) container fails with Compute sriov since /etc/udev/rules.d/70-tripleo-reset-sriov.rules is inaccessible. Container error: "2019-05-27 19:28:45,392 ERROR: 17602 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', 'container-puppet-neutron', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,neutron_config,neutron_agent_ovs,neutron_plugin_ml2,neutron_config,neutron_agent_sriov_numvfs,neutron_sriov_agent_config', '--env', 'NAME=neutron', '--env', 'HOSTNAME=overcloud-computesriov-0', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--log-driver', 'json-file', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmp41e8fzk9:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/dev/log:/dev/log:rw', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-neutron.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', '/lib/modules:/lib/modules:ro', '--volume', '/run/openvswitch:/run/openvswitch:shared,z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhosp15/openstack-neutron-server:20190509.1'] run failed after + mkdir -p /etc/puppet", "+ '[' -n file,file_line,concat,augeas,cron,neutron_config,neutron_agent_ovs,neutron_plugin_ml2,neutron_config,neutron_agent_sriov_numvfs,neutron_sriov_agent_config ']'", "+ TAGS='--tags file,file_line,concat,augeas,cron,neutron_config,neutron_agent_ovs,neutron_plugin_ml2,neutron_config,neutron_agent_sriov_numvfs,neutron_sriov_agent_config'", "+ origin_of_time=/var/lib/config-data/neutron.origin_of_time", "+ touch /var/lib/config-data/neutron.origin_of_time", "+ /usr/bin/puppet apply --summarize --detailed-exitcodes --color=false --logdest syslog --logdest console --modulepath=/etc/puppet/modules:/usr/share/openstack-puppet/modules --tags file,file_line,concat,augeas,cron,neutron_config,neutron_agent_ovs,neutron_plugin_ml2,neutron_config,neutron_agent_sriov_numvfs,neutron_sriov_agent_config /etc/config.pp", "+ rc=6", "+ '[' 6 -ne 2 -a 6 -ne 0 ']'", Puppet error message: overcloud-computeovsdpdksriov-0 puppet-user[16]: Could not set 'file' on ensure: No such file or directory @ dir_s_mkdir - /etc/udev/rules.d/70-tripleo-reset-sriov.rules20190528-16-rx5pz4.lock (file: /etc/puppet/modules/tripleo/manifests/host/sriov/numvfs_persistence.pp, line: 46) Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1.Deploy with existing configs 2. 3. Actual results: since /etc/udev is not mounted in the container, puppet apply fails as /etc/udev/rules.d/70-tripleo-reset-sriov.rules is not availabe. Expected results: /etc/udev is avaialbe in the container-puppet-neutron and exits without error. Additional info:
Created attachment 1576397 [details] compute sriov sos report
Since relevant code has not changed much since previous versions (where it was working) [0], the easy fix looks indeed to bind mount /etc/udev - which was probably the case before [0] https://github.com/openstack/puppet-tripleo/commits/master/manifests/host/sriov/numvfs_persistence.pp
tripleo::profile::base::neutron::sriov is executing the host configuration which results in the host config being run inside of the agent's container. This is wrong and should only be executed through the host config.
Master fix merged, stable/stein backport in progress
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:2811