Bug 1878511

Description of problem:

After upgrading RHOSP13 to RHOSP16.1, I'm following the documentation below to migrate ml2ovs to ovn.
 [1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/networking_with_open_virtual_network/migrating-ml2ovs-to-ovn

While running "ovn_migration.sh setup-mtu-t1", I noticed that neutron dhcp agent containers are not restarted because of missing "docker" command in overcloud nodes.
The playbook execution shows success because it ignores these failures.
~~~
(overcloud) [stack@undercloud-0 ovn_migration]$ ovn_migration.sh setup-mtu-t1
ansible-playbook 2.9.13
  config file = /home/stack/ovn_migration/ansible.cfg
  configured module search path = ['/home/stack/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.6.8 (default, Dec  5 2019, 15:45:45) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
...

TASK [Get the neutron dhcp agent docker id] **************************************************************************************************************************************************************************************************
task path: /home/stack/ovn_migration/playbooks/reduce-dhcp-renewal-time.yml:16
Sunday 13 September 2020  14:50:26 +0000 (0:00:00.836)       0:00:04.606 ****** 
changed: [controller-0] => {"changed": true, "cmd": "docker ps | grep neutron_dhcp | awk '{print $1}'", "delta": "0:00:00.006763", "end": "2020-09-13 14:50:26.998275", "rc": 0, "start": "2020-09-13 14:50:26.991512", "stderr": "/bin/sh: docker: command not found", "stderr_lines": ["/bin/sh: docker: command not found"], "stdout": "", "stdout_lines": []}
...

TASK [Restart neutron dhcp agent] ************************************************************************************************************************************************************************************************************
task path: /home/stack/ovn_migration/playbooks/reduce-dhcp-renewal-time.yml:22
Sunday 13 September 2020  14:50:27 +0000 (0:00:00.715)       0:00:05.322 ****** 
fatal: [controller-0]: FAILED! => {"changed": false, "cmd": "docker restart", "msg": "[Errno 2] No such file or directory: b'docker': b'docker'", "rc": 2}
...ignoring
...
META: ran handlers
META: ran handlers

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
controller-0               : ok=4    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=1   
controller-1               : ok=4    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=1   
controller-2               : ok=4    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=1   
~~~

Actually docker command is not available in overcloud nodes after upgrade.
The migration playbook should use podman, or install podman-docker if it relies on docker command.
~~~
[heat-admin@controller-0 ~]$ rpm -qa | grep podman | sort
podman-1.6.4-16.module+el8.2.0+7659+b700d80e.x86_64
[heat-admin@controller-0 ~]$ which docker
/usr/bin/which: no docker in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/heat-admin/.local/bin:/home/heat-admin/bin)
~~~

After installing podman-docker in overcloud nodes, dhcp agents are restarted properly by the playbook.
~~~
(overcloud) [stack@undercloud-0 ovn_migration]$ ovn_migration.sh setup-mtu-t1
...

TASK [Get the neutron dhcp agent docker id] **************************************************************************************************************************************************************************************************
task path: /home/stack/ovn_migration/playbooks/reduce-dhcp-renewal-time.yml:16
Sunday 13 September 2020  14:59:33 +0000 (0:00:01.136)       0:00:01.252 ****** 
changed: [controller-0] => {"changed": true, "cmd": "docker ps | grep neutron_dhcp | awk '{print $1}'", "delta": "0:00:00.125135", "end": "2020-09-13 14:59:34.368226", "rc": 0, "start": "2020-09-13 14:59:34.243091", "stderr": "Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.", "stderr_lines": ["Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg."], "stdout": "efb2283e823b", "stdout_lines": ["efb2283e823b"]}
...

TASK [Restart neutron dhcp agent] ************************************************************************************************************************************************************************************************************
task path: /home/stack/ovn_migration/playbooks/reduce-dhcp-renewal-time.yml:22
Sunday 13 September 2020  14:59:34 +0000 (0:00:00.897)       0:00:02.150 ****** 
changed: [controller-0] => {"changed": true, "cmd": ["docker", "restart", "efb2283e823b"], "delta": "0:00:10.882992", "end": "2020-09-13 14:59:45.909760", "rc": 0, "start": "2020-09-13 14:59:35.026768", "stderr": "Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.\ntime=\"2020-09-13T14:59:45Z\" level=error msg=\"Failed to remove paths: map[cpuset:/sys/fs/cgroup/cpuset/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope freezer:/sys/fs/cgroup/freezer/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope hugetlb:/sys/fs/cgroup/hugetlb/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope net_cls:/sys/fs/cgroup/net_cls,net_prio/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope net_prio:/sys/fs/cgroup/net_cls,net_prio/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope perf_event:/sys/fs/cgroup/perf_event/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope]\"", "stderr_lines": ["Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.", "time=\"2020-09-13T14:59:45Z\" level=error msg=\"Failed to remove paths: map[cpuset:/sys/fs/cgroup/cpuset/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope freezer:/sys/fs/cgroup/freezer/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope hugetlb:/sys/fs/cgroup/hugetlb/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope net_cls:/sys/fs/cgroup/net_cls,net_prio/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope net_prio:/sys/fs/cgroup/net_cls,net_prio/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope perf_event:/sys/fs/cgroup/perf_event/machine.slice/libpod-efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563.scope]\""], "stdout": "efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563", "stdout_lines": ["efb2283e823bd7f630f5ebe583ef57428bc7b9b4e5649cd06454168039bb0563"]}
...
META: ran handlers
META: ran handlers

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
controller-0               : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
controller-1               : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
controller-2               : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
~~~


Version-Release number of selected component (if applicable):
python3-networking-ovn-migration-tool-7.2.1-0.20200611133439.15f2281.el8ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. Upgrade RHOSP13 to RHOSP16.1 (*1)
2. Follow Migration document

(*1) We can reproduce the same situation by uninstalling podman-docker package from deployed overcloud nodes

Actual results:
neutron dhcp containers are not restarted during "ovn_migration.sh setup-mtu-t1"

Expected results:
neutron dhcp containers are restarted during "ovn_migration.sh setup-mtu-t1"

Additional info: