Description of problem: Running "openstack overcloud update -i overcloud" fails with error message: "ln: failed to create symbolic link '/usr/local/bin/ovs-lib': File exists". This is due to the following code in pacemaker_common_functions.sh: function change_ovs_runtime_mode { ovs_src_service_path="/usr/lib/systemd/system/ovs-vswitchd.service" ovs_service_path="${ovs_src_service_path/usr\/lib/etc}" ovs_db_src_service_path=/usr/lib/systemd/system/ovsdb-server.service ovs_db_service_path="${ovs_db_src_service_path/usr\/lib/etc}" ovs_src_ctl_path="/usr/share/openvswitch/scripts/ovs-ctl" ovs_ctl_path="/usr/local/bin/ovs-ctl" ovs_src_lib_path="/usr/share/openvswitch/scripts/ovs-lib" ovs_lib_path="/usr/local/bin/ovs-lib" ln -s $ovs_src_lib_path $ovs_lib_path which gets triggered further down: function special_case_ovs_upgrade_if_needed { […] local ovs_version=$(rpm -q --queryformat '%{VERSION}' openvswitch) local major_version=`echo $ovs_version | cut -d. -f1` local minor_version=`echo $ovs_version | cut -d. -f2` local version_to_number=$(($major_version*10+$minor_version)) if (( $version_to_number >= 28 )); then change_ovs_2_9_user change_ovs_2_9_perms # ensure that from this minor update onwards, we need the workarounds touch /etc/systemd/system/apply-ovs-runtime-mode-workaround fi # If the apply-*-workaround file is present, ensure it is updated to the # latest from the package with permissions modified. if [ -f "/etc/systemd/system/apply-ovs-runtime-mode-workaround" ]; then change_ovs_runtime_mode fi The issue is that this gets re-run on every update, but after the first update the symlink exists and thus errors out. Either the condition to check if this is needed needs to be improved or "-f" flag should be added. Version-Release number of selected component (if applicable): openstack-tripelo-heat-templates-5.3.10-17.el7ost.noarch How reproducible: always Steps to Reproduce: 1. Install OSP10 (with OVS < 2.9; not sure if this is a required condition) 2. Update OSP10 to a version with at least OVS >= 2.9 3. Update following the documentation, i.e. via "openstack overcloud update -i overcloud" Actual results: Update fails with: "ln: failed to create symbolic link '/usr/local/bin/ovs-lib': File exists". Expected results: Update completes successfully
Re-reading I noticed that I should clarify that you need to run the update 2 times. The first one works (the symlink gets created), the second one fails (the symlink exists and leads to a conflict).
Handling this to dfg networking. The hack mentioned doesn't work anymore as the packaging is doing it now. Maybe Zenghui can help as he did the original workaround.
Reassigning since ovs package doesn't use "/usr/local/" at all
Reproducer: Deploy OSP10 z4 RHEL7.3 Minor update to OSP10 z9 RHEL7.5 Minor update to latest OSP10 RHEL7.6 Anyone who did minor update to Z9 will not be able to update further. This is a serious blocker.
{ "status": "FAILED", "server_id": "fd8d778c-608e-4db6-adb7-9ee5c240e3ae", "config_id": "1db457c8-0dd7-44e4-aee3-82a408a7a1d5", "output_values": { "deploy_stdout": "Started yum_update.sh on server fd8d778c-608e-4db6-adb7-9ee5c240e3ae at Thu Nov 29 10:47:52 UTC 2018\nChecking openstack-nova-migration is installed\nLoaded plugins: product-id, search-disabled-repos, subscription-manager,\n : versionlock\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\nMetadata Cache Created\nChecking for ceph-osd dependency issues\nceph-osd package is required (there are OSD entries in /var/lib/ceph/osd)\nLoaded plugins: product-id, search-disabled-repos, subscription-manager,\n : versionlock\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\nMetadata Cache Created\nSkipping manual upgrade of openvswitch - no restart in postun detected. Performing automated upgrade\nLoaded plugins: product-id, search-disabled-repos, subscription-manager,\n : versionlock\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\nResolving Dependencies\n--> Running transaction check\n---> Package openvswitch.x86_64 0:2.9.0-56.el7fdp will be updated\n---> Package openvswitch.x86_64 0:2.9.0-70.el7fdp.1 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n openvswitch x86_64 2.9.0-70.el7fdp.1 rhelosp-10.0-puddle 7.9 M\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package\n\nTotal size: 7.9 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : openvswitch-2.9.0-70.el7fdp.1.x86_64 1/2 \n Cleanup : openvswitch-2.9.0-56.el7fdp.x86_64 2/2 \n Verifying : openvswitch-2.9.0-70.el7fdp.1.x86_64 1/2 \n Verifying : openvswitch-2.9.0-56.el7fdp.x86_64 2/2 \n\nUpdated:\n openvswitch.x86_64 0:2.9.0-70.el7fdp.1 \n\nComplete!\n", "deploy_stderr": "Repository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nRepository rhelosp-rhel-pending is listed more than once in the configuration\nRepository rhelosp-extras-pending is listed more than once in the configuration\nRepository el7-rhos-test-deps is listed more than once in the configuration\nln: failed to create symbolic link \u2018/usr/local/bin/ovs-lib\u2019: File exists\n", "update_managed_packages": "false", "deploy_status_code": 1 }, "creation_time": "2018-11-27T11:28:49Z", "updated_time": "2018-11-29T10:48:43Z", "input_values": { "update_identifier": "1543488153" }, "action": "UPDATE", "status_reason": "deploy_status_code : Deployment exited with non-zero status code: 1", "id": "e74bfe22-9c41-4ca8-b6a7-473f43555349" }
working on fix : https://code.engineering.redhat.com/gerrit/#/c/157247/
Updating status and component
(In reply to zenghui.shi from comment #9) > working on fix : https://code.engineering.redhat.com/gerrit/#/c/157247/ fix merged downstream, updating status and component
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0055