Since the 3.7 to 3.9 upgrade is multipart, 3.7 to 3.8 to 3.9, can you check which version of openshift is currently running when this happens and/or provide the complete log which would show where in the upgrade phase this fails.
Assigning to master team to evaluate the nature of the failure.
I've created https://github.com/openshift/openshift-docs/pull/8767 which adds this issue to known issues. The fix is to manually edit the failed services removing the duplicate name+port pairs and re-run the storage.
Once the PR merges to docs, I'm going to close this issue as a won't fix.
Comment 14openshift-github-bot
2018-05-01 13:39:35 UTC
Description of problem: Version-Release number of the following components: :>rpm -qa | grep atomic atomic-openshift-clients-3.9.14-1.git.0.4efa2ca.el7.x86_64 atomic-openshift-master-3.9.14-1.git.0.4efa2ca.el7.x86_64 atomic-openshift-utils-3.9.14-1.git.3.c62bc34.el7.noarch atomic-openshift-3.9.14-1.git.0.4efa2ca.el7.x86_64 atomic-registries-1.20.1-9.git436cf5d.el7.x86_64 atomic-openshift-excluder-3.9.14-1.git.0.4efa2ca.el7.noarch atomic-openshift-sdn-ovs-3.9.14-1.git.0.4efa2ca.el7.x86_64 atomic-openshift-docker-excluder-3.9.14-1.git.0.4efa2ca.el7.noarch atomic-openshift-node-3.9.14-1.git.0.4efa2ca.el7.x86_64 Run the Automated In-Place Upgrade Playbook. # ansible-playbook -i </path/to/inventory/file> /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_9/upgrade.yml -vvv Failing at with the error: TASK [Upgrade all storage] **************************************************************************************************************************************************************************** fatal: [mas-3-01.example.com]: FAILED! => {"changed": true, "cmd": ["oc", "adm", "--config=/etc/origin/master/admin.kubeconfig", "migrate", "storage", "--include=*", "--confirm"], "delta": "0:07:06.574833", "end": "2018-04-03 11:22:07.834827", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2018-04-03 11:15:01.259994", "stderr": "", "stderr_lines": [], "stdout": "E0403 11:22:04.100394 error: -n oneid-rtp1 services/consul-ingress: Service \"consul-ingress\" is invalid: spec.ports[5]: Duplicate value: api.ServicePort{Name:\"\", Protocol:\"TCP\", Port:8500, TargetPort:intstr.IntOrString{Type:0, IntVal:0, StrVal:\"\"}, NodePort:0}\nsummary: total=37768 errors=1 ignored=0 unchanged=37767 migrated=0\ninfo: to rerun only failing resources, add --include=services\nerror: 1 resources failed to migrate", "stdout_lines": ["E0403 11:22:04.100394 error: -n oneid-rtp1 services/consul-ingress: Service \"consul-ingress\" is invalid: spec.ports[5]: Duplicate value: api.ServicePort{Name:\"\", Protocol:\"TCP\", Port:8500, TargetPort:intstr.IntOrString{Type:0, IntVal:0, StrVal:\"\"}, NodePort:0}", "summary: total=37768 errors=1 ignored=0 unchanged=37767 migrated=0", "info: to rerun only failing resources, add --include=services", "error: 1 resources failed to migrate"]} to retry, use: --limit @/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_9/upgrade_control_plane.retry When manually ran the command: :>oc adm migrate storage --include=* --confirm E0403 12:07:10.228771 error: -n oneid-rtp1 services/consul-ingress: Service "consul-ingress" is invalid: spec.ports[5]: Duplicate value: api.ServicePort{Name:"", Protocol:"TCP", Port:8500, TargetPort:intstr.IntOrString{Type:0, IntVal:0, StrVal:""}, NodePort:0} summary: total=37769 errors=1 ignored=0 unchanged=37768 migrated=0 info: to rerun only failing resources, add --include=services error: 1 resources failed to migrate Expected results: Then upgrade should be successfull