Description of problem: Upgraded undercloud OSP10 -> OSP11 and immediately after upgrade encountered: 2018-06-21 19:32:05,997 INFO: Not creating default plan "overcloud" because it already exists. 2018-06-21 19:32:06,766 INFO: ############################################################################# Undercloud upgrade complete. The file containing this installation's passwords is at /home/stack/undercloud-passwords.conf. There is also a stackrc file at /home/stack/stackrc. These files are needed to interact with the OpenStack services, and should be secured. ############################################################################# [stack@chrisjffu-undercloud ~]$ nova list ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'sqlalchemy.exc.ProgrammingError'> (HTTP 500) (Request-ID: req-d8f4a23f-c18c-4e29-a1e3-e8d5ce6134ca) Later found out the OSP10 repos have not been entirely disabled during the upgrade. Re-ran the undercloud upgrade with osp10 repos removed properly and it corrected the problem. I am opening this bug for 2 reasons: 1. I want to see if there is anything we could do from to prevent users for getting into this issue 2. Make this issue searchable in case anyone else hit it during the upgrade Version-Release number of selected component (if applicable): OSP10->OSP11 How reproducible: Steps to Reproduce: 1. Deploy OSP10 undercloud 2. Add osp11 repos but don't remove osp10 3. run openstack undercloud upgrade Actual results: nova list fails Expected results: nova list succeeds Additional info: Attaching sosreport
Created attachment 1453729 [details] sosreport-parta
Created attachment 1453730 [details] sos-report-partb
Chris thanks for the report, can you please share more high level info about how was the OSP 10/11 repository situation different on the 1st and 2nd upgrade attempt: Did 1st attempt have OSP 11 repos enabled fully or somehow partially? Did you use subscription manager or rhos-release? In theory having OSP 10 repository enabled alongside OSP 11 shouldn't cause an issue, as the packages in OSP 11 repo would either be newer or the same, so the final set of RPMs should be the same too. That's assuming that *all* OSP 11 content was enabled. (Triaging as medium/medium for now, as re-run of the undercloud upgrade with only OSP 11 repos worked. Let's increase the prio if we manage to reproduce the issue.)
I think this could potentially be a duplicate of BZ#1594265
it has been deployed with rhos release .. both repos (osp10 and osp11) have been enabled at the same time when it failed. I will have to go through that exercise multiple times in the next few days. I'll make note if I see it again.
Got the same error today after osp10->osp11 upgrade (undercloud). Used latest CDN repos. Resolved by workaround from BZ in comment #4 [stack@test1-undercloud ~]$ nova list ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'sqlalchemy.exc.ProgrammingError'> (HTTP 500) (Request-ID: req-490ce0c7-4055-4a25-8bcd-adccc8a43bbd) [stack@test1-undercloud ~]$ openstack server list Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'sqlalchemy.exc.ProgrammingError'> (HTTP 500) (Request-ID: req-da19369a-93bf-44a7-8565-f45f38703ce9) [stack@test1-undercloud ~]$ sudo nova-manage db sync [stack@test1-undercloud ~]$ openstack server list +--------------------------------------+----------------------+--------+-----------------------+------------+ | ID | Name | Status | Networks | Image Name | +--------------------------------------+----------------------+--------+-----------------------+------------+ | c085c93f-8363-4224-ba48-d5a23fdc9f99 | testffu-controller-2 | ACTIVE | ctlplane=172.16.0.114 | | | 69b6c9c7-3545-447f-848e-968342f94bed | testffu-controller-0 | ACTIVE | ctlplane=172.16.0.106 | | | 8d4a79da-227f-4673-936b-7180e663cb4a | testffu-controller-1 | ACTIVE | ctlplane=172.16.0.107 | | | 5e662677-fca1-4a36-8f47-0509df285bfd | testffu-compute-1 | ACTIVE | ctlplane=172.16.0.113 | | | b854618b-0759-4ab6-9e4a-f885c93bf023 | testffu-compute-0 | ACTIVE | ctlplane=172.16.0.108 | | +--------------------------------------+----------------------+--------+-----------------------+------------+
*** This bug has been marked as a duplicate of bug 1594265 ***