Red Hat Bugzilla – Bug 984837
Postgresql cartridges were not been upgraded successfully on some nodes in STG
Last modified: 2015-05-14 19:24:04 EDT
Description of problem: Adding a Postgresql-9.2 cartridge to an app sometimes fails with the following error: Running Command - rhc cartridge add postgresql-9.2 -a nodejsph8v0av -l jizhao+1@redhat.com -p 'redhat' --timeout 360 Adding postgresql-9.2 to application 'nodejsph8v0av' ... Unexpected error: key not found: (postgresql, 9.2, _) Command Return: 1 The problem could root in that some nodes in STG did not get upgraded successfully, there might be even no postgresql-9.2 cartridge yet. Created an app with postgresql-8.4 added, which was still an old versioned postgresql according to the environment variables below: OPENSHIFT_POSTGRES_VERSION=8.4 OPENSHIFT_POSTGRESQL_IDENT=redhat:postgresql:8.4:0.2.0 However, this is not reproduced when I tried again, possibly some nodes are upgraded successfully, but some nodes fails Version-Release number of selected component (if applicable): STG(devenv-stage_406) How reproducible: Sometimes Steps to Reproduce: 1. create an app, for example, a ruby-1.9 app rhc app create rb9 ruby-1.9 2. Add a Postgresql-9.2 cartridge to this app rhc cartridge add postgresql-9.2 -a rb9 Actual results: Adding a Postgresql-9.2 sometimes fails with the error: Adding postgresql-9.2 to application 'nodejsph8v0av' ... Unexpected error: key not found: (postgresql, 9.2, _) Expected results: Adding a Postgresql-9.2 cartridge should be successful in the scenario above. The version in POSTGRESQL_IDENT should be updated to 0.0.3 at least. Additional info:
It sounds like the new postgres cartridge RPM is not available on all the nodes. Ops needs to verify that the new package is installed everywhere.
It is available and installed everywhere, consistency checks passed as well.
Could you test this again? This should be OK now, given Adam's comment above. Thank you.
No, nothing changed in STG since yesterday's deployment (not yet anyways, we do have a deploy scheduled for later today). Also note, I suspect this to be related: https://bugzilla.redhat.com/show_bug.cgi?id=985063
This is expected to be resolved, it's not that some nodes had an issue but there was a strange rails caching issue on the brokers which was causing cache responses to deliver the incorrect list of carts.
Referring to the results of Postgresql-9.2 cases in today's auto test against STG(devenv-stage_410), no failures were detected due to this bug. Thus, move it to VERIFIED. Thanks, Zhe Wang