Description of problem: ----------------------- Auto scaling with 'trust' notifier doesn't work. Aodh sends requests but got 403 error. At the mean time regular alarming is working. Version-Release number of selected component (if applicable): ------------------------------------------------------------- openstack-aodh-notifier-2.0.3-2.el7ost.noarch python-aodh-2.0.3-2.el7ost.noarch openstack-aodh-evaluator-2.0.3-2.el7ost.noarch openstack-aodh-common-2.0.3-2.el7ost.noarch openstack-aodh-listener-2.0.3-2.el7ost.noarch python-aodhclient-0.5.0-1.el7ost.noarch openstack-aodh-api-2.0.3-2.el7ost.noarch openstack-heat-api-cloudwatch-6.0.0-8.el7ost.noarch openstack-heat-api-cfn-6.0.0-8.el7ost.noarch python-heatclient-1.2.0-1.el7ost.noarch openstack-heat-api-6.0.0-8.el7ost.noarch openstack-heat-common-6.0.0-8.el7ost.noarch openstack-heat-engine-6.0.0-8.el7ost.noarch Steps to Reproduce: 1. Create stack with templates attached. 2. Trigger alarms Additional info: ---------------- Virtual setup: 3controllers + 1compute + 1ceph Excerpts from logs/alarms/trusts attached
Can you attach the heat-engine log? The heat-api log doesn't offer any clues beyond what's already in the description.
Hmm, OK, there's no record of the request in the heat-engine log, so it must be failing in heat-api but the logs for that don't offer any more clues other than that it returned a 403 Forbidden error. The keystone logs might conceivably help.
Nothing out of the ordinary in the keystone logs either - the responses are not logged. We're going to need help from somebody who knows how all of this is supposed to work.
I found the issue, it relies in Aodh: you have to use the v3 API to request the trust token, otherwise I believe it's ignored. Before the refactoring of keystoneauth, we used to force v3 by doing replace('v2.0', 'v3') on the URL, which is ugly but works. We have to restore such a hack I belive if we want to continue supporting v2.0 in OSP 9. It's also possible that we can reconfigure aodh with Keystone v3, but I don't know how, especially if we have to support upgrades.
Could it be that this upstream change will configure Aodh with v3? https://review.openstack.org/#/c/365117
I'm going to mark this as a duplicate of bug 1364052. Can you please test when that fix is available? If it doesn't fix this issue then this bug can be de-duped *** This bug has been marked as a duplicate of bug 1364052 ***