Description of problem: The instance delete command fails due to a HAProxy limit reached on the placement URL query from Nova API. Nova API makes a placement GET request of length 22k characters. When this requests goes thru HAProxy we get immediately 400. ~~~ def _get_sharing_providers(self, context, agg_uuids): """Queries the placement API for a list of the resource providers associated with any of the specified aggregates and possessing the MISC_SHARES_VIA_AGGREGATE trait. :param context: The security context :param agg_uuids: Iterable of string UUIDs of aggregates to filter on. :return: A list of dicts of resource provider information, which may be empty if no provider exists with the specified UUID. :raise: ResourceProviderRetrievalFailed on error. """ if not agg_uuids: return [] aggs = ','.join(agg_uuids) url = "/resource_providers?member_of=in:%s&required=%s" % ( aggs, os_traits.MISC_SHARES_VIA_AGGREGATE) resp = self.get(url, version='1.18', global_request_id=context.global_id) if resp.status_code == 200: return resp.json()['resource_providers'] msg = _("[%(placement_req_id)s] Failed to retrieve sharing resource " "providers associated with the following aggregates from " "placement API: %(aggs)s. Got %(status_code)d: %(err_text)s.") args = { 'aggs': aggs, 'status_code': resp.status_code, 'err_text': resp.text, 'placement_req_id': get_placement_request_id(resp), } LOG.error(msg, args) raise exception.ResourceProviderRetrievalFailed(message=msg % args) ~~~ With large number of aggregate UUIDs the default length of 16k characters is exceeded and HAProxy ends up rejecting the request. ~~~ curl -s -H "accept: application/json" -H "User-Agent: nova-scheduler keystoneauth1/3.4.0 python-requests/2.14.2 CPython/2.7.5" -H "OpenStack-API-Version: placement 1.17" -H "X-Auth-Token: $OC_TOKEN" "$PLACEMENT_ENDPOINT/resource_providers?member_of=in:TH1RXv8cd1ulc2Obtzu82mYfIf74vDXHGVwV9vCX9YyzlKwxbuPN7CNLxOI8A2KybBKutkq7AYrtay0BO7tba5XtECf7XjIKqNytqUFGHqWIha1tBga5Vdd2pNCVGq9Zfw1YJ1TM3hTaATsU9Jv3JyMiD2CDlcR7hOgroGaWwYIEezZwiL5uWRmcE873hS5VWBz2rTkaLhsbHcRvLIgBOwJ8sAuQP5jiasik2qtv44kAkNHaAAn5cbZOPeEdjLrQL9k3xWUU2nlfPdXiuiN2flnAPNv38HaGuFkEx3DHolphMRstcLUUQ9LC9hiRnexAmLWuE6uXUfPrfMqPDubPgxyne4LYEZyBjUXAS8EylZIusNVOmz0o4GGbFLNh4f23Qiu373kjsqxfucYQZfo4gMGIDXSmrx36p9hNblN3hu7Ce1Gmkh59wW2ibbefX2ga708XMSdDCVeB9E9xU3VYPdraoC54dSLV7ZgVwE4vWfuTo69w3RwwZyAU4I3J2XkKaJJhh2DqsIEq9luFxg1A5bgYufpX2gcAuLcl8woWvQGTssGtlGG40ajsddcHZy8LNLW0lnJJIKfINPStZusneVU54oD7mv2A3OKw8MYvDk32yB59Iv5gSaQ3NrD41yvTukhaIwymWaeknQitrDDHe1BhwQOUAjiaCkoRLXqAOnd8byMSdxnhTNFEIgDgQRSLqGAt97hQOuQTxaySN9sRbOqNnWmmpsYjJiM5DWB9VhDcJcQN44Bb6DabWuS6wEVPKMvV69bvBgYPtbLen9sGf2cTDc1ZGptiM3ydrpAOwGCXj7z3aaOOEyiTN59F07Ymy38kyWhlXe0AmBPshaaA0hY92bnxgcxdEonxRfS7D94URnMf9lXAaubheZPlQVeN7RGE8fh5zswvbip5sQwAirfDBloiwqRnhdjBq5Uria6oEJmXqZVRnYIMAjXp0MIz0eRgsnvNljHtQDW6IIeZZXPS0Xn6lmQDkOCSQdRYqNPClJsTvlpNb2dFWhbPl3ViPpSVosfNwkEAKVugjaDvt8BlPMbMoYBoL81IWBPeayN2lWy6JC1of1UziVn9NwmFiOdGo4tbu3gHOzKWlxoyKNDJ4vHqkFDE9HePGUjYqeXZVRO6aFvCXq7EQKewAnQd4LeoUSda6xuJK9rzegQDfqogMOmPiE68EYWIzTBiP2N2FVVl4lhrh9XUovrPBZYGIjcNj0izCg0ZYag5Lopj2SgidY9q5b8CQ4QMf96K85tQ0FiL1kxun7b3AkqNIybSnuDOqFNbt406VeG9RPDncL0JwlZ8K7PljaLF4tOCHkgVysGbdf5iS5A1DCdjoWS8B406FjZAj6JGDDjJ4IGVnWUalychwVdFa8Bfh1vkYmzUn234MthMy9fsyzAdFbc7PJ5Wwfzzp0ojd7YZiDFXmvNjgmzPfIngBTc9ZkLnVZ1DYVFwX7RBq44spwms7jX5nkN4y6c153hpnHSoVBx0oJLrxd1Dy3EXUw0EGOOxAoifmHN3avivpP9C6GDq1SEJAdTrTiURehRQ0W9HC1o6T1dDquDd5YtlAVWtJhCVH9sS26yKGmGoD3RUhDVAFtSptUIk93bMUQ3lSEmQjl5fs7Z73PE5cls3AOr9JLGOWJL7whp44vnZmQBpM3HQAHISI293SvLL0PC1hp9PGlWgkNTzkMq4CAV2NMfYnNtcNJK32wox98OUDMuyq1YjjFDL9D6PvmNTBq9RppiB9HGflQi7FXgIvNdJBhzCuhj0j4xeLCJm4kCLgfeHAJfbpIsCrGCKkXF2CaWxJoXPDfaCNH9QIpZIp2Ba2fbmAk0eQoQGYLfxn3tYkY5Q933MJz98RwkBbrNyKEEvPlx5RbC5h7hZOPU9hIPsq0NK9CfAWiM9hNz7rX4sSxH2WwhhlSSlsciJ91rZWqiBQ6tiuls7D9hk3X8enKVBBQ0cwOHYPMFPe4SiVQQyURsYd2eGF1XAdeKdXa72IPbdZVrXhV3z7CGPPASbVXOoJsF3LdctYX6OEZmlvfmLl5RCuF5kjAe6KFP5SIMQyFg1k4Tv2GgWspyUhDb1fUxhQSAWhWX3JVRJKKfCDcYdmvPxRROvsBV3b8dthOGWxS2Ba3jd96O13X8rSUYQxt4HW01KCTp6Ro75WtETiqPgAjzkYs3iPpLQXagGdb9LMBjhC48G1vRoDiZA7XrGDhilh2wQZLB0mdvlSS0IjLas3GM3S9Th2G6oPyi0ClIZOXc9OraJ3NNna8tquC0QYUr5k4hPRkPiMx8j76yXm4cDYKG67cTUaG5zuTkIFDC6jXm8rUZyYygvuYF4rYfj4J8CUou9cYzPgoo9zAhuByUKIX77bgAvfvNRQejIitDz1NsnGyjATf1PptEjlmjTpP33euEOiuiSPBE3sZ9WTbNS9RwyTXmqregmq9J7KoJ4KXrEwnuyy0mL35O5FcfqXht21n5HBc6XkZN4sWgVoDZWG6UOWLBjHhn8Lo7RSzChqrbLGVKE8513pg2ihWHXIyAcsD23o0ai1JRsHZjJuAp1bI4zKat0n4Nwjz4vdlHF0NeLSV1NLdEZFt7ngNNzGIK1pMPBxreon08OmWrRVyCecNNcTK3RTARLvwqmnFmJVGpNEzor1sZP2a5JLgtkYExZNOmtYkSHw0LCX14LwzjN87RS4GgeIu3t9P7HhXSt35ZsLxSI2zeb7of4WHcD5hgtuzK1yjnvw28oYkL6giTVvmfyUlli6x05MQGtobGi1jPZxzHgTgaFxPEaiPrhGFXVgWbAOxMYmWWT3Zj0jrDGcNqPiHH2yhR6SMnboq7alm4hvCL33OAUTrM6vYsbf7kekRarr5LD9D7Znfv6Exvq3RX4dfDXVOIe2yFszvEskfNb8KUmRp6rpujFTxl5RzLz82PNzJDVILBpQraCCT7IlO6CQkE8JtRdMO6cXc3ORJEMkc9EgEyKReuFJp4IVgt0nlbmvqJOG7M27ZXRUW7aaPIc4EQnXFxKqK7rzK3SyT00FuOuxkGthB5kONWCAYxQ4g7dzveaOAtWsU3gsGP9H0hTvgtm7oy7BoEJsdlxxZReAt5vD4hXo8pxQKqY2gh6u032kdyHbXaEVtYChj0sENDv0qJvghIrx86Qk5NyMItRu0l50rpnc9FPdyCOXaD0QutVWJhI73sjx4eXUgfi566gIL5EWdKjUY31fzy6a4fZe9LG6WeOPndB1cvWoDsWJFPyiEbWAkaDzDNpEgFbIAkQPdj3OdSm6Rk0al6TPYw7JlbuvBEKtQ1Qcx0rammbfR7Fa5zYxA9L3ybh7bjuuWI66awTc1nT9ZMzCilMKrUfMJ55DJWw42H4iF52Bylsc0CeODcoMv5TDVsh2uUVSnTPgkTldcg8OAc7LiA9g7DxA67Ig06m3cv4XRrXa1fIxyNrVFxBPyhJSf4d6iDgHtjvfUUn6oQFRQ5HpUsAplzJ7sTRq6AI7JwOBo3TSy5EVlqehzQvhAhwd8w0KCZ6p3yZLRp76Bqqoo11d9tmNOtrTT6BpuQHkjZUkY62Sqa4w1FzGZH2M7KX8xof9Vxate4OgHLXByPUeditDRd3M2NF8qKAytsFcPHzX6Axvv5QyA7DRu7uV4C2dQwwUQpLYmqenkdjYARASy2qa611J2GiOFGKqourR9Gl8mW6SNIcc1gvsJDcNGq18UEIJAjHJ9RgT079BopikIYgXWYP7ct0h38LAWDjpNsSKLYB8uqI8Lyppw5wrelh7hICiZYaxkUENoLvxUNUWPsxZscivYnwANsilCrQR5inwdZXDFOOFpLEG9KuQmCFx4E844HGlvdY0iAY5I2tm1gzkUBCJjzGYIEPVVUzt6RaQCKM5fItJNGOOQKMYdlDNayYTMSkKy3MnVSjdQHZeJ1HBFZpTx7pm7uI448gOkXOJdVCgaMzFEOlEL5x4Lwo9g43O59PjkQZBnOjLTjJFqxpviWAYE0uYdl9XoHQGDGoYUSroSXV7Mo2je8K5f5M7sUzAyZ4FfeH3rAi9FlY6iqgCCkoMR7Jwmm7ibTwHgX7PTUVZ8DcCz3FoH1pu2ri6C8VwGWmXmiHdaqvtC7j04nfFqrbptsXzIVOmEvPFCbeapae34v6XAnTTJbohdCwNEHcymbddhMhsS1FrH6Ytngx6xX4K9RPnXE7idaORhOhrpwwGtJbJRRBBKqZhaVHuesxNKdxTucq8MWRUJfvQMGTLuuIhBQFUeVk6dV7AWGSvUYrBqZlEBTLWLgH3AFxXGRp610WjnBnymp1NUpgp3NhFfjLTfSDkEV2aKvSqpAEhEtDN4xGcym1mDqOwWdxh3OUU0OPjFoKtC9JCRf10JscRLd8wklE4adlcLuo5tj78L4VX2LPMFLjseSmQREH6G5WwiMCsC2orPJYylrbqgIdRljNlXGq21KLgCEWobmDxBuXaCTyfxcRXGLBM3Fx5Jg4LSctm2OESXeIHqlVVAyP2SwesY74WDANfVBdwUfx6bDOMVMfumYwSOrmEokqpiB4Kg3FithgCN1iahNrgKYpR6yuYmubOIKo4diyxXc8QNOAgy7ARHu0aQBeb7Ycck7MG57YWt36OFA2y1ygz0qS2ynhXBnPCqlI3UmGu0UZpTWc7FFCFfYdlKjbGnDDr9e45CUzWyRDzQs7bkG26bIERZ0sXMkqF4F1VHsZMXtemShBSegWNIZbfiypLWasQvl0Z6Qs7iDCXSd2MIynes1FRLKNYOzg14dGwDmiI8NIopHz23U7D7Yu194SdIIWWPlzwCHMkZMZUAWlNDXMDxtL732fx0eieEWPaBom1YYccFWQk8s2MEQWiQy0S9bccHg2VwuXiysDjcoJXROu4PQ8gqRpA30oYJPcykXScoFafko8G6srFl4TfkcdQjQ1RaasfL5c2xGbgNx49n8JVaBsk2dhXtU05UDhbcJYCv4pc0VKhQwdRO6wsWzcyYusuTPRkQUkYAiGrFHUycPDNScRAQcISm3FLKy9SX4M3ByqQWVVx59zl17FK4fjjdSCeqEK7rMja8cGZ1NENNiL9IeSUkHp4Df00aNxvYdWWxSWCwUY01lNDxwvpP9ZIGZ038yafQR5WECVzmSZJA34lpzhPFGTyUigsZmrUdxBNyxKE7CRnBTc1NgbNQ7jLM0ZyrmS9rsJHJeLiE5Kw06ZW4G7LmI8kl2N6YZ2v4WgPd4lHoAT4QNsrFWbOJjCFnOmEP7Rcplg7gpFPo2kjSFGEj0OttzFnQTtQnopheu0r1AKg0vQYugA0bazVzD3eJEkbYDkL8rqdPnbE5Y1b67jHm8B4PquXiHFNhGs7kbCjTQZZ1VHTbnDW3kCby0tCrZptjLIbDhImN2NVRYbdz2yWOEvZINQJ1ZUA80YvfsOmGlGiTn2sVsWVkEHqQYVlFwYyzug83suuDb7Dg4SZO8vezszOZBFNwLfTSIMRBLEz7TekNEHxipusBWPSEOrVgtS7mCr2ZEyyLVCMOxmQRp16U5G7htjS2ZJZoSUGFHs7CxQMkvl5LPycgsTJYWQpw5Z9lLQJvI4iLkaduboFJnbw0dC82RT0JPAJzIxGV6CJXlLzZL30Q2HMyhQSQ1kdh0jGZPQLGtRgpDXnDHvdsZSi0m1fTPSzpMvOTYr6SMoR8BcK.......w78FSFIL7i3G8ghz14surP7OOCLfiMGJORn8aPtR2wCoiK1IYSfRd4jS8VuDawSj6nycmWovJ5QmdKexzj" <html><body><h1>400 Bad request</h1> Your browser sent an invalid request. </body></html> ~~~ Version-Release number of selected component (if applicable): OpenStack 16.2.4 How reproducible: Follow https://access.redhat.com/solutions/3537351 and perform a query with large number of characters. (reproducing the large number of aggregate uuids) Actual results: HAProxy rejects the long query with: ~~~ <html><body><h1>400 Bad request</h1> Your browser sent an invalid request. </body></html> ~~~ Expected results: The query should pass thru the HAProxy: ~~~ curl -s -H "accept: application/json" -H "User-Agent: nova-scheduler keystoneauth1/3.4.0 python-requests/2.14.2 CPython/2.7.5" -H "OpenStack-API-Version: placement 1.17" -H "X-Auth-Token: $OC_TOKEN" "$PLACEMENT_ENDPOINT/resource_providers?member_of=in:1e48a974..." {"errors": [{"status": 400, "title": "Bad Request", "detail": "The server could not comply with the request since it is either malformed or otherwise incorrect.\n\n Invalid query string parameters: Expected 'member_of' parameter to contain valid UUID(s).... ~~~ Additional info: Workaround using: ~~~ tune.bufsize 65536 ~~~ Inside the the global section of /var/lib/config-data/puppet-generated/haproxy/etc/haproxy/haproxy.cfg Then restart the HAProxy bundle ~~~ pcs resource restart haproxy-bundle ~~~
At this point in time we've run out of runway for backporting this to 17.1, which is now limited to critical/urgent customer issues, and CVEs. Closing as UPSTREAM since it was done in a subsequent upstream release.