Bug 1832177

Description:
cloud-init-18.5-6.el7_8.4.x86_64.rpm provisioning failed in Azure.

Version-Release number of selected components (if applicable):

Version:
RHEL-7.9
cloud-init-18.5-6.el7_8.4.x86_64.rpm

How reproducible:
100%

Steps to Reproduce:
1. Create a RHEL-7.9 in Azure with cloud-init-18.5-6.el7_8.4.x86_64.rpm enabled (without WALA).

Actual results:
Provisioning failed.

Expected results:
Provisioning successfully

Additional info:
- Seems the root cause is in cloudinit/url_helper.py. I used the url_helper.py in cloud-init-18.5-6.el7_8.3 to replace it in cloud-init-18.5-6.el7_8.4, and worked well.

2020-05-06 08:19:32,860 - url_helper.py[DEBUG]: [0/11] open 'http://168.63.129.16/machine/?comp=goalstate' with {'url': 'http://168.63.129.16/machine/?comp=goalstate', 'headers': {'User-Agent': 'Cloud-Init/18.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2020-05-06 08:19:32,898 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2020-05-06 08:19:33,899 - url_helper.py[DEBUG]: [1/11] open 'http://168.63.129.16/machine/?comp=goalstate' with {'url': 'http://168.63.129.16/machine/?comp=goalstate', 'headers': {'User-Agent': 'Cloud-Init/18.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2020-05-06 08:19:33,903 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2020-05-06 08:19:34,904 - url_helper.py[DEBUG]: [2/11] open 'http://168.63.129.16/machine/?comp=goalstate' with {'url': 'http://168.63.129.16/machine/?comp=goalstate', 'headers': {'User-Agent': 'Cloud-Init/18.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2020-05-06 08:19:34,908 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2020-05-06 08:19:35,909 - url_helper.py[DEBUG]: [3/11] open 'http://168.63.129.16/machine/?comp=goalstate' with {'url': 'http://168.63.129.16/machine/?comp=goalstate', 'headers': {'User-Agent': 'Cloud-Init/18.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2020-05-06 08:19:35,912 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
......
2020-05-06 08:22:18,534 - url_helper.py[DEBUG]: [10/11] open 'http://168.63.129.16/machine/?comp=goalstate' with {'url': 'http://168.63.129.16/machine/?comp=goalstate', 'headers': {'User-Agent': 'Cloud-Init/18.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2020-05-06 08:22:18,538 - handlers.py[DEBUG]: finish: azure-ds/register_with_azure_and_fetch_data: FAIL: register_with_azure_and_fetch_data
2020-05-06 08:22:18,538 - util.py[DEBUG]: Recursively deleting /run/cloud-init/tmp/tmpqoe6cr
2020-05-06 08:22:18,538 - handlers.py[DEBUG]: finish: azure-ds/get_metadata_from_fabric: FAIL: get_metadata_from_fabric
2020-05-06 08:22:18,538 - DataSourceAzure.py[WARNING]: Error communicating with Azure fabric; You may experience.connectivity issues.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceAzure.py", line 678, in _negotiate
    fabric_data = metadata_func()
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/helpers/azure.py", line 40, in impl
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/helpers/azure.py", line 493, in get_metadata_from_fabric
    return shim.register_with_azure_and_fetch_data(pubkey_info=pubkey_info)
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/helpers/azure.py", line 40, in impl
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/helpers/azure.py", line 428, in register_with_azure_and_fetch_data
    'http://{0}/machine/?comp=goalstate'.format(self.endpoint))
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/helpers/azure.py", line 106, in get
    return url_helper.read_file_or_url(url, headers=headers)
  File "/usr/lib/python2.7/site-packages/cloudinit/url_helper.py", line 107, in read_file_or_url
    exception_cb=exception_cb)
  File "/usr/lib/python2.7/site-packages/cloudinit/url_helper.py", line 307, in readurl
    raise excps[-1]
UrlError: 400 Client Error: Bad Request