Hi Keigo,

I took care of the first review (regarding the scenario tests) and it's being under review at the moment. In order to fix the other tests I introduced a new patch (see the external trackers).

Regarding the tests you mentioned, I didn't find the following one, is it a custom test?
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_create_ebs_image_and_check_boot

Anyway, I tested the two upstream reviews by running the following tests:
tempest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_volume_backed_live_migration
tempest.api.compute.admin.test_live_migration.LiveMigrationRemoteConsolesV26Test.test_volume_backed_live_migration
tempest.api.compute.admin.test_live_migration.LiveMigrationTest.test_volume_backed_live_migration
tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume
tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment
tempest.api.compute.servers.test_device_tagging.TaggedBootDevicesTest.test_tagged_boot_devices
tempest.api.compute.servers.test_device_tagging.TaggedBootDevicesTest_v242.test_tagged_boot_devices
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_volume_backed_server_confirm
tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern

They passed except two tests which failed due to a non-related problem.

tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_boot_server_from_encrypted_volume_luks [38.328830s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/tempest/common/utils/__init__.py", line 89, in wrapper
        return f(*func_args, **func_kwargs)
      File "/usr/lib/python2.7/site-packages/tempest/scenario/test_volume_boot_pattern.py", line 266, in test_boot_server_from_encrypted_volume_luks
        delete_on_termination=False)
      File "/usr/lib/python2.7/site-packages/tempest/scenario/test_volume_boot_pattern.py", line 64, in _boot_instance_from_resource
        return self.create_server(image_id='', **create_kwargs)
      File "/usr/lib/python2.7/site-packages/tempest/scenario/manager.py", line 217, in create_server
        image_id=image_id, **kwargs)
      File "/usr/lib/python2.7/site-packages/tempest/common/compute.py", line 261, in create_test_server
        server['id'])
      File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
        self.force_reraise()
      File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
        six.reraise(self.type_, self.value, self.tb)
      File "/usr/lib/python2.7/site-packages/tempest/common/compute.py", line 232, in create_test_server
        clients.servers_client, server['id'], wait_until)
      File "/usr/lib/python2.7/site-packages/tempest/common/waiters.py", line 76, in wait_for_server_status
        server_id=server_id)
    tempest.exceptions.BuildErrorException: Server 2c3d8de2-af31-4971-898b-cf8baf3e42df failed to build and is in ERROR status
    Details: {u'message': u'Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 2c3d8de2-af31-4971-898b-cf8baf3e42df.', u'code': 500, u'created': u'2019-06-05T09:43:28Z'}

tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_volume_backed_server_confirm [42.458625s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/tempest/lib/decorators.py", line 112, in wrapper
        raise exc
    tempest.lib.exceptions.BadRequest: Bad request
    Details: {u'message': u'No valid host was found. No valid host found for resize', u'code': 400}


I checked the code of those two tests manually and they call the methods at the end which were already modified for setting availability zone, so they should be good.

If you have a moment, please, have a look.

Hi Martin,

Thank you for your work on this bugzilla!
In A RHOSP13 tempest in Customer Portal, tempest/scenario/test_volume_boot_pattern.py has 
~~~
    194     @decorators.idempotent_id('36c34c67-7b54-4b59-b188-02a2f458a63b')
    195     @utils.services('compute', 'volume', 'image')
    196     def test_create_ebs_image_and_check_boot(self):
    197         # create an instance from volume
    198         volume_origin = self._create_volume_from_image()
    199         instance = self._boot_instance_from_resource(
    200             source_id=volume_origin['id'],
    201             source_type='volume',
    202             delete_on_termination=True)
    203         # create EBS image
    204         image = self.create_server_snapshot(instance)
    205 
    206         # delete instance
    207         self._delete_server(instance)
    208 
    209         # boot instance from EBS image
    210         instance = self.create_server(image_id=image['id'])
    211 
    212         # Verify the server was created from the image
    213         created_volume = instance['os-extended-volumes:volumes_attached']
    214         self.assertNotEmpty(created_volume, "No volume attachment found.")
    215         created_volume_info = self.volumes_client.show_volume(
    216             created_volume[0]['id'])['volume']
    217         self.assertEqual(instance['id'],
    218                          created_volume_info['attachments'][0]['server_id'])
    219         self.assertEqual(created_volume[0]['id'],
    220                          created_volume_info['attachments'][0]['volume_id'])
    221         self.assertEqual(
    222             volume_origin['volume_image_metadata']['image_id'],
    223             created_volume_info['volume_image_metadata']['image_id'])
    224 
    225         # delete instance
    226         self._delete_server(instance)
~~~

The method is renamed to test_image_defined_boot_from_volume().
~~~
commit 2db6c275152f4ea0386f411defdd46154a4505a4
Author: Matt Riedemann <mriedem.os>
Date:   Thu Mar 22 18:57:19 2018 -0400

    Cleanup and rename test_create_ebs_image_and_check_boot
    
    I'm trying to debug a failure in a nova change
    in test_create_ebs_image_and_check_boot but
    need more debug information, specifically the
    volume details when it fails. While doing this,
    I'm going to cleanup the comments and variable
    names so this test can actually be understanble.
    
    Also, this renames the test itself since the 'ebs'
    part (AWS Elastic Block Storage) has long been a
    source of confusion.
    
    Change-Id: I4d60adc95d72be609927e36a28fd7c86b5ee86bc
    Related-Bug: #1747693
~~~

Would you be able to add test_image_defined_boot_from_volume into the test case for this bugzilla?

Best Regards,
Keigo Noha

oh, yes, it got renamed. I tested it with a newer tempest so I missed that.

Anyway, I ran also tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_image_defined_boot_from_volume and it's working. It uses methods from tempest/scenario/manager.py which are fixed by the first patch.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2623