Hide Forgot
Description of problem: When running openstack overcloud deploy -n /home/stack/virt/network/network_data_1.yaml and the file network_data_1.yaml doesn't exist, get a big error: Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 925, in take_action self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 374, in _deploy_tripleo_heat_templates_tmpdir new_tht_root, tht_root) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 400, in _deploy_tripleo_heat_templates validate_stack=False) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 206, in update_plan_from_templates passwords = _load_passwords(swift_client, name) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 293, in _load_passwords return plan_env['passwords'] KeyError: 'passwords' because the command is trying to build a plan and fails, without validating first that the file exists. Version-Release number of selected component (if applicable): RHOS_TRUNK-15.0-RHEL-8-20190619.n.1 How reproducible: 100% Steps to Reproduce: 1. deploy undercloud 2. Try to deploy overcloud with a non-existing network file Actual results: Deploy fails because of empty plan Expected results: Error explaining that the file doesn't exist Additional info:
Heya, so basically, we need a pre-flight check that will loop over environment files passed as argument and ensure they are all existing. It shouldn't be that hard to add, I'll have a look. this week. Cheers, C.
So. It didn't happen on the latest puddle (RHOS_TRUNK-15.0-RHEL-8-20190621.n.1) at the first time, got an error that "The following files were not found: /home/stack/virt/network/network_data_1.yaml" And then when I've tried once more, I got exactly the "passwords" error again. Scenario: 1. Try to deploy with a non-existing network file: (undercloud) [stack@undercloud-0 ~]$ ./overcloud_deploy.sh Creating Swift container to store the plan Creating plan from template files in: /tmp/tripleoclient-nwyg8t6l/tripleo-heat-templates Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 925, in take_action self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 374, in _deploy_tripleo_heat_templates_tmpdir new_tht_root, tht_root) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 407, in _deploy_tripleo_heat_templates validate_stack=False) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 169, in create_plan_from_templates plan_env_file, networks_file) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 55, in _upload_templates constants.OVERCLOUD_NETWORKS_FILE, networks_file) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 280, in _upload_file with open(local_filename, 'rb') as file_content: FileNotFoundError: [Errno 2] No such file or directory: '/home/stack/virt/network_data_1.yaml' [Errno 2] No such file or directory: '/home/stack/virt/network_data_1.yaml' 2. Without fixing the file, try to run the command again: (undercloud) [stack@undercloud-0 ~]$ ./overcloud_deploy.sh Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 925, in take_action self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 374, in _deploy_tripleo_heat_templates_tmpdir new_tht_root, tht_root) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 400, in _deploy_tripleo_heat_templates validate_stack=False) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 206, in update_plan_from_templates passwords = _load_passwords(swift_client, name) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 293, in _load_passwords return plan_env['passwords'] KeyError: 'passwords' 'passwords'
Can't reproduce it: [stack@undercloud ~]$ ./deploy-overcloud.sh Error: The following files were not found: /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry-11.yaml [stack@undercloud ~]$ ./deploy-overcloud.sh Output env file exists, moving it to backup. Error: The following files were not found: /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry-11.yaml [stack@undercloud ~]$ ./deploy-overcloud.sh Output env file exists, moving it to backup. Error: The following files were not found: /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry-11.yaml Script: openstack overcloud deploy \ --templates /usr/share/openstack-tripleo-heat-templates/ \ -e /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry-11.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/enable-swap.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ --environment-directory ~/overcloud-yml \ --ntp-server pool.ntp.org --libvirt-type qemu 2>&1 | tee -a ~/install-overcloud.log This is with latest-RHOS_TRUNK-15-RHEL-8 providing python3-tripleo-common-10.7.1-0.20190625154433.5751fc7.el8ost.noarch python3-tripleoclient-11.4.1-0.20190611084233.f36d899.el8ost.noarch openstack-tripleo-common-10.7.1-0.20190625154433.5751fc7.el8ost.noarch The type of the file doesn't count imho (being a network env file, a telemetry env file or whatever). Care to recheck? Cheers, C.
Quick update: network env is passed as "-n", not "-e". In this case, indeed, the plan is created and uploaded without any check on the existence of the passed file for network. Complete output in this case (first run): Output env file exists, moving it to backup. Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 925, in take_action self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 374, in _deploy_tripleo_heat_templates_tmpdir new_tht_root, tht_root) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 400, in _deploy_tripleo_heat_templates validate_stack=False) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 232, in update_plan_from_templates plan_env_file, networks_file) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 55, in _upload_templates constants.OVERCLOUD_NETWORKS_FILE, networks_file) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 280, in _upload_file with open(local_filename, 'rb') as file_content: FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry-11.yaml' [Errno 2] No such file or directory: '/usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry-11.yaml' Removing the current plan files Uploading new plan files And there, since the plan files are uploaded to swift, we hit the issue with the next run, with or without plan correction: Output env file exists, moving it to backup. Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 925, in take_action self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 374, in _deploy_tripleo_heat_templates_tmpdir new_tht_root, tht_root) File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 400, in _deploy_tripleo_heat_templates validate_stack=False) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 206, in update_plan_from_templates passwords = _load_passwords(swift_client, name) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 293, in _load_passwords return plan_env['passwords'] KeyError: 'passwords' 'passwords' Now I know what to look for :).
Correction pushed upstream.
Upstream backport to Stein added.
patch merged upstream stable/stein - it still needs to go down to queens upstream. Upstream Rocky backport added
Pushing right component..
Checked with non-existing files of type: --plan-environment-file --answers-file --networks-file Tried several times, then deployed a correct configuration. Deployment passed. Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:2811