Description of problem: When the following exception is encountered in /usr/lib/python2.7/site-packages/ironic/drivers/modules/image_cache.py, no path/file information is logged making resolving these issues difficult for our customers. 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager [req-15629af3-65e7-4c2a-9239-6d7a1d3f0b73 1ab6656d87f848d3afac1800c2a163b1 c3527b040b9c4145a14b408397393701 - default default] Unexpected error while deploying node 12be0941-e06f-4e54-b9ef-e60f4abeb793: OSError: [Errno 18] Invalid cross-device link 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager Traceback (most recent call last): 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/conductor/manager.py", line 3283, in do_node_deploy 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager new_state = task.driver.deploy.deploy(task) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic_lib/metrics.py", line 60, in wrapped 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager result = f(*args, **kwargs) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/conductor/task_manager.py", line 148, in wrapper 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager return f(*args, **kwargs) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/iscsi_deploy.py", line 462, in deploy 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager cache_instance_image(task.context, node) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic_lib/metrics.py", line 60, in wrapped 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager result = f(*args, **kwargs) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/iscsi_deploy.py", line 134, in cache_instance_image 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager CONF.force_raw_images) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py", line 504, in fetch_images 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager cache.fetch_image(href, path, ctx=ctx, force_raw=force_raw) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/image_cache.py", line 134, in fetch_image 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager href, master_path, dest_path, ctx=ctx, force_raw=force_raw) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/image_cache.py", line 162, in _download_image 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager os.link(master_path, dest_path) 2018-10-25 16:48:28.178 1507 ERROR ironic.conductor.manager OSError: [Errno 18] Invalid cross-device link Version-Release number of selected component (if applicable): rpm -qa | grep ironic python2-ironic-neutron-agent-1.0.0-1.el7ost.noarch openstack-ironic-conductor-10.1.2-4.el7ost.noarch python2-ironicclient-2.2.0-1.el7ost.noarch puppet-ironic-12.4.0-0.20180329034302.8285d85.el7ost.noarch openstack-ironic-api-10.1.2-4.el7ost.noarch python-ironic-inspector-client-3.1.1-1.el7ost.noarch openstack-ironic-common-10.1.2-4.el7ost.noarch openstack-ironic-inspector-7.2.1-0.20180409163360.el7ost.noarch python-ironic-lib-2.12.1-1.el7ost.noarch openstack-ironic-staging-drivers-0.9.0-4.el7ost.noarch How reproducible: Trigger the above exception. Expected results: Output enough related path information to resolve the underlying cause of the exception. Additional info: Judging from the existing code, I suspect this can be accomplished via the addition of "except:" logic for the exception in question to ensure the path information is logged. try: _fetch(ctx, href, tmp_path, force_raw) # NOTE(dtantsur): no need for global lock here - master_path # will have link count >1 at any moment, so won't be cleaned up os.link(tmp_path, master_path) os.link(master_path, dest_path) finally: utils.rmtree_without_raise(tmp_dir)
Can you provide the actual steps executed or more detail so we can properly verify?
Install latest OSP13 Env: openstack-ironic-conductor-10.1.6-4.el7ost.noarch openstack-ironic-common-10.1.6-4.el7ost.noarch openstack-ironic-staging-drivers-0.9.1-1.el7ost.noarch openstack-ironic-api-10.1.6-4.el7ost.noarch openstack-ironic-inspector-7.2.1-5.el7ost.noarch Verified code in place. If any more issue , please raise a new bz.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0092