Bug 1789075
| Summary: | "No lock information provided for value None" error on "openstack overcloud container image upload" | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Roman Safronov <rsafrono> |
| Component: | python-tripleoclient | Assignee: | Alex Schultz <aschultz> |
| Status: | CLOSED ERRATA | QA Contact: | Sasha Smolyak <ssmolyak> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 15.0 (Stein) | CC: | aschultz, hbrock, jschluet, jslagle, mburns |
| Target Milestone: | --- | Keywords: | Triaged, ZStream |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | python-tripleoclient-11.5.3-0.20200114200459.d09212f.el8ost | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-03-05 12:01:54 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
According to Alex Schultz the fix would be https://review.opendev.org/#/c/701568/ 'openstack tripleo container image prepare' should be used instead of 'openstack overcloud container image' commands starting with OSP14. That being said, these should be fixed so they don't error in this way. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0643 |
Description of problem: "openstack overcloud container image upload" failed with "No lock information provided for value None" error Found during ML2/OVS->OVN migration testing on attempt to upload OVN images to local registry: fatal: [undercloud-0]: FAILED! => { "changed": false, "cmd": "source ~/stackrc\n openstack overcloud container image upload --verbose --config-file ~/ovn_container_images.yaml", "delta": "0:00:06.850556", "end": "2020-01-07 19:54:50.466920", "invocation": { "module_args": { "_raw_params": "source ~/stackrc\n openstack overcloud container image upload --verbose --config-file ~/ovn_container_images.yaml", "_uses_shell": true, "argv": null, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "warn": true } }, "msg": "non-zero return code", "rc": 1, "start": "2020-01-07 19:54:43.616364", "stderr": "START with options: overcloud container image upload --verbose --config-file /home/stack/ovn_container_images.yaml\ncommand: overcloud container image upload -> tripleoclient.v1.container_image.UploadImage (auth=False)\nUsing config files: ['/home/stack/ovn_container_images.yaml']\n[rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-ovn-northd:20191223.2] Starting upload image process\n[rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-neutron-metadata-agent-ovn:20191223.2] Starting upload image process\n[rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-ovn-controller:20191223.2] Starting upload image process\n[rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-neutron-server-ovn:20191223.2] Starting upload image process\nNo lock information provided for value None\n[192.168.24.1:8787/rh-osbs/rhosp15-openstack-ovn-northd:20191223.2] Failed uploading the target image\nNo lock information provided for value None\n[192.168.24.1:8787/rh-osbs/rhosp15-openstack-neutron-metadata-agent-ovn:20191223.2] Failed uploading the target image\nNo lock information provided for value None\n[192.168.24.1:8787/rh-osbs/rhosp15-openstack-ovn-controller:20191223.2] Failed uploading the target image\nNo lock information provided for value None\n[192.168.24.1:8787/rh-osbs/rhosp15-openstack-neutron-server-ovn:20191223.2] Failed uploading the target image\nException occured while running the command\nTraceback (most recent call last):\n File \"/usr/lib/python3.6/site-packages/tripleoclient/command.py\", line 32, in run\n super(Command, self).run(parsed_args)\n File \"/usr/lib/python3.6/site-packages/osc_lib/command/command.py\", line 41, in run\n return super(Command, self).run(parsed_args)\n File \"/usr/lib/python3.6/site-packages/cliff/command.py\", line 184, in run\n return_code = self.take_action(parsed_args) or 0\n File \"/usr/lib/python3.6/site-packages/tripleoclient/v1/container_image.py\", line 95, in take_action\n uploader.upload()\n File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 262, in upload\n uploader.run_tasks()\n File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 2191, in run_tasks\n for result in p.map(upload_task, self.upload_tasks):\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 586, in result_iterator\n yield fs.pop().result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 432, in result\n return self.__get_result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 2244, in upload_task\n return uploader.upload_image(task)\n File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 1314, in upload_image\n session=target_session)\n File \"/usr/lib/python3.6/site-packages/tenacity/__init__.py\", line 292, in wrapped_f\n return self.call(f, *args, **kw)\n File \"/usr/lib/python3.6/site-packages/tenacity/__init__.py\", line 358, in call\n do = self.iter(retry_state=retry_state)\n File \"/usr/lib/python3.6/site-packages/tenacity/__init__.py\", line 319, in iter\n return fut.result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 425, in result\n return self.__get_result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib/python3.6/site-packages/tenacity/__init__.py\", line 361, in call\n result = fn(*args, **kwargs)\n File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 907, in _cross_repo_mount\n uploaded_layers=cls._global_view_proxy())\n File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_export.py\", line 174, in cross_repo_mount\n uploaded_layers, layer, scope='local')\n File \"/usr/lib/python3.6/site-packages/tripleo_common/utils/image.py\", line 22, in uploaded_layers_details\n known_layer = uploaded_layers.get(layer, None)\nAttributeError: 'NoneType' object has no attribute 'get'\n'NoneType' object has no attribute 'get'\nEND return value: 1", "stderr_lines": [ "START with options: overcloud container image upload --verbose --config-file /home/stack/ovn_container_images.yaml", "command: overcloud container image upload -> tripleoclient.v1.container_image.UploadImage (auth=False)", "Using config files: ['/home/stack/ovn_container_images.yaml']", "[rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-ovn-northd:20191223.2] Starting upload image process", "[rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-neutron-metadata-agent-ovn:20191223.2] Starting upload image process", "[rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-ovn-controller:20191223.2] Starting upload image process", "[rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-neutron-server-ovn:20191223.2] Starting upload image process", "No lock information provided for value None", "[192.168.24.1:8787/rh-osbs/rhosp15-openstack-ovn-northd:20191223.2] Failed uploading the target image", "No lock information provided for value None", "[192.168.24.1:8787/rh-osbs/rhosp15-openstack-neutron-metadata-agent-ovn:20191223.2] Failed uploading the target image", "No lock information provided for value None", "[192.168.24.1:8787/rh-osbs/rhosp15-openstack-ovn-controller:20191223.2] Failed uploading the target image", "No lock information provided for value None", "[192.168.24.1:8787/rh-osbs/rhosp15-openstack-neutron-server-ovn:20191223.2] Failed uploading the target image", "Exception occured while running the command", "Traceback (most recent call last):", " File \"/usr/lib/python3.6/site-packages/tripleoclient/command.py\", line 32, in run", " super(Command, self).run(parsed_args)", " File \"/usr/lib/python3.6/site-packages/osc_lib/command/command.py\", line 41, in run", " return super(Command, self).run(parsed_args)", " File \"/usr/lib/python3.6/site-packages/cliff/command.py\", line 184, in run", " return_code = self.take_action(parsed_args) or 0", " File \"/usr/lib/python3.6/site-packages/tripleoclient/v1/container_image.py\", line 95, in take_action", " uploader.upload()", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 262, in upload", " uploader.run_tasks()", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 2191, in run_tasks", " for result in p.map(upload_task, self.upload_tasks):", " File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 586, in result_iterator", " yield fs.pop().result()", " File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 432, in result", " return self.__get_result()", " File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result", " raise self._exception", " File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run", " result = self.fn(*self.args, **self.kwargs)", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 2244, in upload_task", " return uploader.upload_image(task)", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 1314, in upload_image", " session=target_session)", " File \"/usr/lib/python3.6/site-packages/tenacity/__init__.py\", line 292, in wrapped_f", " return self.call(f, *args, **kw)", " File \"/usr/lib/python3.6/site-packages/tenacity/__init__.py\", line 358, in call", " do = self.iter(retry_state=retry_state)", " File \"/usr/lib/python3.6/site-packages/tenacity/__init__.py\", line 319, in iter", " return fut.result()", " File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 425, in result", " return self.__get_result()", " File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result", " raise self._exception", " File \"/usr/lib/python3.6/site-packages/tenacity/__init__.py\", line 361, in call", " result = fn(*args, **kwargs)", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 907, in _cross_repo_mount", " uploaded_layers=cls._global_view_proxy())", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_export.py\", line 174, in cross_repo_mount", " uploaded_layers, layer, scope='local')", " File \"/usr/lib/python3.6/site-packages/tripleo_common/utils/image.py\", line 22, in uploaded_layers_details", " known_layer = uploaded_layers.get(layer, None)", "AttributeError: 'NoneType' object has no attribute 'get'", "'NoneType' object has no attribute 'get'", "END return value: 1" ], "stdout": "", "stdout_lines": [] } to retry, use: --limit @/home/rsafrono/.infrared/plugins/tripleo-ovn-migration/migration/infrared/tripleo-ovn-migration/main.retry PLAY RECAP *********************************************************************************************************************************************************************************** undercloud-0 : ok=31 changed=9 unreachable=0 failed=1 ERROR Playbook "/home/rsafrono/.infrared/plugins/tripleo-ovn-migration/./migration/infrared/tripleo-ovn-migration/main.yml" failed! (overcloud) [stack@undercloud-0 ~]$ cat /home/stack/ovn_container_images.yaml container_images: - imagename: rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-ovn-northd:20191223.2 - imagename: rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-ovn-controller:20191223.2 - imagename: rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-neutron-server-ovn:20191223.2 - imagename: rhos-qe-mirror-tlv.usersys.redhat.com:5002/rh-osbs/rhosp15-openstack-neutron-metadata-agent-ovn:20191223.2 (overcloud) [stack@undercloud-0 ~]$ cat /etc/containers/registries.conf # This is a system-wide configuration file used to # keep track of registries for various container backends. # It adheres to TOML format and does not support recursive # lists of registries. # The default location for this configuration file is /etc/containers/registries.conf. # The only valid categories are: 'registries.search', 'registries.insecure', # and 'registries.block'. [registries.search] registries = ['registry.redhat.io', 'registry.access.redhat.com', 'quay.io', 'docker.io'] # If you need to access insecure registries, add the registry's fully-qualified name. # An insecure registry is one that does not have a valid SSL certificate or only does HTTP. [registries.insecure] registries = ['192.168.24.1', '192.168.24.3', 'rhos-qe-mirror-tlv.usersys.redhat.com:5002', 'docker-registry.upshift.redhat.com'] # If you need to block pull access from a registry, uncomment the section below # and add the registries fully-qualified name. # # Docker only [registries.block] registries = [] Version-Release number of selected component (if applicable): 15.0-RHEL-8/RHOS_TRUNK-15.0-RHEL-8-20191223.n.1 the issue relevant also for OSP16 How reproducible: 100% Steps to Reproduce: 1. Deploy OSP15 Note: I used this CI job: https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/network/view/networking-ovn/job/DFG-network-networking-ovn-15_director-rhel-virthost-3cont_2comp-ipv4-vxlan-ml2ovs-to-ovn-migration/239/ 2. Run OVN migration procedure I did it with infrared a) imported workspace from the above job b) infrared plugin remove all c) infrared plugin add all d) infrared plugin add https://github.com/openstack/networking-ovn.git e) infrared tripleo-ovn-migration -vvv --version 15 --registry-mirror rhos-qe-mirror-tlv.usersys.redhat.com:5000 --registry-prefix rhosp15-openstack --registry-namespace rh-osbs Actual results: The process failed on TASK [prepare-migration : Upload the ovn docker images to the local registry] Container images were not uploaded to the local registry Expected results: Container images uploaded successfully to the local registry Additional info: