Bug 1685621
| Summary: | Mistral's eventlet monkeypatching breaks git-python | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Damien Ciabrini <dciabrin> |
| Component: | openstack-tripleo-common | Assignee: | Emilien Macchi <emacchi> |
| Status: | CLOSED ERRATA | QA Contact: | Sasha Smolyak <ssmolyak> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 15.0 (Stein) | CC: | apetrich, aschultz, emacchi, jjoyce, jschluet, mburns, michele, slinaber, tvignaud |
| Target Milestone: | beta | Keywords: | Triaged |
| Target Release: | 15.0 (Stein) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-tripleo-common-10.4.1-0.20190308154822.63ab018.el8ost | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-09-21 11:20:32 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Submitted an upstream fix. The issue as discussed upstream is that gitpython subprocess is not being properly patched by mistral. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:2811 |
Description of problem: When deploying an overcloud, the openstack overcloud commands uses mistral to signal progress on the deployment, and fails: Enabling ssh admin - COMPLETE. Waiting for messages on queue 'tripleo' with no timeout. The action raised an exception [action_ex_id=23ec3e50-9a0f-448c-8f86-5f7553357fe3, action_cls='<class 'mistral.actions.action_factory.GetOvercloudConfig'>', attributes='{}', params='{'container': 'overcloud', 'container_config': 'overclou d-config'}'] module 'select' has no attribute 'poll' Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 29, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 945, in take_action verbosity=self.app_args.verbose_level) File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py", line 323, in config_download raise exceptions.DeploymentError("Overcloud configuration failed.") tripleoclient.exceptions.DeploymentError: Overcloud configuration failed. Overcloud configuration failed. Full error: 2019-03-02 03:43:20.127 1 WARNING mistral.executors.default_executor [req-0568807f-eb53-4a31-a05b-afcf0631330b 07085b77ed23481ab53cc174dfb53f08 ec0093095eae49b1a5e67d54894e2b0d - default default] The action raised an exception [action_ex_ id=23ec3e50-9a0f-448c-8f86-5f7553357fe3, action_cls='<class 'mistral.actions.action_factory.GetOvercloudConfig'>', attributes='{}', params='{'container': 'overcloud', 'container_config': 'overcloud-config'}'] module 'select' has no attribute 'poll': AttributeError: module 'select' has no attribute 'poll' 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor Traceback (most recent call last): 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib/python3.6/site-packages/mistral/executors/default_executor.py", line 114, in run_action 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor result = action.run(action_ctx) 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib/python3.6/site-packages/tripleo_common/actions/config.py", line 87, in run 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor commit_message=message) 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib/python3.6/site-packages/tripleo_common/utils/config.py", line 492, in download_config 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor git_repo = self.initialize_git_repo(config_dir) 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib/python3.6/site-packages/tripleo_common/utils/config.py", line 140, in initialize_git_repo 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor repo = git.Repo.init(dirname) 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib/python3.6/site-packages/git/repo/base.py", line 764, in init 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor git.init(**kwargs) 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib/python3.6/site-packages/git/cmd.py", line 440, in <lambda> 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor return lambda *args, **kwargs: self._call_process(name, *args, **kwargs) 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib/python3.6/site-packages/git/cmd.py", line 834, in _call_process 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor return self.execute(make_call(), **_kwargs) 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib/python3.6/site-packages/git/cmd.py", line 587, in execute 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor stdout_value, stderr_value = proc.communicate() 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib64/python3.6/subprocess.py", line 863, in communicate 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor stdout, stderr = self._communicate(input, endtime, timeout) 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib64/python3.6/subprocess.py", line 1521, in _communicate 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor with _PopenSelector() as selector: 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor File "/usr/lib64/python3.6/selectors.py", line 348, in __init__ 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor self._poll = select.poll() 2019-03-02 03:43:20.127 1 ERROR mistral.executors.default_executor AttributeError: module 'select' has no attribute 'poll' Quoting Michele: """ This is (another) eventlet monkeypatching issue. Reproducer: import git import eventlet from eventlet import timeout as ev_timeout import sys eventlet.monkey_patch( os=True, select=True, socket=True, thread=False if '--use-debugger' in sys.argv else True, time=True) """ Version-Release number of selected component (if applicable): openstack-mistral-executor:20190226.1 How reproducible: Always Steps to Reproduce: 1. deploy an overcloud 2. 3. Actual results: mistral fails, deployment never proceeds Expected results: no failure Additional info: