Bug 1891200
| Summary: | rabbitmq-limits validation is broken because max_fds is no longer defined from rabbit's point of view | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Alex Stupnikov <astupnik> | |
| Component: | openstack-tripleo-validations | Assignee: | chrisbro <chrisbro> | |
| Status: | CLOSED ERRATA | QA Contact: | nlevinki <nlevinki> | |
| Severity: | low | Docs Contact: | ||
| Priority: | low | |||
| Version: | 16.1 (Train) | CC: | aschultz, cfields, chrisbro, dabarzil, gchamoul, jjoyce, johfulto, jpodivin, jschluet, lmiccini, slinaber, tvignaud, uemit.seren | |
| Target Milestone: | z8 | Keywords: | Reopened, Triaged | |
| Target Release: | 16.1 (Train on RHEL 8.2) | |||
| Hardware: | All | |||
| OS: | All | |||
| Whiteboard: | ||||
| Fixed In Version: | openstack-tripleo-validations-11.3.2-1.20210913123540.4db92ba.el8ost | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1987090 (view as bug list) | Environment: | ||
| Last Closed: | 2022-01-25 16:49:07 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1912879 | |||
| Bug Blocks: | 1987090 | |||
*** Bug 1896409 has been marked as a duplicate of this bug. *** Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.4 director bug fix advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:0817 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.4 director bug fix advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:0817 This validator is still broken so I'm opening this bug back up.
run: openstack tripleo validator run --validation rabbitmq-limits
run: openstack tripleo validator show run 03d3af96-8612-4f15-b5a2-c6b999b75f78
"hosts": {
"overcloud-controller-0": {
"_ansible_no_log": false,
"action": "command",
"changed": false,
"cmd": "\"podman\" exec $(\"podman\" ps -q --filter \"name=rabbitmq\" | head -1) rabbitmqctl eval 'sysctl -n fs.file-max'",
"delta": "0:00:01.557734",
"end": "2021-04-16 20:58:32.249211",
"failed": true,
"invocation": {
"module_args": {
"_raw_params": "\"podman\" exec $(\"podman\" ps -q --filter \"name=rabbitmq\" | head -1) rabbitmqctl eval 'sysctl -n fs.file-max'",
"_uses_shell": true,
"argv": null,
"chdir": null,
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"stdin_add_newline": true,
"strip_empty_ends": true,
"warn": true
}
},
"msg": "non-zero return code",
"rc": 64,
"start": "2021-04-16 20:58:30.691477",
"stderr": "Error (argument validation): \"syntax error before: fs\"\nArguments given:\n\teval sysctl -n fs.file-max\n\n\u001b[1mUsage\u001b[0m\n\nrabbitmqctl [--node <node>] [--longnames] [--quiet] eval <expression>\nError: non zero exit code: 64: OCI runtime error",
"stderr_lines": [
"Error (argument validation): \"syntax error before: fs\"",
"Arguments given:",
"\teval sysctl -n fs.file-max",
"",
"\u001b[1mUsage\u001b[0m",
"",
"rabbitmqctl [--node <node>] [--longnames] [--quiet] eval <expression>",
"Error: non zero exit code: 64: OCI runtime error"
],
"stdout": "",
"stdout_lines": []
}
},
"name": "Get file_descriptors total_limit",
"status": "FAILED"
}
The 'eval' statement expect ERLANG code and no longer belongs here. This works: podman exec $(podman ps -q --filter \"name=rabbitmq\" | head -1) sysctl -n fs.file-max Backports of the patch with fix are waiting for review. This is still occuring on OSP 16.2 The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days |
Description of problem: rabbitmq-limits validation uses "rabbitmqctl eval" command to obtain max_fds value from RabbitMQ instance running inside podman container [1]. In customer's environment and in my RHOSP 16.1 lab this check is broken: max_fds value is undefined from rabbit perspective: # podman exec 1e1970fab36f rabbitmqctl eval 'proplists:get_value(max_fds, erlang:system_info(check_io)).' undefined And check fails with the following error: "task": { "hosts": { "overcloud-controller-0": { "_ansible_no_log": false, "action": "fail", "changed": false, "failed": true, "failed_when_result": true, "msg": "undefined must be greater than or equal to 16384" } }, "name": "Verify the actual limit exceeds the minimal value", "status": "FAILED" } I am not sure if this check is still relevant in its current state: AFAIU we enforce various limitations using podman settings. In any case would like to escalate to developers and ask for a fix. [1] https://github.com/openstack/tripleo-validations/blob/stable/train/roles/rabbitmq_limits/tasks/main.yml#L13