Bug 1727865
| Summary: | Initialization of cyrus-sasl library is not thread safe causing libvirt client crashes | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Daniel Berrangé <berrange> | ||||
| Component: | libvirt | Assignee: | Daniel Berrangé <berrange> | ||||
| Status: | CLOSED ERRATA | QA Contact: | yafu <yafu> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 8.0 | CC: | jdenemar, rbalakri, yalzhang | ||||
| Target Milestone: | rc | ||||||
| Target Release: | 8.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | libvirt-6.0.0-17.el8 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1727866 (view as bug list) | Environment: | |||||
| Last Closed: | 2020-11-04 02:53:02 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1727866 | ||||||
| Attachments: |
|
||||||
Fix is upstream in
commit 7722606b369e569529ac30130f04abe17cae57cf (HEAD -> master, origin/master)
Author: Daniel P. Berrangé <berrange>
Date: Mon Jul 8 11:32:38 2019 +0100
rpc: ensure thread safe initialization of SASL library
Reproduced with libvirt-daemon-4.5.0-37.el8.x86_64. Verified with libvirt-daemon-6.0.0-27.module+el8.3.0+7602+4b93512e.x86_64. Test steps: 1. Edit /etc/libvirt/libvirtd.conf to set: auth_unix_rw = "sasl" 2. Edit /etc/sasl2/libvirt.conf to set: mech-list: scram-sha-1 3.generate sasl user and restart libvirtd service #echo redhat |saslpasswd2 -p -a libvirt redhat #systemctl restart libvirtd 4. virsh -c qemu:///system list to check that it prompts for user + password correctly 5. Compile attached demo with "gcc -g -o saslrace saslrace.c -lvirt -lpthread" 6.Run './saslrace' and no coredump happened： # ./saslrace Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Waiting for threads to initialize Telling threads to proceed Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: Please enter your authentication name: test Please enter your password: Please enter your password: Open qemu:///system test Please enter your password: Open qemu:///system test Please enter your password: Open qemu:///system test Please enter your password: Open qemu:///system Please enter your password: Open qemu:///system test Please enter your password: Open qemu:///system ... Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: virt:rhel and virt-devel:rhel security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:4676 |
Created attachment 1588340 [details] SASL race demo program Description of problem: The sasl_client_init and/or sasl_server_init methods are required in order to initialize the SASL library global state. Libvirt currently assumes the methods are no-ops if called more than once which is only true if the calls are serialized. The protection against multiple invocation uses a simple integer counter with no atomic access/increments, nor is any libpthread protection used. This is a clear race for threads. Version-Release number of selected component (if applicable): libvirt-5.0.0-11.el8 How reproducible: Fairly often in a threaded test case Steps to Reproduce: 1. Edit /etc/libvirt/libvirt.conf to set auth_unix_rw = "sasl" 2. Edit /etc/sasl2/libvirt.conf to set mech-list: scram-sha-1 3. As root run "saslpasswd2 -a libvirt test" and enter "123456" as password when prompted 4. systemctl restart libvirtd 5. virsh -c qemu:///system list to check that it prompts for user + password correctly 6. Compile attached demo with "gcc -g -o saslrace saslrace.c -lvirt -lpthread" 7. ./saslrace Actual results: The test will often reporting that it is unable to start SASL (no mechanism available), and will also often end up crashing eg Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Waiting for threads to initialize Notifying we are ready Notifying we are ready Telling threads to proceed Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt libvirt: XML-RPC error : authentication failed: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found) Open failed libvirt: XML-RPC error : authentication failed: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found) Open failed libvirt: XML-RPC error : authentication failed: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found) Open failed libvirt: XML-RPC error : authentication failed: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found) Open failed libvirt: XML-RPC error : authentication failed: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found) Open failed libvirt: XML-RPC error : authentication failed: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found) Open failed Segmentation fault (core dumped) Expected results: The test should run without errors and without crash Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Notifying we are ready Waiting for threads to initialize Notifying we are ready Notifying we are ready Telling threads to proceed Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Opening libvirt Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Open qemu:///system Additional info: