Bug 2046194
Summary: | CVE-2022-0485 libnbd: nbdcopy ignore read and write errors - destination image corrupted [rhel-9.0] | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | Nir Soffer <nsoffer> |
Component: | libnbd | Assignee: | Richard W.M. Jones <rjones> |
Status: | CLOSED ERRATA | QA Contact: | Vera <vwu> |
Severity: | medium | Docs Contact: | |
Priority: | high | ||
Version: | CentOS Stream | CC: | bstinson, eblake, jwboyer, kkiwi, lersek, mxie, rjones, tyan, tzheng, virt-maint, vwu, xiaodwan |
Target Milestone: | rc | Keywords: | Security, SecurityTracking, Triaged |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | libnbd-1.10.5-1.el9 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2022-05-17 12:51:02 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 2045718, 2050324 |
Description
Nir Soffer
2022-01-26 11:28:56 UTC
Created attachment 1855494 [details]
nbdkit plugin for injecting read and write errors
Upstream patch proposed: https://listman.redhat.com/archives/libguestfs/2022-February/msg00039.html Hello Vera, would it possible for you to QA ACK this please when you are back from holiday. https://gitlab.com/redhat/centos-stream/rpms/libnbd/-/merge_requests/14 (Waiting on QA ACK) Verified with libnbd-1.10.4-1.el9.x86_64 Reproduced with libnbd-1.10.3-1.el9.x86_64 Steps: 1. Create source image # dd if=/dev/zero bs=1M count=4 status=none | tr "\0" "B" > src.img # hexdump -C src.img 00000000 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 |BBBBBBBBBBBBBBBB| * 00400000 2. Create destination image # dd if=/dev/zero bs=1M count=4 status=none | tr "\0" "A" > dst.img # hexdump -C dst.img 00000000 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 |AAAAAAAAAAAAAAAA| * 00400000 3. Copy from nbdkit to destination image # nbdcopy nbd://localhost dst.img; echo $? 0 4. Check destination image # hexdump -C dst.img ...... 002655c0 65 5f 65 78 74 34 5f 5f 6d 62 61 6c 6c 6f 63 0a |e_ext4__mballoc.| 002655d0 66 66 66 66 66 66 66 66 38 31 33 63 32 36 35 30 |ffffffff813c2650| 002655e0 20 74 20 5f 5f 62 70 66 5f 74 72 61 63 65 5f 65 | t __bpf_trace_e| 002655f0 78 74 34 5f 64 69 72 65 63 74 5f 49 4f 5f 65 78 |xt4_direct_IO_ex| 00265600 69 74 0a 66 66 66 66 66 66 66 66 38 31 33 63 32 |it.ffffffff813c2| 00265610 36 36 30 20 74 20 5f 5f 62 70 66 5f 74 72 61 63 |660 t __bpf_trac| 00265620 65 5f 65 78 74 34 5f 65 78 74 5f 68 61 6e 64 6c |e_ext4_ext_handl| 00265630 65 5f 75 6e 77 72 69 74 74 65 6e 5f 65 78 74 65 |e_unwritten_exte| 00265640 6e 74 73 0a 66 66 66 66 66 66 66 66 38 31 33 63 |nts.ffffffff813c| 00265650 32 36 37 30 20 74 20 5f 5f 62 70 66 5f 74 72 61 |2670 t __bpf_tra| 00265660 63 65 5f 65 78 74 34 5f 72 65 6d 6f 76 65 5f 62 |ce_ext4_remove_b| 00265670 6c 6f 63 6b 73 0a 66 66 66 66 66 66 66 66 38 31 |locks.ffffffff81| 00265680 33 63 32 36 38 30 20 74 20 5f 5f 62 70 66 5f 74 |3c2680 t __bpf_t| 00265690 72 61 63 65 5f 65 78 74 34 5f 65 73 5f 73 68 72 |race_ext4_es_shr| 002656a0 69 6e 6b 0a 66 66 66 66 66 66 66 66 38 31 33 63 |ink.ffffffff813c| 002656b0 32 36 39 30 20 74 20 5f 5f 62 70 66 5f 74 72 61 |2690 t __bpf_tra| 002656c0 63 65 5f 65 78 74 34 5f 66 69 6e 64 5f 64 65 6c |ce_ext4_find_del| 002656d0 61 6c 6c 6f 63 5f 72 61 6e 67 65 0a 66 66 66 66 |alloc_range.ffff| ...... Verified with libnbd-1.10.4-1.el9.x86_64 Steps: 1. Create source image # dd if=/dev/zero bs=1M count=4 status=none | tr "\0" "A" > dst.img # hexdump -C dst.img 00000000 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 |AAAAAAAAAAAAAAAA| * 00400000 2. Create destination image # dd if=/dev/zero bs=1M count=4 status=none | tr "\0" "B" > src.img # hexdump -C src.img 00000000 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 |BBBBBBBBBBBBBBBB| * 00400000 3. Copy from nbdkit to destination image # nbdcopy nbd://localhost dst.img; echo $? 0 4. Check destination image # hexdump -C dst.img 00000000 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 |AAAAAAAAAAAAAAAA| * 00400000 Moving to verified:Tested Hi Vera, I don't understand the reproducer. Is there a missing step between 2. and 3.? ie. a step where you run an NBD server? Anyway I have a much simpler, one-line reproducer: $ nbdkit -U - eval get_size=' echo 100M ' pread=' if [ $4 -lt 10000000 ]; then dd if=/dev/zero count=$3 iflag=count_bytes ; else echo EIO Failed >&2 ; exit 1; fi ' --run 'nbdcopy $uri null: -p' For libnbd-1.10.3-1.el9.x86_64: ... nbdkit: eval[1]: error: /tmp/nbdkitDT21TP/pread: Failed nbdkit: eval[1]: error: /tmp/nbdkitDT21TP/pread: Failed nbdkit: eval[1]: error: /tmp/nbdkitDT21TP/pread: Failed nbdkit: eval[1]: error: /tmp/nbdkitDT21TP/pread: Failed █ 100% [****************************************] $ echo $? 0 Notice that even though the server sent errors to nbdcopy, it succeeded. For libnbd-1.10.4-1.el9.x86_64: $ nbdkit -U - eval get_size=' echo 100M ' pread=' if [ $4 -lt 10000000 ]; then dd if=/dev/zero count=$3 iflag=count_bytes ; else echo EIO Failed >&2 ; exit 1; fi ' --run 'nbdcopy $uri null: -p' nbdkit: eval[1]: error: /tmp/nbdkitQtKlRw/pread: Failed read at offset 10223616 failed: Input/output error nbdkit: eval[1]: error: /tmp/nbdkitQtKlRw/pread: Failed nbdkit: eval[1]: error: write error reply: Broken pipe $ echo $? 1 The command correctly fails with the new nbdcopy. Verified with the version: libnbd-1.10.4-1.el9.x86_64 # nbdkit -U - eval get_size=' echo 100M ' pread=' if [ $4 -lt 10000000 ]; then dd if=/dev/zero count=$3 iflag=count_bytes ; else echo EIO Failed >&2 ; exit 1; fi ' --run 'nbdcopy $uri null: -p' nbdkit: eval[1]: error: /tmp/nbdkit0BrU0q/pread: Failed read at offset 10223616 failed: Input/output error nbdkit: eval[1]: error: /tmp/nbdkit0BrU0q/pread: Failed nbdkit: eval[1]: error: /tmp/nbdkit0BrU0q/pread: Failed nbdkit: eval[1]: error: write error reply: Broken pipe # echo $? 1 Moving to Verified. Vera, FYI libnbd-1.10.5-1.el9 contains some extra fixes which are related to this CVE: https://gitlab.com/nbdkit/libnbd/-/commit/e15864c364aef710a7826b2b25c88a360f9819d6 https://gitlab.com/nbdkit/libnbd/-/commit/56d2611bd6fcdb559ee5ff11532dec75eb2f8472 https://gitlab.com/nbdkit/libnbd/-/commit/c97b12493c01b09b4faf41120cc11cf03780b51b https://gitlab.com/nbdkit/libnbd/-/commit/c79706af4e7475bf58861a143b77b77a54e7a1cd If you want to test these, maybe the only thing to test is that the two new APIs appear (nbd_set_pread_initialize, nbd_get_pread_initialize), and the libnbd-security(3) man page has been updated with the CVE. Verified with the version: libnbd-1.10.5-1.el9.x86_64 1. Verify the nbdcopy: # nbdkit -U - eval get_size=' echo 100M ' pread=' if [ $4 -lt 10000000 ]; then dd if=/dev/zero count=$3 iflag=count_bytes ; else echo EIO Failed >&2 ; exit 1; fi ' --run 'nbdcopy $uri null: -p' nbdkit: eval[1]: error: /tmp/nbdkitsjMPHS/pread: Failed read at offset 10223616 failed: Input/output error nbdkit: eval[1]: error: /tmp/nbdkitsjMPHS/pread: Failed nbdkit: eval[1]: error: /tmp/nbdkitsjMPHS/pread: Failed nbdkit: eval[1]: error: write error reply: Broken pipe # echo $? 1 2. Verify the two new APIs:(nbd_aio_command_completed, nbd_get_pread_initialize) appear in the nbdsh; # nbdsh Welcome to nbdsh, the shell for interacting with Network Block Device (NBD) servers. The ‘nbd’ module has already been imported and there is an open NBD handle called ‘h’. h.connect_tcp("remote", "10809") # Connect to a remote server. h.get_size() # Get size of the remote disk. buf = h.pread(512, 0, 0) # Read the first sector. exit() or Ctrl-D # Quit the shell help(nbd) # Display documentation nbd> help(nbd) .... | aio_command_completed(self, cookie) | ▶ check if the command completed | | Return true if the command completed. If this function | returns true then the command was successful and it has | been retired. Return false if the command is still in | flight. This can also fail with an error in case the | command failed (in this case the command is also | retired). A command is retired either via this command, | or by using a completion callback which returns 1. | | The "cookie" parameter is the positive unique 64 bit | cookie for the command, as returned by a call such as | "nbd.aio_pread". ...... | get_pread_initialize(self) | ▶ see whether libnbd pre-initializes read buffers | | Return whether libnbd performs a pre-initialization of a | buffer passed to "nbd.pread" and similar to all zeroes, | as set by "nbd.set_pread_initialize". .... 3. Check the doc on the CVE; Download the src pkg from brew: # ls libnbd-1.10.5-1.el9.src.rpm check the docs: # rpmbuild -rp libnbd-1.10.5-1.el9.src.rpm # cat /root/rpmbuild/BUILD/libnbd-1.10.5/docs/libnbd-security.pod |grep 2022 -A 5 =head2 CVE-2022-0485 silent data corruption when using L<nbdcopy(1)> See the full announcement here: L<https://listman.redhat.com/archives/libguestfs/2022-February/msg00104.html> =head1 SEE ALSO L<libnbd(3)>. -- Copyright (C) 2019-2022 Red Hat Inc. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (new packages: libnbd), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:2409 |