Bug 2106975
| Summary: | [RHEL9.2] remove deprecated usage of bidirectional pipe | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | zhentang <zhetang> | ||||
| Component: | qemu-kvm | Assignee: | Marc-Andre Lureau <marcandre.lureau> | ||||
| qemu-kvm sub component: | Devices | QA Contact: | liunana <nanliu> | ||||
| Status: | CLOSED WONTFIX | Docs Contact: | |||||
| Severity: | medium | ||||||
| Priority: | low | CC: | chayang, coli, dzheng, jinzhao, juzhang, kkiwi, kraxel, lcheng, marcandre.lureau, nanliu, virt-maint, xuwei | ||||
| Version: | 9.1 | Keywords: | Triaged | ||||
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2022-11-29 19:52:16 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
I can reproduce this issue with a OVMF guest. And I can't reproduce this issue with a SEABIOS vm. Steps: 1. # mkfifo /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log 2. Boot ovmf guest with qemu command line: -chardev pipe,id=charserial1,path=/tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log \ -device isa-serial,id=serial1,chardev=charserial1 \ -chardev file,id=firmware,path=/tmp/edk2-2.log \ -device isa-debugcon,iobase=0x402,chardev=firmware \ 3. Check edk2 log and can get the firmware error log info: InstallProtocolInterface: D3B36F2C-D551-11D4-9A46-0090273FC14D 0 InstallProtocolInterface: D3B36F2D-D551-11D4-9A46-0090273FC14D 0 ASSERT /builddir/build/BUILD/edk2-16779ede2d36/MdeModulePkg/Universal/Console/ConSplitterDxe/ConSplitter.c(3116): !(((INTN)(RETURN_STATUS)(Status)) < 0) Hi Cong, Could your team please help to check this? Seems it is the ovmf issue. Feel free to change this if you have any different comments. Thanks. Best regards Nana Liu Test Env for Comment 1: 5.14.0-127.el9.x86_64 edk2-ovmf-20220526git16779ede2d36-2.el9.noarch qemu-kvm-7.0.0-8.el9.x86_64 Guest: 5.14.0-127.el9.x86_64 (In reply to liunana from comment #1) > I can reproduce this issue with a OVMF guest. And I can't reproduce this > issue with a SEABIOS vm. > > Steps: > 1. # mkfifo /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log > > 2. Boot ovmf guest with qemu command line: > -chardev > pipe,id=charserial1,path=/tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log \ > -device isa-serial,id=serial1,chardev=charserial1 \ > -chardev file,id=firmware,path=/tmp/edk2-2.log \ > -device isa-debugcon,iobase=0x402,chardev=firmware \ > > > 3. Check edk2 log and can get the firmware error log info: > InstallProtocolInterface: D3B36F2C-D551-11D4-9A46-0090273FC14D 0 > InstallProtocolInterface: D3B36F2D-D551-11D4-9A46-0090273FC14D 0 > ASSERT > /builddir/build/BUILD/edk2-16779ede2d36/MdeModulePkg/Universal/Console/ > ConSplitterDxe/ConSplitter.c(3116): !(((INTN)(RETURN_STATUS)(Status)) < 0) > > > > Hi Cong, > > Could your team please help to check this? Seems it is the ovmf issue. > Feel free to change this if you have any different comments. > Thanks. > > > Best regards > Nana Liu Tested with commands[1], not hit this issue. Tested with commands[2] hit this issue. Gerd, can you help take a look at it. Many thanks. Versions: kernel-5.14.0-111.el9.x86_64 qemu-kvm-7.0.0-6.el9 edk2-ovmf-20220526git16779ede2d36-2.el9.noarch [1] -chardev socket,path=/var/tmp/avocado__wundsot/serial-serial0-20220616-043524-GraiIcqU,id=chardev_serial0,server=on,wait=off \ -device isa-serial,id=serial0,chardev=chardev_serial0 \ [2] -chardev pipe,id=charserial1,path=/tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log \ -device isa-serial,id=serial1,chardev=charserial1 \ (In reply to liunana from comment #1) > Steps: > 1. # mkfifo /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log > -chardev > pipe,id=charserial1,path=/tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log \ This is wrong. You need *two* pipes: /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log.in and /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log.out (In reply to Gerd Hoffmann from comment #4) > (In reply to liunana from comment #1) > > Steps: > > 1. # mkfifo /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log > > > -chardev > > pipe,id=charserial1,path=/tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log \ > > This is wrong. You need *two* pipes: > /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log.in and > /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log.out Hi Gerd, Thanks for checking this, I can boot up the ovmf guest successfully with two pipes. But could you please explain more about this? Seems the way to create a single pipe 'mkfifo /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log' shouldn't block the vm booting. And I can boot up a seabios vm with one single pipe. -chardev socket,id=charserial0,path=/tmp/testserial,server,nowait \ -device isa-serial,chardev=charserial0,id=serial0 \ -chardev pipe,id=charserial1,path=/tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log \ -device isa-serial,id=serial1,chardev=charserial1 \ The same command line doesn't work for a ovmf guest. Thank you in advance! Best regards Nana Liu > But could you please explain more about this? > Seems the way to create a single pipe 'mkfifo > /tmp/avocado_64lsem4o/serial_pipe_TrAM-1.log' shouldn't block the vm booting. Well, with a single pipe you'll create a loopback, anything written to the serial device will be sent back as input. That simply can't work. > And I can boot up a seabios vm with one single pipe. Pure luck that the input doesn't cause boot problems. Test Env:
5.14.0-127.el9.x86_64
qemu-kvm-7.0.0-8.el9.x86_64
edk2-ovmf-20220526git16779ede2d36-2.el9.noarch
Guest: 5.14.0-127.el9.x86_64
Test scenrios:
1. Boot a vm with one-way pipe + one isa-serial
qemu cmdline:
-chardev pipe,id=charserial1,path=/tmp/one-way.pipe \
-device isa-serial,id=serial1,chardev=charserial1 \
test results:
seabios vm:
a. guest keeps loading into the kernel entry screen in a loop
b. firmware log:
enter handle_19:
NULL
Booting from Hard Disk...
Booting from 0000:7c00
ovmf vm:
a. guest is stuck at a black screen.
b. firmware log:
ASSERT /builddir/build/BUILD/edk2-16779ede2d36/MdeModulePkg/Universal/Console/ConSplitterDxe/ConSplitter.c(3116): !(((INTN)(RETURN_STATUS)(Status)) < 0)
2. Boot a vm with one socket chardev + one-way pipe + two isa-serial
qemu cmdline:
-chardev socket,id=chardev_serial0,server=on,path=/tmp/serial-serial0,wait=off \
-device isa-serial,id=serial0,chardev=chardev_serial0 \
-chardev pipe,id=charserial1,path=/tmp/one-way.pipe \
-device isa-serial,id=serial1,chardev=charserial1 \
test results:
seabios vm:
a. guest can boot up sccessfully, and the one-way pipe also works well, I can send data from guest to host by /de/ttyS1 inside guest. But seems it can't send date from host to guest.
ovmf vm:
a. guest boots into the Boot Manager screen then it is stuck at a black screen
b. firmware log:
ASSERT /builddir/build/BUILD/edk2-16779ede2d36/MdeModulePkg/Universal/Console/ConSplitterDxe/ConSplitter.c(3116): !(((INTN)(RETURN_STATUS)(Status)) < 0)
3. Boot a vm with one-way pipe + virtio-serial
qemu cmdline:
-device pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,chassis=5 \
-device virtio-serial-pci,id=virtio-serial0,bus=pcie-root-port-4,max_ports=31,addr=0x0 \
-chardev pipe,id=charserial1,path=/tmp/one-way.pipe \
-device virtserialport,chardev=charserial1,name=port0,bus=virtio-serial0.0,id=port0 \
test results:
seabios vm:
a. guest can boot up sccessfully, and the one-way pipe also works well, I can send data between guest and host by /dev/vport2p1 inside guest.
ovmf vm:
a. guest can boot up sccessfully, and the one-way pipe also works well, I can send data between guest and host by /dev/vport2p1 inside guest.
4. Boot a vm with two pipe files + isa-serial
create pipe:
# mkfifo mypipe.in
# mkfifo mypipe.out
qemu cmdline:
-chardev pipe,id=charserial1,path=/tmp/mypipe \
-device isa-serial,id=serial1,chardev=charserial1 \
test results:
seabios vm:
a. guest boots up sccessfully, and can get the guest boot log from mypipe.out.
Besides, can send data from guest to host by mypipe.out on host and /dev/ttyS0 inside guest, and can send data from host to guest by mypipe.in on host an /dev/ttyS0 inside guest.
But for the data transfer from host to guest, sometime the guest only can recieve one data character string even I send two data character string.
And the connection will disconnect automitically inside guest after a few data character string, I use the command 'cat /dev/ttyS0' inside guest, please also check this if it is a issue.
I would like to put the data character string here "fkdh3243254353534fsf" that I transfered between host and guest.
ovmf vm: the same behavior with above seabios vm.
Hi Gerd and Marc-André,
Would you please help to check these tests? Seems one-way pipe works on some conditions. Thanks a lot!
Best regards
Nana Liu
> Hi Gerd and Marc-André,
>
> Would you please help to check these tests? Seems one-way pipe works on some
> conditions. Thanks a lot!
As mentioned before, this is pure luck.
It doesn't imply this is a valid test case.
A pipe is basically a *single* buffer you can write to and read from.
This is different from a socket where you have two buffers, one for each direction.
Therefore you need two pipes, one for each direction.
Not sure why qemu allows a single-pipe configuration in the first place.
1. Boot a vm with one-way pipe + one isa-serial
qemu cmdline:
-chardev pipe,id=charserial1,path=/tmp/one-way.pipe \
-device isa-serial,id=serial1,chardev=charserial1 \
test results:
seabios vm:
a. guest keeps loading into the kernel entry screen in a loop
Fwiw, I do not observe that behaviour with a RHEL8 VM. Did you change the kernel arguments or guest configuration to interact with serial?
(In reply to Marc-Andre Lureau from comment #9) > 1. Boot a vm with one-way pipe + one isa-serial > qemu cmdline: > -chardev pipe,id=charserial1,path=/tmp/one-way.pipe \ > -device isa-serial,id=serial1,chardev=charserial1 \ > test results: > seabios vm: > a. guest keeps loading into the kernel entry screen in a loop > > Fwiw, I do not observe that behaviour with a RHEL8 VM. Did you change the > kernel arguments or guest configuration to interact with serial? No, no special configuration for guest. And I still can reproduce this with a RHEL8.7 VM. And I upload the video to the attachments, could you please help to check the Comment 10? This is the RHEL8 guest kernel cmdline: # cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-4.18.0-409.el8.x86_64 root=/dev/mapper/rhel-root ro reboot=pci biosdevname=0 crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap console=ttyS0,115200 This is the RHEL9 guest kernel cmdline. # cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.14.0-131.el9.x86_64 root=/dev/mapper/rhel-root ro console=tty0 reboot=pci biosdevname=0 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap console=ttyS0,115200 And I still can reproduce this with or without the guest kernel parameter "console=ttyS0,115200". Best regards Nana Liu (In reply to liunana from comment #11) > > Fwiw, I do not observe that behaviour with a RHEL8 VM. Did you change the > > kernel arguments or guest configuration to interact with serial? > > No, no special configuration for guest. > And I still can reproduce this with a RHEL8.7 VM. And I upload the video to > the attachments, could you please help to check the Comment 10? Most likely, the pipe isn't flushed, or no process is reading it before qemu. Therefore, you get garbage in the serial input and can trigger firmware reboot. I think we keep that behaviour for historical reasons, and systems that do support bidirectional pipes: https://lists.nongnu.org/archive/html/qemu-devel/2006-12/msg00160.html I sent an RFC patch to deprecate usage of old syntax that can only work with non-portable bidirectional pipes: https://patchew.org/QEMU/20220726083232.2567756-1-marcandre.lureau@redhat.com/ (In reply to Marc-Andre Lureau from comment #12) > (In reply to liunana from comment #11) > > > Fwiw, I do not observe that behaviour with a RHEL8 VM. Did you change the > > > kernel arguments or guest configuration to interact with serial? > > > > No, no special configuration for guest. > > And I still can reproduce this with a RHEL8.7 VM. And I upload the video to > > the attachments, could you please help to check the Comment 10? > > Most likely, the pipe isn't flushed, or no process is reading it before > qemu. Therefore, you get garbage in the serial input and can trigger > firmware reboot. > > I think we keep that behaviour for historical reasons, and systems that do > support bidirectional pipes: > https://lists.nongnu.org/archive/html/qemu-devel/2006-12/msg00160.html > > I sent an RFC patch to deprecate usage of old syntax that can only work with > non-portable bidirectional pipes: > https://patchew.org/QEMU/20220726083232.2567756-1-marcandre.lureau@redhat. > com/ Thank you, Marc-Andre. Seems this patch is for RHEL9.2, right? And would you please help to take this bz and change the priority? Thanks a lot! Best regards Nana Liu (In reply to liunana from comment #13) > Seems this patch is for RHEL9.2, right? The patch is a RFC for upstream. The soonest it will get is qemu 7.1 release. > And would you please help to take this bz and change the priority? Let's assign it to me for now, and use low priority. Eventually, we may close this bug as CLOSED/UPSTREAM though, as it's not really something customer can run into with libvirt. Otherwise, we can keep it open until the next QEMU rebase. https://lists.nongnu.org/archive/html/qemu-devel/2022-08/msg00972.html https://lists.nongnu.org/archive/html/qemu-devel/2022-08/msg00974.html Assumption is this will make the qemu-7.2 rebase Closing this as wontfix as there was no consensus in the upstream review of the request. It is not felt this is a customer or production issue, rather it’s more of a developer problem with the usage of pipes. If using pipes, then you should supply both input and output as noted above. The review requested changes to add extra layers / code that eventually would be deprecated in the long run and is not a desired effort. If this gets fixed by upstream some day, then great, but for the purposes of downstream testing, using input and output pipes. |
Created attachment 1896923 [details] boot stuck at boot manager Description of problem: Ovmf guest os can not boot with pipe type serial device Version-Release number of selected component (if applicable): libvirt-8.4.0-2.el9 qemu-kvm-7.0.0-8.el9 How reproducible: 100% Steps to Reproduce: 1. prepare a ovmf guest # virt-install --connect qemu:///system -n vm-bios --hvm --accelerate -r 2048 --vcpus=2 --os-variant rhel8.5 --disk path=/var/lib/libvirt/images/RHEL-9.1.0-20220708.0-x86_64-ovmf.qcow2,bus=virtio,format=qcow2 --network network=default,model=virtio --import --noreboot --noautoconsole --serial pty --debug --memballoon model=virtio --cpu host,disable=vmx --graphics vnc --video virtio --machine q35 --boot uefi 2. attach a pipe serial device # vi /tmp/pipe.xml <serial type='pipe'> <source path='/tmp/avocado_64lsem4o/serial_pipe_TrAM-0.log'/> <target type='isa-serial' port='1'> <model name='isa-serial'/> </target> <alias name='serial1'/> </serial> # virsh attach-device vm-bios pipe.xml --config 3. virsh start vm-bios # virsh start vm-bios Actual results: vm boot stuck at boot manager Expected results: vm can boot success Additional info: Not reproduced for seabios guest. 1.install a seabios vm # virt-install --connect qemu:///system -n vm-seabios --hvm --accelerate -r 2048 --vcpus=2 --os-variant rhel8.5 --disk path=/var/lib/libvirt/images/RHEL-9.1.0-20220708.0-x86_64.qcow2,bus=virtio,format=qcow2 --network network=default,model=virtio --import --noreboot --noautoconsole --serial pty --debug --memballoon model=virtio --cpu host,disable=vmx --graphics vnc --video virtio --machine q35 2. # virsh attach-device vm-seabios /tmp/pipe.xml --config #virsh start vm-seabios 3. # virsh console vm-seabios Connected to domain 'vm-seabios' Escape character is ^] (Ctrl + ]) localhost login: