Bug 1609699 - virtio-vsock - guest kernel panic with ctrl+c after hot-unplug
Summary: virtio-vsock - guest kernel panic with ctrl+c after hot-unplug
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: kernel
Version: 7.6
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Stefano Garzarella
QA Contact: FuXiangChun
URL:
Whiteboard:
Depends On:
Blocks: 1676595 1693996
TreeView+ depends on / blocked
 
Reported: 2018-07-30 08:13 UTC by yafu
Modified: 2019-08-06 12:08 UTC (History)
7 users (show)

Fixed In Version: kernel-3.10.0-1008.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1676595 (view as bug list)
Environment:
Last Closed: 2019-08-06 12:08:16 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2019:2029 None None None 2019-08-06 12:08:38 UTC

Description yafu 2018-07-30 08:13:22 UTC
Description of problem:
Guest os kernel panic when using 'ctrl+c' to break nc-vsock in guest os after hotunplug vsock device.

Version-Release number of selected component (if applicable):
kernel-3.10.0-928.el7.x86_64
libvirt-4.5.0-4.el7.x86_64
qemu-kvm-rhev-2.12.0-8.el7.x86_64

How reproducible:
100%

Steps to reproduce:
1.Add vsock model on host os:
#modprobe vhost_vsock

2.Start a guest with vsock device:
#virsh start rhel7.6
Domain rhel7.6 started

#virsh dumpxml rhel7.6 | grep -A5 vsock
# virsh dumpxml rhel7.6 | grep -A5 vsock
    <vsock model='virtio'>
      <cid auto='no' address='3'/>
      <alias name='ua-04c3388d-4e33-4023-84de-a2205c777asdfdsf'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
    </vsock>


3.Download nc-vosck.c both on guest and host:
#git clone https://github.com/stefanha/nc-vsock.git

4.Add vsock model on guest os:
#modprobe vhost_vsock

5.Start listening socket inside guest:
#./nc-vsock -l 1234

6.Hotunplug the vsock device
#virsh detach-device rhel7.6 vsock.xml

7.Using 'ctrl-c' to break the nc-vsock started in step5 in the gust os

Actual results:
Guest os kernel panic after step 7.

Expected results:
Guest os should work well after step 7.

Additional info:
1.vmcore-dmesg:
[  438.425937] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
[  438.425942] IP: [<ffffffffc092d72d>] __vsock_release+0x2d/0x160 [vsock]
[  438.425947] PGD 0 
[  438.425949] Oops: 0000 [#1] SMP 
[  438.425951] Modules linked in: vmw_vsock_virtio_transport vmw_vsock_virtio_transport_common vsock tcp_lp fuse uinput devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iosf_mbi snd_hda_codec_generic crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep joydev snd_seq pcspkr snd_seq_device snd_pcm sg virtio_balloon parport_pc
[  438.425979]  parport snd_timer snd i2c_piix4 soundcore ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_gpu drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm 8139too virtio_net drm_panel_orientation_quirks virtio_blk virtio_scsi virtio_console crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_piix libata 8139cp mii virtio_pci virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod
[  438.425999] CPU: 0 PID: 3757 Comm: nc-vsock Kdump: loaded Not tainted 3.10.0-928.el7.x86_64 #1
[  438.426001] Hardware name: Red Hat KVM, BIOS 1.11.0-2.el7 04/01/2014
[  438.426003] task: ffff954abdf69040 ti: ffff954aeb4d0000 task.ti: ffff954aeb4d0000
[  438.426004] RIP: 0010:[<ffffffffc092d72d>]  [<ffffffffc092d72d>] __vsock_release+0x2d/0x160 [vsock]
[  438.426007] RSP: 0018:ffff954aeb4d3bd0  EFLAGS: 00010282
[  438.426008] RAX: 0000000000000000 RBX: ffff954af92062e8 RCX: 0000000000000001
[  438.426010] RDX: ffff954a97e43e10 RSI: ffff954a97e43e00 RDI: ffff954af9206140
[  438.426011] RBP: ffff954aeb4d3bf8 R08: 0000000000000000 R09: 0000000000000000
[  438.426012] R10: ffff954ae5502030 R11: ffff954a97e43e10 R12: ffff954af9206140
[  438.426013] R13: ffff954ae5502030 R14: ffff954a89d38e40 R15: ffff954afc08a7a0
[  438.426015] FS:  0000000000000000(0000) GS:ffff954affc00000(0000) knlGS:0000000000000000
[  438.426017] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  438.426018] CR2: 0000000000000010 CR3: 000000000ac10000 CR4: 00000000003606f0
[  438.426022] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  438.426023] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  438.426024] Call Trace:
[  438.426028]  [<ffffffffc092d876>] vsock_release+0x16/0x30 [vsock]
[  438.426032]  [<ffffffff99612765>] sock_release+0x25/0x90
[  438.426035]  [<ffffffff996127e2>] sock_close+0x12/0x20
[  438.426037]  [<ffffffff99241bcc>] __fput+0xec/0x260
[  438.426039]  [<ffffffff99241e2e>] ____fput+0xe/0x10
[  438.426042]  [<ffffffff990bd65b>] task_work_run+0xbb/0xe0
[  438.426045]  [<ffffffff9909ca31>] do_exit+0x2d1/0xa40
[  438.426048]  [<ffffffff992627e4>] ? mntput+0x24/0x40
[  438.426049]  [<ffffffff99241c66>] ? __fput+0x186/0x260
[  438.426052]  [<ffffffff9909d21f>] do_group_exit+0x3f/0xa0
[  438.426065]  [<ffffffff990ae0ee>] get_signal_to_deliver+0x1ce/0x5e0
[  438.426067]  [<ffffffff990c1d90>] ? wake_up_var+0x30/0x30
[  438.426079]  [<ffffffff9902b527>] do_signal+0x57/0x6f0
[  438.426081]  [<ffffffff996130a0>] ? SYSC_accept4+0x1e0/0x230
[  438.426084]  [<ffffffff992fa7a5>] ? sock_has_perm+0x75/0x90
[  438.426086]  [<ffffffff99762f7e>] ? _raw_spin_unlock_bh+0x1e/0x20
[  438.426088]  [<ffffffff99618980>] ? release_sock+0x120/0x170
[  438.426090]  [<ffffffff9902bc32>] do_notify_resume+0x72/0xc0
[  438.426092]  [<ffffffff9976e098>] int_signal+0x12/0x17
[  438.426094] Code: 44 00 00 55 48 85 ff 48 89 e5 41 57 41 56 41 55 41 54 49 89 fc 53 0f 84 17 01 00 00 48 8b 05 3b 3d 00 00 49 8d 9c 24 a8 01 00 00 <48> 8b 40 10 e8 ba 5f a5 d8 31 f6 4c 89 e7 e8 f0 9d ce d8 48 89 
[  438.426123] RIP  [<ffffffffc092d72d>] __vsock_release+0x2d/0x160 [vsock]
[  438.426126]  RSP <ffff954aeb4d3bd0>
[  438.426127] CR2: 0000000000000010

Comment 6 Jan Stancek 2019-02-20 18:16:39 UTC
Patch(es) committed on kernel-3.10.0-1008.el7

Comment 9 FuXiangChun 2019-02-21 06:27:36 UTC
Reproduced bug with 3.10.0-957.5.1.el7.x86_64 from qemu level.

1)On host
#modprobe vsock
#modprobe vhost_vsock

2)Boot RHEL7.6.z guest with vhost-vsock-pci

/usr/libexec/qemu-kvm -name guest=q35-seabios,debug-threads=on -machine pc,accel=kvm,usb=off,vmport=off,dump-guest-core=off,kernel_irqchip=split -cpu Broadwell,vmx=on -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -uuid 34cc0dae-8998-480c-b2db-171ce1e7461a -no-user-config -nodefaults -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device virtio-scsi-pci,id=scsi0 -device virtio-serial-pci,id=virtio-serial0 -drive file=/home/choma/BZ-1677007/rhel7-6-z.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=drive-virtio-disk01,id=virtio-disk0 -device scsi-hd,drive=drive-virtio-disk0,bootindex=1 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:ee:67:31 -spice port=5931,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1 -chardev spicevmc,id=charredir0,name=usbredir -device virtio-balloon-pci,id=balloon0 -msg timestamp=on -vnc :2 -monitor stdio \ 
-device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=3 \

3)Inside guest

#./nc-vsock -l 123456

4)unhotplug vhost-vosck-pci
(qemu) device_del vhost-vsock-pci0

5)press ctrl+c to stop nc-vosck process

result:
Guest os kernel panic like comment0.


Verified bug with the fixed kernel 3.10.0-1008.el7.x86_64.

Guest works well after ctrl+c.

So, move this bug to verified.

Comment 11 errata-xmlrpc 2019-08-06 12:08:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2029


Note You need to log in before you can comment on or make changes to this bug.