This bug has been migrated to another issue tracking site. It has been closed here and may no longer be being monitored.

If you would like to get updates for this issue, or to participate in it, you may do so at Red Hat Issue Tracker .
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2176919 - INFO: task systemd-udevd:20742 blocked for more than 122 seconds.
Summary: INFO: task systemd-udevd:20742 blocked for more than 122 seconds.
Keywords:
Status: CLOSED MIGRATED
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: iscsi-initiator-utils
Version: 9.3
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: rc
: ---
Assignee: Chris Leech
QA Contact: Martin Hoyer
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-03-09 16:29 UTC by Martin Hoyer
Modified: 2023-09-23 12:58 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-09-23 12:58:44 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker   RHEL-8057 0 None Migrated None 2023-09-23 12:58:42 UTC
Red Hat Issue Tracker RHELPLAN-151289 0 None None None 2023-03-09 16:31:39 UTC

Description Martin Hoyer 2023-03-09 16:29:07 UTC 
Description of problem:
After repeatedly logging in and out to iSCSI target using qedi interfaces, call trace was observed on dmesg:


[ 7906.133082] scsi 1:0:2:1: Direct-Access     LIO-ORG  st86backstore    4.0  PQ: 0 ANSI: 6
[ 7906.143294] scsi 1:0:2:1: alua: supports implicit and explicit TPGS
[ 7906.150292] scsi 1:0:2:1: alua: device naa.60014056e1c42009a0c47b4a3bfc7365 port group 0 rel port 1
[ 7906.160502] sd 1:0:2:1: Attached scsi generic sg3 type 0
[ 7906.160774] sd 1:0:2:1: [sdd] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
[ 7906.175170] sd 1:0:2:1: [sdd] Write Protect is off
[ 7906.180522] sd 1:0:2:1: [sdd] Mode Sense: 43 00 00 08
[ 7906.180918] sd 1:0:2:1: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 7906.191324] sd 1:0:2:1: [sdd] Preferred minimum I/O size 512 bytes
[ 7906.198217] sd 1:0:2:1: [sdd] Optimal transfer size 262144 bytes
[ 7906.214771] sd 1:0:2:1: [sdd] Attached SCSI disk
[ 7911.550804] [0000:02:00.5]:[qedi_offload_work:888]:2: Offload conn TIMEOUT iscsi_cid=1, qedi_ep=0000000002fa705b
[ 7917.139541] scsi 2:0:3:1: Direct-Access     LIO-ORG  st86backstore    4.0  PQ: 0 ANSI: 6
[ 7917.150408] scsi 2:0:3:1: alua: supports implicit and explicit TPGS
[ 7917.157408] scsi 2:0:3:1: alua: device naa.60014056e1c42009a0c47b4a3bfc7365 port group 0 rel port 1
[ 7917.167612] sd 2:0:3:1: Attached scsi generic sg4 type 0
[ 7917.167971] sd 2:0:3:1: [sde] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
[ 7917.182327] sd 2:0:3:1: [sde] Write Protect is off
[ 7917.187677] sd 2:0:3:1: [sde] Mode Sense: 43 00 00 08
[ 7917.188116] sd 2:0:3:1: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 7917.198540] sd 2:0:3:1: [sde] Preferred minimum I/O size 512 bytes
[ 7917.205455] sd 2:0:3:1: [sde] Optimal transfer size 262144 bytes
[ 7917.223530] sd 2:0:3:1: [sde] Attached SCSI disk
[ 7962.361415] device-mapper: multipath: 253:4: Failing path 8:16.
[ 7962.370106] [0000:02:00.5]:[qedi_iscsi_event_cb:129]:2: Recv Unknown Event 12
[ 7962.383875] scsi 1:0:2:1: alua: Detached
[ 7962.388476] [0000:02:00.4]:[qedi_iscsi_event_cb:129]:1: Recv Unknown Event 12
[ 7962.400898] scsi 1:0:1:1: rejecting I/O to dead device
[ 7962.404237] [0000:02:00.5]:[qedi_iscsi_event_cb:129]:2: Recv Unknown Event 12
[ 7962.406673] scsi 1:0:1:1: alua: rtpg failed, result 65536
[ 7962.420240] [0000:02:00.4]:[qedi_iscsi_event_cb:129]:1: Recv Unknown Event 12
[ 7962.420648] naa.60014056e1c42009a0c47b4a3bfc7365: no device found for rtpg
[ 7962.436289] device-mapper: multipath: 253:4: Failing path 8:32.
[ 7962.442933] device-mapper: multipath: 253:4: Failing path 8:64.
[ 7962.449599] I/O error, dev dm-4, sector 41942912 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2
[ 7962.459862] I/O error, dev dm-4, sector 41942912 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
[ 7962.469673] Buffer I/O error on dev dm-4, logical block 5242864, async page read
[ 7963.511808] scsi 2:0:0:1: alua: Detached
[ 7963.516347] device-mapper: multipath: 253:4: Failing path 8:32.
[ 7963.522987] device-mapper: multipath: 253:4: Failing path 8:64.
[ 7963.550810] scsi 1:0:1:1: alua: Detached
[ 7963.555324] device-mapper: multipath: 253:4: Failing path 8:64.
[ 7963.562132] device-mapper: uevent: dm_send_uevents: skipping sending uevent for lost device
[ 7963.576827] scsi 2:0:3:1: alua: Detached
[ 8113.278942] INFO: task systemd-udevd:20742 blocked for more than 122 seconds.
[ 8113.286904]       Not tainted 5.14.0-284.el9.x86_64 #1
[ 8113.292640] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8113.301382] task:systemd-udevd   state:D stack:    0 pid:20742 ppid:   663 flags:0x00004006
[ 8113.310708] Call Trace:
[ 8113.313430]  <TASK>
[ 8113.315762]  __schedule+0x248/0x620
[ 8113.319658]  ? radix_tree_node_ctor+0x50/0x50
[ 8113.324524]  schedule+0x5a/0xc0
[ 8113.328030]  io_schedule+0x42/0x70
[ 8113.331827]  folio_wait_bit_common+0x147/0x380
[ 8113.336789]  ? file_check_and_advance_wb_err+0xd0/0xd0
[ 8113.342527]  truncate_inode_pages_range+0x34e/0x500
[ 8113.347991]  blkdev_flush_mapping+0x58/0xc0
[ 8113.352664]  blkdev_put_whole+0x3a/0x50
[ 8113.356961]  blkdev_put+0x9f/0x1b0
[ 8113.360770]  blkdev_close+0x18/0x20
[ 8113.364681]  __fput+0x91/0x250
[ 8113.368106]  task_work_run+0x59/0x90
[ 8113.372112]  do_exit+0x262/0x4a0
[ 8113.375732]  do_group_exit+0x2d/0x90
[ 8113.379736]  get_signal+0x9d3/0xa00
[ 8113.383647]  ? new_sync_read+0xf9/0x180
[ 8113.387953]  arch_do_signal_or_restart+0x25/0x100
[ 8113.393206]  ? vfs_read+0x13c/0x190
[ 8113.397115]  exit_to_user_mode_loop+0x9c/0x130
[ 8113.402093]  exit_to_user_mode_prepare+0xb6/0x100
[ 8113.407360]  syscall_exit_to_user_mode+0x12/0x30
[ 8113.412531]  do_syscall_64+0x69/0x90
[ 8113.416538]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 8113.422194] RIP: 0033:0x7fbcfe53eaf2
[ 8113.426204] RSP: 002b:00007ffc1c42fae8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 8113.434671] RAX: fffffffffffffffc RBX: 000055ace72ba9a8 RCX: 00007fbcfe53eaf2
[ 8113.442652] RDX: 0000000000040000 RSI: 000055ace72ba9b8 RDI: 000000000000000f
[ 8113.450631] RBP: 000055ace72084e0 R08: 0000000000000003 R09: 0000000000000000
[ 8113.458612] R10: 0000000000040000 R11: 0000000000000246 R12: 000000027ff80000
[ 8113.466591] R13: 0000000000040000 R14: 000055ace72ba990 R15: 000055ace7208530
[ 8113.474590]  </TASK>

[ 8236.159046] INFO: task systemd-udevd:20742 blocked for more than 245 seconds.
[ 8236.167014]       Not tainted 5.14.0-284.el9.x86_64 #1
[ 8236.172744] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8236.181483] task:systemd-udevd   state:D stack:    0 pid:20742 ppid:   663 flags:0x00004006
[ 8236.190809] Call Trace:
[ 8236.193540]  <TASK>
[ 8236.195881]  __schedule+0x248/0x620
[ 8236.199777]  ? radix_tree_node_ctor+0x50/0x50
[ 8236.204641]  schedule+0x5a/0xc0
[ 8236.208146]  io_schedule+0x42/0x70
[ 8236.211942]  folio_wait_bit_common+0x147/0x380
[ 8236.216894]  ? file_check_and_advance_wb_err+0xd0/0xd0
[ 8236.222631]  truncate_inode_pages_range+0x34e/0x500
[ 8236.228083]  blkdev_flush_mapping+0x58/0xc0
[ 8236.232753]  blkdev_put_whole+0x3a/0x50
[ 8236.237034]  blkdev_put+0x9f/0x1b0
[ 8236.240831]  blkdev_close+0x18/0x20
[ 8236.244725]  __fput+0x91/0x250
[ 8236.248127]  task_work_run+0x59/0x90
[ 8236.252118]  do_exit+0x262/0x4a0
[ 8236.255721]  do_group_exit+0x2d/0x90
[ 8236.259704]  get_signal+0x9d3/0xa00
[ 8236.263599]  ? new_sync_read+0xf9/0x180
[ 8236.267881]  arch_do_signal_or_restart+0x25/0x100
[ 8236.273132]  ? vfs_read+0x13c/0x190
[ 8236.277026]  exit_to_user_mode_loop+0x9c/0x130
[ 8236.281990]  exit_to_user_mode_prepare+0xb6/0x100
[ 8236.287242]  syscall_exit_to_user_mode+0x12/0x30
[ 8236.292387]  do_syscall_64+0x69/0x90
[ 8236.296378]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 8236.302016] RIP: 0033:0x7fbcfe53eaf2
[ 8236.306006] RSP: 002b:00007ffc1c42fae8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 8236.314455] RAX: fffffffffffffffc RBX: 000055ace72ba9a8 RCX: 00007fbcfe53eaf2
[ 8236.322420] RDX: 0000000000040000 RSI: 000055ace72ba9b8 RDI: 000000000000000f
[ 8236.330386] RBP: 000055ace72084e0 R08: 0000000000000003 R09: 0000000000000000
[ 8236.338352] R10: 0000000000040000 R11: 0000000000000246 R12: 000000027ff80000
[ 8236.346317] R13: 0000000000040000 R14: 000055ace72ba990 R15: 000055ace7208530
[ 8236.354284]  </TASK>

Version-Release number of selected component (if applicable):
kernel-5.14.0-284.el9.x86_64
device-mapper-multipath-libs-0.8.7-20.el9.x86_64
device-mapper-multipath-0.8.7-20.el9.x86_64
systemd-udev-252-8.el9.x86_64

How reproducible:
once

Additional info:
I wasn't sure what component to open this against, please re-assign as needed
Comment 1 Martin Hoyer 2023-03-09 17:19:59 UTC 
So I suspect it was caused by the iSCSI initiator trying to connect to a target which it had no network path to.

After running
`iscsiadm -m discovery -t st -p 172.16.0.20 -I qedi0`
172.16.1.20:3260,1 iqn.1994-05.com.redhat:stqe-lio
172.16.0.20:3260,1 iqn.1994-05.com.redhat:stqe-lio
`iscsiadm -m discovery -t st -p 172.16.1.20 -I qedi1`
172.16.1.20:3260,1 iqn.1994-05.com.redhat:stqe-lio
172.16.0.20:3260,1 iqn.1994-05.com.redhat:stqe-lio

target will provide two portals for each interface, however the 172.16.1 is not reachable by qedi0 and 172.16.0 is not reachable by qedi1.

So when user instead of specifying a portal and interface simply runs `iscsiadm -m node -l`, timeouts will arise and this call trace can be reproduced.
Comment 2 Ben Marzinski 2023-03-22 16:20:26 UTC 
Reassigning to iscsi-initiator-utils based on comment #1. Assuming that the multipath device was set to queue_if_no_path, this is exactly what you'd expect if you had no paths to your device.  I'm not sure that there's actually a bug here. But if there is, and it further investigation points to multipath, feel free to reassign it back.
Comment 3 liunana 2023-06-27 11:07:26 UTC 
Hi, I reported one similar bug on RHEL8.9 with high reproducibility: Bug 2217866 - [RHEL8.9] INFO: task systemd-udevd:2691 blocked for more than 120 seconds with over 256 vcpus, please check.
Comment 4 shreepat 2023-08-24 15:14:47 UTC 
SNO (SPR-EE) running 4.14.0-ec4 ,with edu workload saw  kernel hang INFO: task kube-apiserver:646656 blocked for more than 122 seconds
[kni@infra609-1 sno]$ oc get nodes
NAME                           STATUS   ROLES                                            
  AGE   VERSION
gnb-spr.flexranspr.cars2.lab   Ready    control-plane,master,worker,worker-cnf,worker-rt   24h   v1.27.3+4aaeaec

[ 4670.588214] device 34149aa9ca3d7b0 entered promiscuous mode
[ 4670.907849] IPv6: ADDRCONF(NETDEV_CHANGE): 0eee13a20bec4f8: link becomes ready        
[ 4670.962075] device 0eee13a20bec4f8 entered promiscuous mode
[ 4671.028931] XFS (dm-2): Mounting V5 Filesystem
[ 4671.060761] XFS (dm-2): Ending clean mount
[ 4916.346441] INFO: task systemd:1 blocked for more than 122 seconds.
[ 4916.346782]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.347188] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.347545] task:systemd         state:D stack:    0 pid:    1 ppid:     0 flags:0x000
00002
[ 4916.347549] Call Trace:
[ 4916.347550]  <TASK>
[ 4916.347552]  __schedule+0x25b/0x660
[ 4916.347557]  schedule+0x5d/0xd0
[ 4916.347559]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.347561]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.347563]  cgroup_kn_lock_live+0x43/0xe0
[ 4916.347565]  cgroup_mkdir+0x32/0x1c0
[ 4916.347566]  kernfs_iop_mkdir+0x54/0x80
[ 4916.347569]  vfs_mkdir+0x155/0x220
[ 4916.347571]  do_mkdirat+0x131/0x160
[ 4916.347573]  __x64_sys_mkdir+0x48/0x70
[ 4916.347574]  do_syscall_64+0x59/0x90
[ 4916.347576]  ? do_syscall_64+0x69/0x90
[ 4916.347578]  ? __context_tracking_enter+0x7c/0x110
[ 4916.347579]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.347580]  ? do_syscall_64+0x69/0x90
[ 4916.347581]  ? __context_tracking_enter+0x7c/0x110
[ 4916.347582]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.347583]  ? do_syscall_64+0x69/0x90
[ 4916.347584]  ? do_syscall_64+0x69/0x90
[ 4916.347585]  ? __context_tracking_enter+0x7c/0x110
[ 4916.347586]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.347588] RIP: 0033:0x7f74cc3367bb
[ 4916.347590] RSP: 002b:00007fff07c43c38 EFLAGS: 00000246 ORIG_RAX: 0000000000000053    
[ 4916.347592] RAX: ffffffffffffffda RBX: 0000560818d1edc0 RCX: 00007f74cc3367bb
[ 4916.347592] RDX: 0000000000000000 RSI: 00000000000001ed RDI: 0000560818ee1410
[ 4916.347593] RBP: 00007fff07c43c70 R08: 000000000000007f R09: 0000000000000000
[ 4916.347594] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[ 4916.347595] R13: 00000000000000f3 R14: 00007f74cc79d09f R15: 00000000000000f3
[ 4916.347596]  </TASK>
[ 4916.347874] INFO: task systemd-journal:1721 blocked for more than 122 seconds.
[ 4916.348214]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.348562] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.349080] task:systemd-journal state:D stack:    0 pid: 1721 ppid:     1 flags:0x000
00002
[ 4916.349082] Call Trace:
[ 4916.349083]  <TASK>
[ 4916.349084]  __schedule+0x25b/0x660
[ 4916.349093]  schedule+0x5d/0xd0
[ 4916.349094]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.349096]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.349097]  proc_cgroup_show+0x48/0x410
[ 4916.349099]  proc_single_show+0x4a/0xc0
[ 4916.349102]  seq_read_iter+0x11d/0x4b0
[ 4916.349104]  ? rt_spin_unlock+0x13/0x40
[ 4916.349105]  seq_read+0xe6/0x130
[ 4916.349107]  vfs_read+0x94/0x190
[ 4916.349110]  ksys_read+0x62/0xe0
[ 4916.349111]  do_syscall_64+0x59/0x90
[ 4916.349113]  ? __context_tracking_enter+0x7c/0x110
[ 4916.349114]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.349115]  ? do_syscall_64+0x69/0x90
[ 4916.349117]  ? __context_tracking_enter+0x7c/0x110
[ 4916.349118]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.349118]  ? do_syscall_64+0x69/0x90
[ 4916.349120]  ? do_syscall_64+0x69/0x90
[ 4916.349121]  ? __context_tracking_enter+0x7c/0x110
[ 4916.349122]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.349123] RIP: 0033:0x7f694e751b2c
[ 4916.349124] RSP: 002b:00007ffe10256d20 EFLAGS: 00000246 ORIG_RAX: 0000000000000000    
[ 4916.349125] RAX: ffffffffffffffda RBX: 000055e1759eb260 RCX: 00007f694e751b2c
[ 4916.349126] RDX: 0000000000000400 RSI: 000055e1759ec590 RDI: 0000000000000014
[ 4916.349126] RBP: 00007f694e80a5e0 R08: 0000000000000000 R09: 0000000000000001
[ 4916.349127] R10: 0000000000001000 R11: 0000000000000246 R12: 00007f694db1e320
[ 4916.349128] R13: 0000000000000d68 R14: 00007f694e8099e0 R15: 0000000000000d68
[ 4916.349129]  </TASK>
[ 4916.349135] INFO: task ovs-vswitchd:4298 blocked for more than 122 seconds.
[ 4916.349499]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.350046] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.350442] task:ovs-vswitchd    state:D stack:    0 pid: 4298 ppid:     1 flags:0x004
00002
[ 4916.350444] Call Trace:
[ 4916.350444]  <TASK>
[ 4916.350445]  __schedule+0x25b/0x660
[ 4916.350448]  schedule+0x5d/0xd0
[ 4916.350449]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.350451]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.350452]  rtnetlink_rcv_msg+0xf7/0x390
[ 4916.350455]  ? rtnl_calcit.isra.0+0x140/0x140
[ 4916.350457]  netlink_rcv_skb+0x4e/0x100
[ 4916.350459]  netlink_unicast+0x245/0x360
[ 4916.350461]  netlink_sendmsg+0x238/0x490
[ 4916.350463]  sock_sendmsg+0x5f/0x70
[ 4916.350465]  ____sys_sendmsg+0x230/0x270
[ 4916.350466]  ? import_iovec+0x17/0x20
[ 4916.350468]  ? sendmsg_copy_msghdr+0x7d/0xa0
[ 4916.350470]  ? path_lookupat+0xa2/0x1c0
[ 4916.350471]  ___sys_sendmsg+0x81/0xc0
[ 4916.350473]  ? update_load_avg+0x80/0x7e0
[ 4916.350476]  ? update_load_avg+0x80/0x7e0
[ 4916.350478]  ? __perf_event_task_sched_in+0x69/0x2f0
[ 4916.350480]  ? __fget_light+0x9f/0x130
[ 4916.350482]  __sys_sendmsg+0x59/0xa0
[ 4916.350484]  do_syscall_64+0x59/0x90
[ 4916.350486]  ? __context_tracking_enter+0x7c/0x110
[ 4916.350487]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.350489] RIP: 0033:0x7f2caacd7a2d
[ 4916.350489] RSP: 002b:00007fff11216f20 EFLAGS: 00000293 ORIG_RAX: 000000000000002e    
[ 4916.350490] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2caacd7a2d
[ 4916.350491] RDX: 0000000000000000 RSI: 00007fff11216fb0 RDI: 0000000000000012
[ 4916.350492] RBP: 00007fff11217ce8 R08: 0000000000000000 R09: 0000000000000034
[ 4916.350492] R10: 00007f2caad82c80 R11: 0000000000000293 R12: 000055780ba7a790
[ 4916.350495] R13: 0000000000000001 R14: 000000000006ac97 R15: 00007fff11216fb0
[ 4916.350497]  </TASK>
[ 4916.350498] INFO: task NetworkManager:4530 blocked for more than 122 seconds.
[ 4916.350895]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.351217] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.351428] task:NetworkManager  state:D stack:    0 pid: 4530 ppid:     1 flags:0x000
00002
[ 4916.351430] Call Trace:
[ 4916.351430]  <TASK>
[ 4916.351431]  __schedule+0x25b/0x660
[ 4916.351433]  schedule+0x5d/0xd0
[ 4916.351435]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.351436]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.351437]  rtnetlink_rcv_msg+0xf7/0x390
[ 4916.351439]  ? rtnl_calcit.isra.0+0x140/0x140
[ 4916.351440]  netlink_rcv_skb+0x4e/0x100
[ 4916.351442]  netlink_unicast+0x245/0x360
[ 4916.351444]  netlink_sendmsg+0x238/0x490
[ 4916.351445]  sock_sendmsg+0x5f/0x70
[ 4916.351447]  ____sys_sendmsg+0x230/0x270
[ 4916.351448]  ? import_iovec+0x17/0x20
[ 4916.351449]  ? sendmsg_copy_msghdr+0x7d/0xa0
[ 4916.351451]  ___sys_sendmsg+0x81/0xc0
[ 4916.351452]  ? reweight_entity+0x14d/0x160
[ 4916.351455]  ? __perf_event_task_sched_in+0x69/0x2f0
[ 4916.351457]  ? __tick_nohz_task_switch+0x22/0x80
[ 4916.351460]  ? raw_spin_rq_unlock+0x18/0x70
[ 4916.351461]  ? finish_task_switch.isra.0+0x98/0x2f0
[ 4916.351461]  ? __fget_light+0x9f/0x130
[ 4916.351463]  __sys_sendmsg+0x59/0xa0
[ 4916.351465]  do_syscall_64+0x59/0x90
[ 4916.351466]  ? __context_tracking_enter+0x7c/0x110
[ 4916.351468]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.351469] RIP: 0033:0x7fb2e11cca2d
[ 4916.351470] RSP: 002b:00007ffc0a694020 EFLAGS: 00000293 ORIG_RAX: 000000000000002e    
[ 4916.351471] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fb2e11cca2d
[ 4916.351471] RDX: 0000000000000000 RSI: 00007ffc0a694060 RDI: 000000000000000d
[ 4916.351472] RBP: 0000559c3fc14040 R08: 0000000000000000 R09: 0000000000000000
[ 4916.351472] R10: 0000000000000100 R11: 0000000000000293 R12: 0000000000000000
[ 4916.351473] R13: 0000000000000000 R14: 0000000000000000 R15: 0000559c3fbfa850
[ 4916.351474]  </TASK>
[ 4916.351475] INFO: task agetty:4598 blocked for more than 122 seconds.
[ 4916.351692]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.351903] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.353746] task:agetty          state:D stack:    0 pid: 4598 ppid:     1 flags:0x000
00002
[ 4916.353748] Call Trace:
[ 4916.353748]  <TASK>
[ 4916.353749]  __schedule+0x25b/0x660
[ 4916.353751]  schedule+0x5d/0xd0
[ 4916.353753]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.353754]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.353756]  __netlink_dump_start+0xc4/0x2e0
[ 4916.353758]  ? rtnl_fill_ifinfo+0xfb0/0xfb0
[ 4916.353759]  rtnetlink_rcv_msg+0x287/0x390
[ 4916.353761]  ? rtnl_fill_ifinfo+0xfb0/0xfb0
[ 4916.353762]  ? rtnl_calcit.isra.0+0x140/0x140
[ 4916.353763]  netlink_rcv_skb+0x4e/0x100
[ 4916.353765]  netlink_unicast+0x245/0x360
[ 4916.353766]  netlink_sendmsg+0x238/0x490
[ 4916.353768]  sock_sendmsg+0x5f/0x70
[ 4916.353770]  __sys_sendto+0xf0/0x160
[ 4916.353772]  ? __context_tracking_enter+0x7c/0x110
[ 4916.353773]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.353774]  ? do_syscall_64+0x69/0x90
[ 4916.353775]  ? __context_tracking_enter+0x7c/0x110
[ 4916.353776]  __x64_sys_sendto+0x20/0x30
[ 4916.353778]  do_syscall_64+0x59/0x90
[ 4916.353779]  ? __context_tracking_enter+0x7c/0x110
[ 4916.353780]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.353782] RIP: 0033:0x7fb099880a9a
[ 4916.353782] RSP: 002b:00007ffc0ec88a68 EFLAGS: 00000246 ORIG_RAX: 000000000000002c    
[ 4916.353783] RAX: ffffffffffffffda RBX: 00007ffc0ec89bc0 RCX: 00007fb099880a9a
[ 4916.353784] RDX: 0000000000000014 RSI: 00007ffc0ec89b00 RDI: 0000000000000007
[ 4916.353784] RBP: 00007ffc0ec89b50 R08: 00007ffc0ec89ac0 R09: 000000000000000c
[ 4916.353785] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffc0ec89ac0
[ 4916.353786] R13: 00007ffc0ec89b00 R14: 0000000000000002 R15: 000055c8c27a1360
[ 4916.353787]  </TASK>
[ 4916.353839] INFO: task crio:9946 blocked for more than 122 seconds.
[ 4916.354069]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.354300] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.354518] task:crio            state:D stack:    0 pid: 9946 ppid:     1 flags:0x000
00002
[ 4916.354520] Call Trace:
[ 4916.354520]  <TASK>
[ 4916.354521]  __schedule+0x25b/0x660
[ 4916.354523]  schedule+0x5d/0xd0
[ 4916.354525]  rt_mutex_slowlock_block.constprop.0+0x30/0x150
[ 4916.354526]  __rt_mutex_slowlock.constprop.0+0x6f/0xf0
[ 4916.354527]  ? migrate_enable+0xd5/0x150
[ 4916.354529]  __rwbase_read_lock+0x65/0xd0
[ 4916.354530]  walk_component+0x149/0x1d0
[ 4916.354532]  link_path_walk.part.0.constprop.0+0x25e/0x3c0
[ 4916.354533]  ? path_init+0x326/0x4d0
[ 4916.354534]  path_parentat+0x3c/0x90
[ 4916.354535]  __filename_parentat+0xd8/0x1e0
[ 4916.354537]  do_rmdir+0x45/0x1a0
[ 4916.354539]  ? getname_flags.part.0+0x48/0x1b0
[ 4916.354540]  __x64_sys_unlinkat+0x4a/0x60
[ 4916.354541]  do_syscall_64+0x59/0x90
[ 4916.354543]  ? __context_tracking_enter+0x7c/0x110
[ 4916.354544]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.354544]  ? do_syscall_64+0x69/0x90
[ 4916.354546]  ? __context_tracking_enter+0x7c/0x110
[ 4916.354547]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.355111]  ? do_syscall_64+0x69/0x90
[ 4916.355113]  ? do_syscall_64+0x69/0x90
[ 4916.355114]  ? __context_tracking_enter+0x7c/0x110
[ 4916.355116]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.355117] RIP: 0033:0x55778f76594e
[ 4916.355118] RSP: 002b:000000c001ca7328 EFLAGS: 00000206 ORIG_RAX: 0000000000000107    
[ 4916.355119] RAX: ffffffffffffffda RBX: ffffffffffffff9c RCX: 000055778f76594e
[ 4916.355120] RDX: 0000000000000200 RSI: 000000c0046de4d0 RDI: ffffffffffffff9c
[ 4916.355120] RBP: 000000c001ca7368 R08: 0000000000000000 R09: 0000000000000000
[ 4916.355121] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
[ 4916.355121] R13: 0000000000000008 R14: 000000c000e2e340 R15: 000000c001780c00
[ 4916.355123]  </TASK>
[ 4916.355139] INFO: task kubelet:580750 blocked for more than 122 seconds.
[ 4916.355374]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.355614] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.355863] task:kubelet         state:D stack:    0 pid:580750 ppid:     1 flags:0x00
004002
[ 4916.355864] Call Trace:
[ 4916.355865]  <TASK>
[ 4916.355866]  __schedule+0x25b/0x660
[ 4916.355868]  schedule+0x5d/0xd0
[ 4916.355870]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.355871]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.355872]  cgrp_css_online+0x38/0x100
[ 4916.355874]  online_css+0x19/0x60
[ 4916.355877]  cgroup_apply_control_enable+0x219/0x320
[ 4916.355879]  cgroup_mkdir+0xdf/0x1c0
[ 4916.355881]  kernfs_iop_mkdir+0x54/0x80
[ 4916.355883]  vfs_mkdir+0x155/0x220
[ 4916.355884]  do_mkdirat+0x131/0x160
[ 4916.355885]  __x64_sys_mkdirat+0x4a/0x70
[ 4916.355886]  do_syscall_64+0x59/0x90
[ 4916.355888]  ? __context_tracking_enter+0x7c/0x110
[ 4916.355889]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.355890]  ? do_syscall_64+0x69/0x90
[ 4916.355891]  ? do_syscall_64+0x69/0x90
[ 4916.355893]  ? __context_tracking_enter+0x7c/0x110
[ 4916.355893]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.355894]  ? do_syscall_64+0x69/0x90
[ 4916.355896]  ? __context_tracking_enter+0x7c/0x110
[ 4916.355896]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.355897]  ? do_syscall_64+0x69/0x90
[ 4916.355898]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.355899]  ? do_syscall_64+0x69/0x90
[ 4916.355900]  ? do_syscall_64+0x69/0x90
[ 4916.355902]  ? __context_tracking_enter+0x7c/0x110
[ 4916.355902]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.355904] RIP: 0033:0x408f6e
[ 4916.355905] RSP: 002b:000000c00cd28740 EFLAGS: 00000206 ORIG_RAX: 0000000000000102    
[ 4916.355906] RAX: ffffffffffffffda RBX: ffffffffffffff9c RCX: 0000000000408f6e
[ 4916.355906] RDX: 00000000000001ed RSI: 000000c003562cb0 RDI: ffffffffffffff9c
[ 4916.355907] RBP: 000000c00cd28780 R08: 0000000000000000 R09: 0000000000000000
[ 4916.355907] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
[ 4916.355908] R13: 000000000000001e R14: 000000c004539ba0 R15: 000000c0069cf400
[ 4916.355909]  </TASK>
[ 4916.355918] INFO: task kubelet:1398217 blocked for more than 122 seconds.
[ 4916.358195]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.358463] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.358730] task:kubelet         state:D stack:    0 pid:1398217 ppid:     1 flags:0x0
0004002
[ 4916.358732] Call Trace:
[ 4916.358732]  <TASK>
[ 4916.358733]  __schedule+0x25b/0x660
[ 4916.358736]  schedule+0x5d/0xd0
[ 4916.358737]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.358739]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.358740]  __netlink_dump_start+0xc4/0x2e0
[ 4916.358742]  ? validate_linkmsg+0x110/0x110
[ 4916.358744]  rtnetlink_rcv_msg+0x287/0x390
[ 4916.358745]  ? validate_linkmsg+0x110/0x110
[ 4916.358746]  ? rtnl_calcit.isra.0+0x140/0x140
[ 4916.358748]  netlink_rcv_skb+0x4e/0x100
[ 4916.358749]  netlink_unicast+0x245/0x360
[ 4916.358751]  netlink_sendmsg+0x238/0x490
[ 4916.358752]  sock_sendmsg+0x5f/0x70
[ 4916.358754]  __sys_sendto+0xf0/0x160
[ 4916.358756]  ? netlink_bind+0x2f9/0x350
[ 4916.358758]  __x64_sys_sendto+0x20/0x30
[ 4916.358759]  do_syscall_64+0x59/0x90
[ 4916.358761]  ? __context_tracking_enter+0x7c/0x110
[ 4916.358762]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.358763]  ? do_syscall_64+0x69/0x90
[ 4916.358764]  ? __context_tracking_enter+0x7c/0x110
[ 4916.358765]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.358766]  ? do_syscall_64+0x69/0x90
[ 4916.358767]  ? __context_tracking_enter+0x7c/0x110
[ 4916.358768]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.358769] RIP: 0033:0x408f6e
[ 4916.358770] RSP: 002b:000000c00943b588 EFLAGS: 00000206 ORIG_RAX: 000000000000002c    
[ 4916.358771] RAX: ffffffffffffffda RBX: 000000000000002e RCX: 0000000000408f6e
[ 4916.358771] RDX: 0000000000000011 RSI: 000000c005b5eae0 RDI: 000000000000002e
[ 4916.358772] RBP: 000000c00943b5c8 R08: 000000c005b5e9fc R09: 000000000000000c
[ 4916.358772] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000001
[ 4916.358773] R13: 0000000000000075 R14: 000000c0000dbd40 R15: 000000c002878c00
[ 4916.358774]  </TASK>
[ 4916.358776] INFO: task kubelet:1398222 blocked for more than 122 seconds.
[ 4916.359050]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.359349] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.359645] task:kubelet         state:D stack:    0 pid:1398222 ppid:     1 flags:0x0
0000002
[ 4916.359647] Call Trace:
[ 4916.359647]  <TASK>
[ 4916.359648]  __schedule+0x25b/0x660
[ 4916.359651]  schedule+0x5d/0xd0
[ 4916.359652]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.359654]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.359655]  __netlink_dump_start+0xc4/0x2e0
[ 4916.359657]  ? validate_linkmsg+0x110/0x110
[ 4916.359658]  rtnetlink_rcv_msg+0x287/0x390
[ 4916.359659]  ? validate_linkmsg+0x110/0x110
[ 4916.359660]  ? rtnl_calcit.isra.0+0x140/0x140
[ 4916.359662]  netlink_rcv_skb+0x4e/0x100
[ 4916.359663]  netlink_unicast+0x245/0x360
[ 4916.359665]  netlink_sendmsg+0x238/0x490
[ 4916.359666]  sock_sendmsg+0x5f/0x70
[ 4916.359668]  __sys_sendto+0xf0/0x160
[ 4916.359669]  ? rt_spin_unlock+0x13/0x40
[ 4916.359670]  ? __handle_mm_fault+0x1ad/0x240
[ 4916.359673]  __x64_sys_sendto+0x20/0x30
[ 4916.359674]  do_syscall_64+0x59/0x90
[ 4916.359676]  ? handle_mm_fault+0xc5/0x2a0
[ 4916.359677]  ? do_user_addr_fault+0x1ea/0x6a0
[ 4916.359680]  ? __context_tracking_enter+0x7c/0x110
[ 4916.359681]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.359682] RIP: 0033:0x7fc343a06af6
[ 4916.359683] RSP: 002b:00007fc26cfbf580 EFLAGS: 00000293 ORIG_RAX: 000000000000002c    
[ 4916.359684] RAX: ffffffffffffffda RBX: 00007fc26cfc06b0 RCX: 00007fc343a06af6
[ 4916.359684] RDX: 0000000000000014 RSI: 00007fc26cfc06b0 RDI: 000000000000002d
[ 4916.359685] RBP: 00007fc26cfc0700 R08: 00007fc26cfc0654 R09: 000000000000000c
[ 4916.359686] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000000
[ 4916.359686] R13: 0000000000001b8f R14: 000000000000002d R15: 0000000000000002
[ 4916.359687]  </TASK>
[ 4916.359707] INFO: task kube-apiserver:646656 blocked for more than 122 seconds.       
[ 4916.360017]       Tainted: G        W        --------  ---  5.14.0-284.23.1.rt14.308.e
l9_2.x86_64 #1
[ 4916.362360] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
[ 4916.362678] task:kube-apiserver  state:D stack:    0 pid:646656 ppid:  7600 flags:0x00
000000
[ 4916.362680] Call Trace:
[ 4916.362681]  <TASK>
[ 4916.362681]  __schedule+0x25b/0x660
[ 4916.362684]  schedule+0x5d/0xd0
[ 4916.362685]  rt_mutex_slowlock_block.constprop.0+0x33/0x170
[ 4916.362687]  rt_mutex_slowlock.constprop.0+0xd9/0x150
[ 4916.362688]  __netlink_dump_start+0xc4/0x2e0
[ 4916.362690]  ? validate_linkmsg+0x110/0x110
[ 4916.362691]  rtnetlink_rcv_msg+0x287/0x390
[ 4916.362692]  ? validate_linkmsg+0x110/0x110
[ 4916.362694]  ? rtnl_calcit.isra.0+0x140/0x140
[ 4916.362695]  netlink_rcv_skb+0x4e/0x100
[ 4916.362696]  netlink_unicast+0x245/0x360
[ 4916.362698]  netlink_sendmsg+0x238/0x490
[ 4916.362699]  sock_sendmsg+0x5f/0x70
[ 4916.362701]  __sys_sendto+0xf0/0x160
[ 4916.362702]  ? _copy_to_user+0x21/0x30
[ 4916.362704]  ? __sys_getsockname+0xb9/0xc0
[ 4916.362706]  __x64_sys_sendto+0x20/0x30
[ 4916.362707]  do_syscall_64+0x59/0x90
[ 4916.362709]  ? syscall_exit_to_user_mode+0x2d/0x30
[ 4916.362710]  ? do_syscall_64+0x69/0x90
[ 4916.362711]  ? do_syscall_64+0x69/0x90
[ 4916.362712]  ? __context_tracking_enter+0x7c/0x110
[ 4916.362713]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 4916.362715] RIP: 0033:0x7f5edef2a40a
[ 4916.362715] RSP: 002b:00007f5e88b626a0 EFLAGS: 00000293 ORIG_RAX: 000000000000002c    
[ 4916.362716] RAX: ffffffffffffffda RBX: 00000000000001ba RCX: 00007f5edef2a40a
[ 4916.362717] RDX: 0000000000000014 RSI: 00007f5e88b637d0 RDI: 00000000000001ba
[ 4916.362717] RBP: 0000000000000000 R08: 00007f5e88b63774 R09: 000000000000000c
[ 4916.362718] R10: 0000000000000000 R11: 0000000000000293 R12: 00007f5e88b637d0
[ 4916.362718] R13: 0000000000000014 R14: 0000000000000000 R15: 00007f5e88b63774
[ 4916.362720]  </TASK>
[ 6634.296763] usb 1-10.2: USB disconnect, device number 4
[ 6723.300666] usb 1-10.2: new high-speed USB device number 7 using xhci_hcd
[ 6723.390710] usb 1-10.2: New USB device found, idVendor=413c, idProduct=0006, bcdDevice
= 0.00
[ 6723.390725] usb 1-10.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 6723.390739] usb 1-10.2: Product: DRAC 5 Virtual Keyboard and Mouse
[ 6723.390741] usb 1-10.2: Manufacturer: DELL
[ 6723.390745] usb 1-10.2: SerialNumber: DELL413C-1
[ 6723.402387] input: DELL DRAC 5 Virtual Keyboard and Mouse as /devices/pci0000:00/0000:
00:14.0/usb1/1-10/1-10.2/1-10.2:1.0/0003:413C:0006.0003/input/input5
[ 6723.402701] hid-generic 0003:413C:0006.0003: input,hidraw0: USB HID v1.01 Mouse [DELL 
DRAC 5 Virtual Keyboard and Mouse] on usb-0000:00:14.0-10.2/input0
[ 6723.409390] input: DELL DRAC 5 Virtual Keyboard and Mouse as /devices/pci0000:00/0000:
00:14.0/usb1/1-10/1-10.2/1-10.2:1.1/0003:413C:0006.0004/input/input6
[ 6723.461636] hid-generic 0003:413C:0006.0004: input,hidraw1: USB HID v1.01 Keyboard [DE
LL DRAC 5 Virtual Keyboard and Mouse] on usb-0000:00:14.0-10.2/input1
[ 9334.328770] usb 1-10.2: USB disconnect, device number 7
[ 9354.212480] usb 1-10.2: new high-speed USB device number 8 using xhci_hcd
[ 9354.302576] usb 1-10.2: New USB device found, idVendor=413c, idProduct=0006, bcdDevice
= 0.00
[ 9354.302588] usb 1-10.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3      
[ 9354.302614] usb 1-10.2: Product: DRAC 5 Virtual Keyboard and Mouse
[ 9354.302617] usb 1-10.2: Manufacturer: DELL
[ 9354.302619] usb 1-10.2: SerialNumber: DELL413C-1
[ 9354.314265] input: DELL DRAC 5 Virtual Keyboard and Mouse as /devices/pci0000:00/0000:
00:14.0/usb1/1-10/1-10.2/1-10.2:1.0/0003:413C:0006.0005/input/input7
[ 9354.314571] hid-generic 0003:413C:0006.0005: input,hidraw0: USB HID v1.01 Mouse [DELL 
DRAC 5 Virtual Keyboard and Mouse] on usb-0000:00:14.0-10.2/input0
[ 9354.321259] input: DELL DRAC 5 Virtual Keyboard and Mouse as /devices/pci0000:00/0000:
00:14.0/usb1/1-10/1-10.2/1-10.2:1.1/0003:413C:0006.0006/input/input8
[ 9354.372833] hid-generic 0003:413C:0006.0006: input,hidraw1: USB HID v1.01 Keyboard [DE
LL DRAC 5 Virtual Keyboard and Mouse] on usb-0000:00:14.0-10.2/input1
[12991.288759] usb 1-10.2: USB disconnect, device number 8
[32398.910543] device 0eee13a20bec4f8 left promiscuous mode
[32443.891968] device e2162d3acbf0cf0 left promiscuous mode
[32443.972593] device 34149aa9ca3d7b0 left promiscuous mode
[32453.559797] device 99332fb09ac65ae left promiscuous mode
[32453.681492] device 89174a4dada52e8 left promiscuous mode
[32453.808700] device 2a397660c50a5a7 left promiscuous mode
[32454.933726] device 67657252ff6ae32 left promiscuous mode
[32455.046123] device 3e6c06cec102fa0 left promiscuous mode
[32455.046977] device 9ede242a8fcbfc1 left promiscuous mode
[32455.096435] device 154513e3d456b28 left promiscuous mode
[32455.610263] device 9372cffc28bb47a left promiscuous mode
[32468.446918] device 847f28ac216d565 left promiscuous mode
[32480.711882] XFS (dm-2): Unmounting Filesystem
[32480.716358] XFS (dm-1): Unmounting Filesystem
[32482.116038] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[32482.116074] IPv6: ADDRCONF(NETDEV_CHANGE): 38a653c61dc3407: link becomes ready        
[32482.128813] IPv6: ADDRCONF(NETDEV_CHANGE): 0a410f8c79e3eb0: link becomes ready        
[32482.136820] device 38a653c61dc3407 entered promiscuous mode
[32482.217241] device 0a410f8c79e3eb0 entered promiscuous mode
[32482.601494] IPv6: ADDRCONF(NETDEV_CHANGE): 44481d2f8b63e09: link becomes ready        
[32482.623784] device 44481d2f8b63e09 entered promiscuous mode
[32482.647258] IPv6: ADDRCONF(NETDEV_CHANGE): f94efc262c9c9db: link becomes ready        
[32482.685266] device f94efc262c9c9db entered promiscuous mode
[32485.232916] device f94efc262c9c9db left promiscuous mode
[32485.285983] device 44481d2f8b63e09 left promiscuous mode
[32496.622850] device 0a410f8c79e3eb0 left promiscuous mode
[32496.694570] device 38a653c61dc3407 left promiscuous mode
[32518.775025] device 75130156e7bc8f6 left promiscuous mode
[32653.846555] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[32653.846587] IPv6: ADDRCONF(NETDEV_CHANGE): 73652cbfb5eb51d: link becomes ready        
[32653.863406] device 73652cbfb5eb51d entered promiscuous mode
[32653.866796] IPv6: ADDRCONF(NETDEV_CHANGE): 40766b049955fea: link becomes ready        
[32653.869921] IPv6: ADDRCONF(NETDEV_CHANGE): 1b69c8448e1ecfb: link becomes ready        
[32653.912908] IPv6: ADDRCONF(NETDEV_CHANGE): a01ed3a7ed98f7e: link becomes ready        
[32653.936512] device a01ed3a7ed98f7e entered promiscuous mode
[32653.936725] device 40766b049955fea entered promiscuous mode
[32653.938687] device 1b69c8448e1ecfb entered promiscuous mode
[32655.826354] device 1b69c8448e1ecfb left promiscuous mode
[32666.026716] device a01ed3a7ed98f7e left promiscuous mode
[32666.089269] device 73652cbfb5eb51d left promiscuous mode
[32839.610158] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[32839.610202] IPv6: ADDRCONF(NETDEV_CHANGE): 2f752032fc9bfd0: link becomes ready        
[32839.628661] device 2f752032fc9bfd0 entered promiscuous mode
[32840.344168] device daf36ada4d9b6d1 left promiscuous mode
[33269.246096] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[33269.246134] IPv6: ADDRCONF(NETDEV_CHANGE): c941a4fa7fe53a0: link becomes ready        
[33269.247094] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[33269.247119] IPv6: ADDRCONF(NETDEV_CHANGE): 02717019bb53218: link becomes ready        
[33269.269034] device c941a4fa7fe53a0 entered promiscuous mode
[33269.308788] device 02717019bb53218 entered promiscuous mode
[33281.355467] device c941a4fa7fe53a0 left promiscuous mode
[33282.263291] device 02717019bb53218 left promiscuous mode
[33500.819688] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[33500.819722] IPv6: ADDRCONF(NETDEV_CHANGE): 9ce3364745e118b: link becomes ready        
[33500.837261] device 9ce3364745e118b entered promiscuous mode
[33503.045631] device 9ce3364745e118b left promiscuous mode
[33917.430573] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[33917.430606] IPv6: ADDRCONF(NETDEV_CHANGE): 24c8a28525c9d42: link becomes ready        
[33917.449107] device 24c8a28525c9d42 entered promiscuous mode
[33929.743089] device 24c8a28525c9d42 left promiscuous mode
[33937.137322] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[33937.137359] IPv6: ADDRCONF(NETDEV_CHANGE): cbe253da0f4a4b5: link becomes ready
[33937.159823] device cbe253da0f4a4b5 entered promiscuous mode
[33949.859808] device cbe253da0f4a4b5 left promiscuous mode
[34400.821267] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[34400.821297] IPv6: ADDRCONF(NETDEV_CHANGE): 1fec8d7bdfd0ef7: link becomes ready        
[34400.847991] device 1fec8d7bdfd0ef7 entered promiscuous mode
[34403.307014] device 1fec8d7bdfd0ef7 left promiscuous mode
[34536.689938] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[34536.689969] IPv6: ADDRCONF(NETDEV_CHANGE): 9b25e15dc308adc: link becomes ready        
[34536.709088] device 9b25e15dc308adc entered promiscuous mode
[34537.033976] IPv6: ADDRCONF(NETDEV_CHANGE): 4b7636ef13bdde4: link becomes ready        
[34537.066824] device 4b7636ef13bdde4 entered promiscuous mode
[34549.008466] device 9b25e15dc308adc left promiscuous mode
[34549.925951] device 4b7636ef13bdde4 left promiscuous mode
[35146.654005] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[35146.654045] IPv6: ADDRCONF(NETDEV_CHANGE): c519ee47224db23: link becomes ready        
[35146.672573] device c519ee47224db23 entered promiscuous mode
[35151.602456] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[35151.602483] IPv6: ADDRCONF(NETDEV_CHANGE): 8a5824df13638c0: link becomes ready
[35151.631328] device 8a5824df13638c0 entered promiscuous mode
[35159.211859] device c519ee47224db23 left promiscuous mode
[35164.215974] device 8a5824df13638c0 left promiscuous mode
[35300.810784] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[35300.810822] IPv6: ADDRCONF(NETDEV_CHANGE): 59b8b1cf5514794: link becomes ready        
[35300.829375] device 59b8b1cf5514794 entered promiscuous mode
[35302.527812] device 59b8b1cf5514794 left promiscuous mode
[35747.255356] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[35747.255389] IPv6: ADDRCONF(NETDEV_CHANGE): a090ac5169a5dcf: link becomes ready        
[35747.273011] device a090ac5169a5dcf entered promiscuous mode
[35760.334388] device a090ac5169a5dcf left promiscuous mode
[36200.839159] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[36200.839190] IPv6: ADDRCONF(NETDEV_CHANGE): 06b1bc24a4a2f9b: link becomes ready        
[36200.860618] device 06b1bc24a4a2f9b entered promiscuous mode
[36202.726014] device 06b1bc24a4a2f9b left promiscuous mode
[36351.271186] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[36351.271223] IPv6: ADDRCONF(NETDEV_CHANGE): e08bd42118f1c8e: link becomes ready        
[36351.289307] device e08bd42118f1c8e entered promiscuous mode
[36353.748244] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[36353.748291] IPv6: ADDRCONF(NETDEV_CHANGE): 402e419f741a3e9: link becomes ready        
[36353.766203] device 402e419f741a3e9 entered promiscuous mode
[36363.522758] device e08bd42118f1c8e left promiscuous mode
[36366.510956] device 402e419f741a3e9 left promiscuous mode
[36992.565225] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[36992.565262] IPv6: ADDRCONF(NETDEV_CHANGE): f1289ea18160bab: link becomes ready        
[36992.585853] device f1289ea18160bab entered promiscuous mode
[37005.040271] device f1289ea18160bab left promiscuous mode
[37100.812846] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[37100.812880] IPv6: ADDRCONF(NETDEV_CHANGE): d54ff4c20154a3a: link becomes ready        
[37100.832440] device d54ff4c20154a3a entered promiscuous mode
[37103.213152] device d54ff4c20154a3a left promiscuous mode
[37222.671079] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[37222.671121] IPv6: ADDRCONF(NETDEV_CHANGE): 0503012a975efb0: link becomes ready        
[37222.697916] device 0503012a975efb0 entered promiscuous mode
[37235.823712] device 0503012a975efb0 left promiscuous mode
[37601.750631] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[37601.750667] IPv6: ADDRCONF(NETDEV_CHANGE): c90a59ce3e37ed9: link becomes ready        
[37601.775854] device c90a59ce3e37ed9 entered promiscuous mode
[37614.195442] device c90a59ce3e37ed9 left promiscuous mode
[38000.824823] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[38000.824855] IPv6: ADDRCONF(NETDEV_CHANGE): 62ef967e5dbcbee: link becomes ready        
[38000.846237] device 62ef967e5dbcbee entered promiscuous mode
[38003.357574] device 62ef967e5dbcbee left promiscuous mode
[38013.422137] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[38013.422179] IPv6: ADDRCONF(NETDEV_CHANGE): d46b9f4ab91f24e: link becomes ready        
[38013.450586] device d46b9f4ab91f24e entered promiscuous mode
[38025.587331] device d46b9f4ab91f24e left promiscuous mode
[38244.200543] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[38244.200575] IPv6: ADDRCONF(NETDEV_CHANGE): e2b203bbb218473: link becomes ready        
[38244.221740] device e2b203bbb218473 entered promiscuous mode
[38256.517690] device e2b203bbb218473 left promiscuous mode
[38653.294234] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[38653.294285] IPv6: ADDRCONF(NETDEV_CHANGE): 5a7dc598a1b4909: link becomes ready
Comment 5 RHEL Program Management 2023-09-23 12:56:07 UTC 
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.
Comment 6 RHEL Program Management 2023-09-23 12:58:44 UTC 
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.
Note You need to log in before you can comment on or make changes to this bug.