Description of problem: ---------------------- Following bug 2025750 where the NFT files are missing with 4.9.1, a different result is seen with 4.8.3-58. Only 1 nft file is created per worker node, but they are 0-byte files (all 3 nodes). Version-Release number of selected component (if applicable): ------------------------------------------------------------ 4.8.3-58 How reproducible: ---------------- 100% Steps to Reproduce: ------------------ 1. run the must-gather command: oc adm must-gather --image=registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8@sha256:55de08ec6250b047eef1cebce02483bfcca1a52f908df33cc6a1ad7f38915416 --dest-dir=/tmp/pytest-of-cnv-qe-jenkins/pytest-2/must_gather0 &> /tmp/pytest-of-cnv-qe-jenkins/pytest-2/must_gather0/output.txt 2. check the dest dir from the oc adm command under "nodes" subdir. Actual results: -------------- 1 zero-byte nft file per node bearing the name "nft--": [cnv-qe-jenkins@c01-besso-48-wsbc2-executor nodes]$ pwd /tmp/pytest-of-cnv-qe-jenkins/pytest-current/must_gathercurrent/registry-redhat-io-container-native-virtualization-cnv-must-gather-rhel8-sha256-0d749dd542d0506050e8428b6bfacf60e86396b0f94f80f393b5cfb02ea0302a/nodes [cnv-qe-jenkins@c01-besso-48-wsbc2-executor nodes]$ ll *worker* c01-besso-48-wsbc2-worker-0-4hpc4: total 2196 -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 571931 Dec 2 12:11 audit.log -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 172 Dec 2 12:11 bridge -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1535393 Dec 2 12:11 c01-besso-48-wsbc2-worker-0-4hpc4_logs_NetworkManager -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 156 Dec 2 12:11 dev_vfio -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 87690 Dec 2 12:11 dmesg drwxr-xr-x. 4 cnv-qe-jenkins cnv-qe-jenkins 35 Dec 2 12:12 etc -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 10902 Dec 2 12:11 ip.txt -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 13800 Dec 2 12:11 lspci -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 nft-- -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 opt-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 388 Dec 2 12:11 proc_cmdline -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 sys_sriov_numvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 sys_sriov_totalvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1326 Dec 2 12:11 var-lib-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 3 Dec 2 12:11 vlan c01-besso-48-wsbc2-worker-0-kk7f7: total 2232 -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 595169 Dec 2 12:11 audit.log -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 172 Dec 2 12:11 bridge -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1540900 Dec 2 12:11 c01-besso-48-wsbc2-worker-0-kk7f7_logs_NetworkManager -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 156 Dec 2 12:11 dev_vfio -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 89270 Dec 2 12:11 dmesg drwxr-xr-x. 4 cnv-qe-jenkins cnv-qe-jenkins 35 Dec 2 12:12 etc -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 12768 Dec 2 12:11 ip.txt -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 13800 Dec 2 12:11 lspci -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 nft-- -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 opt-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 388 Dec 2 12:11 proc_cmdline -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 sys_sriov_numvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 sys_sriov_totalvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1326 Dec 2 12:11 var-lib-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 3 Dec 2 12:11 vlan c01-besso-48-wsbc2-worker-0-xrslh: total 2272 -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 617156 Dec 2 12:11 audit.log -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 173 Dec 2 12:11 bridge -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1556447 Dec 2 12:11 c01-besso-48-wsbc2-worker-0-xrslh_logs_NetworkManager -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 156 Dec 2 12:11 dev_vfio -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 97104 Dec 2 12:11 dmesg drwxr-xr-x. 4 cnv-qe-jenkins cnv-qe-jenkins 35 Dec 2 12:12 etc -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 12375 Dec 2 12:11 ip.txt -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 13800 Dec 2 12:11 lspci -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 nft-- -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 opt-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 388 Dec 2 12:11 proc_cmdline -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 sys_sriov_numvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Dec 2 12:11 sys_sriov_totalvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1326 Dec 2 12:11 var-lib-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 3 Dec 2 12:11 vlan [cnv-qe-jenkins@c01-besso-48-wsbc2-executor nodes]$ Expected results: ---------------- same as 4.8.3-34, all nft files should exist. Additional info: --------------- in 4.8.3-34, the content of one of the worker nodes subdir contains files with "nft-" prefix: -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 469 Nov 22 22:36 nft-ip6-filter -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 460 Nov 22 22:36 nft-ip6-mangle -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 668 Nov 22 22:36 nft-ip6-nat -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 2431 Nov 22 22:36 nft-ip-filter -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 524 Nov 22 22:36 nft-ip-mangle -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 79507 Nov 22 22:36 nft-ip-nat -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 403 Nov 22 22:36 nft-ip-raw Additionally, the test automation fails (and passed with 4.8.3-34).
Since the target version is 4.10 this one can be closed please as duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=2025750 working on a fix https://github.com/kubevirt/must-gather/pull/116
*** This bug has been marked as a duplicate of bug 2025750 ***
Reopening this, as this is still not working against 4.9.z and possibly against 4.8.z. bug 2025750 has been addressed and verified against 4.10.0. If this is not planned to be fixed against 4.8.z or 4.9.z then please let us know, so that we can mark the associated automation as invalid. Following is snippet from must-gather collected on a 4.9.3 cluster. =================== /tmp/pytest-d8fa8ce1-7f95-4892-bf82-7329d894143f/must_gather0/registry-redhat-io-container-native-virtualization-cnv-must-gather-rhel8-sha256-2f5de8714f80d1b7f0a55f51b20df8ee28efc685385254b3b00019ccd42135d0/nodes/c01-dbn-493-p4xkv-worker-0-zrplw: total 4788 drwxr-xr-x. 4 cnv-qe-jenkins cnv-qe-jenkins 35 Feb 23 18:37 etc -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 3969457 Feb 23 18:36 c01-dbn-493-p4xkv-worker-0-zrplw_logs_NetworkManager -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 618455 Feb 23 18:36 audit.log -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 13800 Feb 23 18:36 lspci -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 411 Feb 23 18:36 proc_cmdline -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 259629 Feb 23 18:36 dmesg -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 156 Feb 23 18:36 dev_vfio -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1326 Feb 23 18:36 var-lib-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 opt-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 sys_sriov_totalvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 sys_sriov_numvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 nft-- -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 3 Feb 23 18:36 vlan -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 175 Feb 23 18:36 bridge -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 8934 Feb 23 18:36 ip.txt tmp/pytest-d8fa8ce1-7f95-4892-bf82-7329d894143f/must_gather0/registry-redhat-io-container-native-virtualization-cnv-must-gather-rhel8-sha256-2f5de8714f80d1b7f0a55f51b20df8ee28efc685385254b3b00019ccd42135d0/nodes/c01-dbn-493-p4xkv-worker-0-pwxgb: total 4376 drwxr-xr-x. 4 cnv-qe-jenkins cnv-qe-jenkins 35 Feb 23 18:37 etc -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 3715056 Feb 23 18:36 c01-dbn-493-p4xkv-worker-0-pwxgb_logs_NetworkManager -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 616858 Feb 23 18:36 audit.log -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 13800 Feb 23 18:36 lspci -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 411 Feb 23 18:36 proc_cmdline -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 97469 Feb 23 18:36 dmesg -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 156 Feb 23 18:36 dev_vfio -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1326 Feb 23 18:36 var-lib-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 opt-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 sys_sriov_totalvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 sys_sriov_numvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 nft-- -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 3 Feb 23 18:36 vlan -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 174 Feb 23 18:36 bridge -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 10761 Feb 23 18:36 ip.txt /tmp/pytest-d8fa8ce1-7f95-4892-bf82-7329d894143f/must_gather0/registry-redhat-io-container-native-virtualization-cnv-must-gather-rhel8-sha256-2f5de8714f80d1b7f0a55f51b20df8ee28efc685385254b3b00019ccd42135d0/nodes/c01-dbn-493-p4xkv-worker-0-pp6dg: total 4380 drwxr-xr-x. 4 cnv-qe-jenkins cnv-qe-jenkins 35 Feb 23 18:36 etc -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 3718064 Feb 23 18:36 c01-dbn-493-p4xkv-worker-0-pp6dg_logs_NetworkManager -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 616624 Feb 23 18:36 audit.log -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 13800 Feb 23 18:36 lspci -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 411 Feb 23 18:36 proc_cmdline -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 92721 Feb 23 18:36 dmesg -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 156 Feb 23 18:36 dev_vfio -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 1326 Feb 23 18:36 var-lib-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 opt-cni-bin -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 sys_sriov_totalvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 sys_sriov_numvfs -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 0 Feb 23 18:36 nft-- -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 84 Feb 23 18:36 vlan -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 170 Feb 23 18:36 bridge -rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins 13273 Feb 23 18:36 ip.txt
Hi, See please https://bugzilla.redhat.com/show_bug.cgi?id=2028458#c1 We are not planning to fix it, unless it will be mandatory needed Thanks