Bug 2025750

Summary: must-gather | nft files are not collected for nodes
Product: Container Native Virtualization (CNV) Reporter: ibesso <ibesso>
Component: NetworkingAssignee: oshoval
Status: CLOSED ERRATA QA Contact: Yossi Segev <ysegev>
Severity: unspecified Docs Contact:
Priority: low    
Version: 4.9.1CC: cnv-qe-bugs, mmuench, ocs-bugs, oshoval, phoracek, resoni, sabose, ysegev
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.10.0-103 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-16 15:56:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2048960    
Bug Blocks:    

Description ibesso 2021-11-22 23:30:38 UTC
Description of problem (please be detailed as possible and provide log snippests):
---------------------------------------------------------------------------------
NFT files are not created.



Version of all relevant components (if applicable):
--------------------------------------------------
CNV 4.9.1-23
must-gather 4.9.1-3



Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
--------------------------------------------------
degradation: previously, those files were collected.



Is there any workaround available to the best of your knowledge?
---------------------------------------------------------------
no.



Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
---------------------------------------
1 - same test as before should pass.



Can this issue reproducible?
---------------------------
yes, 100% (by running the automation test)



Can this issue reproduce from the UI?
------------------------------------
reproducible via oc must-gather commands.



If this is a regression, please provide more details to justify this:
--------------------------------------------------------------------
1. test passed before on 4.9.1-23 before https://github.com/kubevirt/must-gather/pull/100/files.
2. 4.8.3-34 - works.



Steps to Reproduce:
------------------
1. run the following command:
oc adm must-gather --image=registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8@sha256:4793d2331f033f734c12bb3d6784e2cf2efdbfde26d99dc5e77ce7c2e544c4c3 --dest-dir=/tmp/pytest/must_gather0

2. check the dest dir from the oc adm command under "nodes" subdir.



Actual results:
--------------
No NFT files created.
Only the *_logs_NetworkManager file was created.



Expected results:
----------------
NFT files should be created, as in previous versions.



Additional info:
---------------
in 4.8.3-34, the content of one of the worker nodes subdir contains files with "nft-" prefix:
-rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins     469 Nov 22 22:36 nft-ip6-filter
-rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins     460 Nov 22 22:36 nft-ip6-mangle
-rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins     668 Nov 22 22:36 nft-ip6-nat
-rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins    2431 Nov 22 22:36 nft-ip-filter
-rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins     524 Nov 22 22:36 nft-ip-mangle
-rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins   79507 Nov 22 22:36 nft-ip-nat
-rw-r--r--. 1 cnv-qe-jenkins cnv-qe-jenkins     403 Nov 22 22:36 nft-ip-raw

Comment 2 Rewant 2021-11-25 09:52:26 UTC
As this is related to CNV, moving it to CNV must-gather.

Comment 3 Israel Pinto 2021-11-28 08:33:14 UTC
Petr, is should on network no?

Comment 4 Petr Horáček 2021-11-29 08:38:13 UTC
Yes. Reassigning.

Comment 5 oshoval 2022-01-24 07:26:53 UTC
With the fix, all the tables will be collected to one file "nftables"
using a simpler command that collects all the tables
please adjust the tests accordingly

Comment 6 oshoval 2022-01-24 09:53:03 UTC
*** Bug 2028458 has been marked as a duplicate of this bug. ***

Comment 7 Yossi Segev 2022-02-01 10:36:45 UTC
Verified on a cluster with
Client Version (oc): 4.10.0-202201281850.p0.g7c299f1.assembly.stream-7c299f1
Server Version: 4.10.0-fc.4
Kubernetes Version: v1.23.0+d30ebbc
CNV: v4.10.0-629
CNV must-gather: v4.10.0-104

1. Find the URL of the CNV must-gather image in CNV CSV:
$ oc get csv -n openshift-cnv kubevirt-hyperconverged-operator.v4.10.0 -oyaml | less

Search for the must-gather image:
registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8@sha256:2bfb6e79f259589623edd293daa32341f778a8a392120ad085e4a4fc58bce251

3. Run must-gather using the CNV image:
$ oc adm must-gather --image=registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8@sha256:2bfb6e79f259589623edd293daa32341f778a8a392120ad085e4a4fc58bce251 --dest_dir=mg-out

4. Verify the summarized nftables file exists for each nodes, and includes the various IP contents.
$ find mg-out/ -name *tables*
mg-out/registry-redhat-io-container-native-virtualization-cnv-must-gather-rhel8-sha256-2bfb6e79f259589623edd293daa32341f778a8a392120ad085e4a4fc58bce251/nodes/ssp-rn-410-30-l7whq-worker-0-7b8dj/nftables
mg-out/registry-redhat-io-container-native-virtualization-cnv-must-gather-rhel8-sha256-2bfb6e79f259589623edd293daa32341f778a8a392120ad085e4a4fc58bce251/nodes/ssp-rn-410-30-l7whq-worker-0-dhh22/nftables
mg-out/registry-redhat-io-container-native-virtualization-cnv-must-gather-rhel8-sha256-2bfb6e79f259589623edd293daa32341f778a8a392120ad085e4a4fc58bce251/nodes/ssp-rn-410-30-l7whq-worker-0-zr4vk/nftables

$ less -N mg-out/registry-redhat-io-container-native-virtualization-cnv-must-gather-rhel8-sha256-2bfb6e79f259589623edd293daa32341f778a8a392120ad085e4a4fc58bce251/nodes/ssp-rn-410-30-l7whq-worker-0-7b8dj/nftables

      1 table ip nat {
      2         chain PREROUTING {
      3                 type nat hook prerouting priority dstnat; policy accept;
      4                 counter packets 255634 bytes 27357422 jump KUBE-SERVICES
      5                 counter packets 55628 bytes 4098753 jump KUBE-PORTALS-CONTAINER
      ...
   2018 table ip6 nat {
   2019         chain PREROUTING {
   2020                 type nat hook prerouting priority dstnat; policy accept;
   2021         }
      ....
   2053 table ip filter {
   2054         chain INPUT {
   2055                 type filter hook input priority filter; policy accept;
   2056                 counter packets 56884316 bytes 38996277306 jump KUBE-NODEPORTS
      ...
   2134 table ip mangle {
   2135         chain PREROUTING {
   2136                 type filter hook prerouting priority mangle; policy accept;
   2164 table ip6 filter {
   2165         chain INPUT {
   2166                 type filter hook input priority filter; policy accept;
      ...
   2186 table ip6 mangle {
   2187         chain PREROUTING {
   2188                 type filter hook prerouting priority mangle; policy accept;
      ...
   2210 table ip raw {
   2211         chain PREROUTING {
   2212                 type filter hook prerouting priority raw; policy accept;
      ...

Comment 12 errata-xmlrpc 2022-03-16 15:56:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Virtualization 4.10.0 Images security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0947