Bug 2218309
| Summary: | odf must-gather does not collect multus NetworkAttachmentDefinitions | |||
|---|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat OpenShift Data Foundation | Reporter: | Blaine Gardner <brgardne> | |
| Component: | must-gather | Assignee: | yati padia <ypadia> | |
| Status: | CLOSED ERRATA | QA Contact: | Oded <oviner> | |
| Severity: | low | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 4.13 | CC: | etamir, muagarwa, odf-bz-bot, oviner, tdesala, ypadia | |
| Target Milestone: | --- | |||
| Target Release: | ODF 4.14.0 | |||
| Hardware: | All | |||
| OS: | All | |||
| Whiteboard: | ||||
| Fixed In Version: | 4.14.0-132 | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2232482 (view as bug list) | Environment: | ||
| Last Closed: | 2023-11-08 18:52:10 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 2232482 | |||
I see this is merged into the m-g main codebase and is in 4.14. @etamir would you like to target a 4.13.z backport as well? What about 4.12.z for any support exceptions? @ypadia would you be able to take the action to make sure backport BZs get created for 4.13.z and 4.12.z? I'd like to get this off of my todo list to keep focusing on Rook BZs. Thanks :) Yati, please create clones for 4.13/4.12. Reach out to Sunil for adding the backports to the respective z-streams. Oded, please provide qa_ack Yati, please create clones bugs for 4.13/4.12 Bug fixed.
SetUp:
ODF Version: odf-operator.v4.14.0-115.stable
OCP Version: 4.14.0-0.nightly-2023-08-11-055332
Platform: Vsphere
Test Process:
1. Collect mg:
$ oc adm must-gather --image=quay.io/rhceph-dev/ocs-must-gather:latest-4.14
2. Check log content:
$ pwd
/tmp/tmphpy1wpbo_ocs_logs/oviner21-14/ocs_must_gather/quay-io-rhceph-dev-ocs-must-gather-sha256-427ab3d9756b1096dc2afbf4126ac0e1ce2d367b8b302f584d1403129ae84714/namespaces/all
$ ls -l | grep net
-rw-r--r--. 1 oviner oviner 3042 Aug 24 15:39 desc_net_attach_def_all_ns
-rw-r--r--. 1 oviner oviner 1852 Aug 24 15:39 get_yaml_net_attach_def_all_ns
$ cat desc_net_attach_def_all_ns | more
Name: cluster-net
Namespace: default
Labels: <none>
Annotations: <none>
API Version: k8s.cni.cncf.io/v1
Kind: NetworkAttachmentDefinition
Metadata:
Creation Timestamp: 2023-08-24T08:36:26Z
Generation: 1
Managed Fields:
API Version: k8s.cni.cncf.io/v1
Fields Type: FieldsV1
fieldsV1:
f:spec:
.:
f:config:
Manager: kubectl-create
Operation: Update
Time: 2023-08-24T08:36:26Z
Resource Version: 366999
UID: 3799c619-2774-4c2f-a634-a89a37d556f8
Spec:
Config: { "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.30.0/24" } }
Events: <none>
$ cat get_yaml_net_attach_def_all_ns | more
apiVersion: v1
items:
- apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
creationTimestamp: "2023-08-24T08:36:26Z"
generation: 1
name: cluster-net
namespace: default
resourceVersion: "366999"
uid: 3799c619-2774-4c2f-a634-a89a37d556f8
spec:
config: '{ "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode":
"bridge", "ipam": { "type": "whereabouts", "range": "192.168.30.0/24" } }'
- apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
creationTimestamp: "2023-08-24T08:36:26Z"
generation: 1
name: public-net
namespace: default
resourceVersion: "366998"
uid: 22a5a8e2-c580-4e10-9eb2-b12b92bffffd
spec:
config: '{ "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode":
"bridge", "ipam": { "type": "whereabouts", "range": "192.168.20.0/24" } }'
- apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
creationTimestamp: "2023-08-24T12:24:15Z"
generation: 1
The NAD collection code appears twice [on release-4.14 branch]. https://github.com/red-hat-storage/odf-must-gather/blob/release-4.14/must-gather/collection-scripts/gather_namespaced_resources#L229-L239 I opened a new PR to fix it https://github.com/red-hat-storage/odf-must-gather/pull/60 SetUp: ODF Version: odf-operator.v4.14.0-126.stable OCP Version: 4.14.0-0.nightly-2023-09-02-132842 Platform: BM Test Process: 1. Collect mg: $ oc adm must-gather --image=quay.io/rhceph-dev/ocs-must-gather:latest-4.14 2.Check NAD oviner:all$ pwd namespaces/all oviner:all$ ls | grep net desc_net_attach_def_all_ns get_yaml_net_attach_def_all_ns Oded, I closed the PR. You need to open a PR on main branch and not release-4.14 dirtectly. Can you please open a new PR against main branch? The code that performs the NAD collection is duplicated.
I also checked the image itself.
$ podman pull quay.io/rhceph-dev/ocs-must-gather:latest-4.14
$ podman run --entrypoint /bin/bash -it quay.io/rhceph-dev/ocs-must-gather:latest-4.14
[root@eafcd3003a6d /]#
[root@eafcd3003a6d bin]# pwd
/usr/bin
[root@eafcd3003a6d bin]# cat gather_namespaced_resources
...
...
...
echo "collecting network-attachment-definitions of oc get network-attachment-definitions all namespaces" | tee -a "${BASE_COLLECTION_PATH}/gather-debug.log"
if [ -n "$(oc get network-attachment-definitions --no-headers -A | awk '{print $1}')" ]; then
{ oc get network-attachment-definitions -oyaml --all-namespaces; } >>"${BASE_COLLECTION_PATH}/namespaces/all/get_yaml_net_attach_def_all_ns"
{ oc describe network-attachment-definitions --all-namespaces; } >>"${BASE_COLLECTION_PATH}/namespaces/all/desc_net_attach_def_all_ns"
fi
echo "collecting network-attachment-definitions of oc get network-attachment-definitions all namespaces" | tee -a "${BASE_COLLECTION_PATH}/gather-debug.log"
if [ "$(oc get network-attachment-definitions --no-headers -A | awk '{print $1}')" != "" ] ; then
{ oc get network-attachment-definitions -oyaml --all-namespaces; } >> "${BASE_COLLECTION_PATH}/namespaces/all/get_yaml_net_attach_def_all_ns"
{ oc describe network-attachment-definitions --all-namespaces; } >> "${BASE_COLLECTION_PATH}/namespaces/all/desc_net_attach_def_all_ns"
fi
Build was done before the PR was merged, next build must contain the changes. bug fixed:
SetUp:
OCP Version: 4.14.0-0.nightly-2023-09-26-124507
ODF Version: odf-operator.v4.14.0-139.stable
platform: Vsphere
Test process:
1.Install OCP4.14+ODF4.14 [without multus]
2.Create NAD:
---
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: public-net
namespace: default
labels: {}
annotations: {}
spec:
config: '{ "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.20.0/24" } }'
---
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: cluster-net
namespace: default
labels: {}
annotations: {}
spec:
config: '{ "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.30.0/24" } }'
3.Collect MG:
$ oc adm must-gather --image=quay.io/rhceph-dev/ocs-must-gather:latest-4.14
4.Check content:
oviner:all$ pwd
.../namespaces/all
oviner:all$ ls | grep net
desc_net_attach_def_all_ns
get_yaml_net_attach_def_all_ns
$ cat desc_net_attach_def_all_ns
Name: cluster-net
Namespace: default
Labels: <none>
Annotations: <none>
API Version: k8s.cni.cncf.io/v1
Kind: NetworkAttachmentDefinition
Metadata:
Creation Timestamp: 2023-09-27T15:14:15Z
Generation: 1
Managed Fields:
API Version: k8s.cni.cncf.io/v1
Fields Type: FieldsV1
fieldsV1:
f:spec:
.:
f:config:
Manager: kubectl-create
Operation: Update
Time: 2023-09-27T15:14:15Z
Resource Version: 158138
UID: ea10451a-53d2-4c79-9cca-e87f93db05b8
Spec:
Config: { "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.30.0/24" } }
Events: <none>
Name: public-net
Namespace: default
Labels: <none>
Annotations: <none>
API Version: k8s.cni.cncf.io/v1
Kind: NetworkAttachmentDefinition
Metadata:
Creation Timestamp: 2023-09-27T15:14:15Z
Generation: 1
Managed Fields:
API Version: k8s.cni.cncf.io/v1
Fields Type: FieldsV1
fieldsV1:
f:spec:
.:
f:config:
Manager: kubectl-create
Operation: Update
Time: 2023-09-27T15:14:15Z
Resource Version: 158135
UID: af9ef2af-2842-445e-88c2-e70e07423148
Spec:
Config: { "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.20.0/24" } }
Events: <none>
$ cat get_yaml_net_attach_def_all_ns
apiVersion: v1
items:
- apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
creationTimestamp: "2023-09-27T15:14:15Z"
generation: 1
name: cluster-net
namespace: default
resourceVersion: "158138"
uid: ea10451a-53d2-4c79-9cca-e87f93db05b8
spec:
config: '{ "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode":
"bridge", "ipam": { "type": "whereabouts", "range": "192.168.30.0/24" } }'
- apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
creationTimestamp: "2023-09-27T15:14:15Z"
generation: 1
name: public-net
namespace: default
resourceVersion: "158135"
uid: af9ef2af-2842-445e-88c2-e70e07423148
spec:
config: '{ "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode":
"bridge", "ipam": { "type": "whereabouts", "range": "192.168.20.0/24" } }'
kind: List
metadata:
resourceVersion: ""
selfLink: ""
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.14.0 security, enhancement & bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:6832 |
Description of problem (please be detailed as possible and provide log snippests): The ODF must-gather does not collect NetworkAttachmentDefinitions that affect Multus. The must-gather should be updated to collect NetworkAttachmentDefinition resources from the "default" and "openshift-storage" namespaces. Version of all relevant components (if applicable): I would hope this can be part of an upcoming 4.13.z release. Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? No. Is there any workaround available to the best of your knowledge? OCP must-gather collects NetworkAttachmentDefinitions Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 1 Can this issue reproducible? Yes. Can this issue reproduce from the UI? N/A If this is a regression, please provide more details to justify this: N/A Steps to Reproduce: It is not necessary to install ODF with multus enabled to repro! First, create these two NetworkAttachmentDefinitions: apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: cluster-net namespace: openshift-storage spec: config: '{ "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.30.0/24" } }' apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: cluster-net spec: config: '{ "cniVersion": "0.3.1", "type": "macvlan", "master": "br-ex", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.30.0/24" } }' Next, collect an ODF must-gather Both NADs should be collected but are not. The first NAD is in the openshift-storage NS, and the second is in the default NS. NADs from other NSes do not need to be collected for ODF.