Bug 1697814

Summary: [network-operator]Multus pod cannot be running due to "ERROR: unknown parameter --multus-log-level"
Product: OpenShift Container Platform Reporter: zhaozhanqi <zzhao>
Component: NetworkingAssignee: Douglas Smith <dosmith>
Status: CLOSED ERRATA QA Contact: Meng Bo <bmeng>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.1.0CC: aos-bugs, chuyu, dosmith, xxia
Target Milestone: ---Keywords: BetaBlocker, TestBlocker
Target Release: 4.1.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:47:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1690243, 1691055, 1695197, 1695209    

Description zhaozhanqi 2019-04-09 07:23:18 UTC
Description of problem:
When installing the cluster using build 4.0.0-0.nightly-2019-04-08-225815. 
found the multus pod cannot be running with error "ERROR: unknown parameter "--multus-log-level"


Version-Release number of selected component (if applicable):
4.0.0-0.nightly-2019-04-08-225815

How reproducible:
always

Steps to Reproduce:
1. setup cluster with build 4.0.0-0.nightly-2019-04-08-225815
2. check the multus pod
3.

Actual results:
# oc get pod -n openshift-multus
NAME           READY   STATUS             RESTARTS   AGE
multus-8djm7   0/1     CrashLoopBackOff   40         172m
multus-c74ph   0/1     CrashLoopBackOff   36         164m
multus-snhbv   0/1     CrashLoopBackOff   40         172m
multus-xhqhz   0/1     CrashLoopBackOff   37         166m
multus-zt5qt   0/1     CrashLoopBackOff   42         172m

# oc logs multus-c74ph -n openshift-multus
ERROR: unknown parameter "--multus-log-level"
This is an entrypoint script for Multus CNI to overlay its binary and 
configuration into locations in a filesystem. The configuration & binary file 
will be copied to the corresponding configuration directory. When 
/entrypoint.sh: line 20: --multus-conf-file=auto: command not found
 is used, 00-multus.conf will be automatically 
generated from the CNI configuration file of the master plugin (the first file 
in lexicographical order in cni-conf-dir).

./entrypoint.sh
	-h --help
	--cni-conf-dir=/host/etc/cni/net.d
	--cni-bin-dir=/host/opt/cni/bin
	--multus-conf-file=auto
	--multus-bin-file=/usr/src/multus-cni/bin/multus
	--multus-kubeconfig-file-host=/etc/kubernetes/cni/net.d/multus.d/multus.kubeconfig
	--namespace-isolation=true



Expected results:

multus should works well

Additional info:

Comment 1 Casey Callendrello 2019-04-09 09:12:04 UTC
This is definitely a bug, but I don't think this is a testblocker for 1691055. What seems to be the issue there?

Comment 2 Douglas Smith 2019-04-09 12:27:59 UTC
The cause of this one is almost certainly an out of date Multus being built. If the operator has specified this parameter for log level, and the version of Multus being built doesn't have this parameter in the entrypoint script, then... This will happen.

I'll dig up the details on what's being built and attempt to correct it. Details coming shortly.

Comment 3 Douglas Smith 2019-04-09 13:24:33 UTC
Looks like the commit in question is this commitish: 4aecbd21339aaa1c7874a5e71d3d9efc852fdfce

I've taken a look at both https://github.com/openshift/ose-multus-cni & https://github.com/openshift/multus-cni -- apparently the openshift/ose-multus-cni:release branch does NOT have this commit. In theory, we should've been building from the openshift/multus-cni:master branch.

In the meanwhile, I'm going to update the openshift/ose-multus-cni:release branch, and I'll get in touch with ART and verify where these are being built from.

Comment 4 Douglas Smith 2019-04-09 13:44:23 UTC
Ok, this is a build issue. Basically the gist is -- I need to have this commit reverted: https://github.com/openshift/ocp-build-data/commit/17e8a12042ff016191202af41484ea6bf3c79b25

This is related to this JIRA ticket @ https://jira.coreos.com/browse/ART-501 . This was closed 5 days ago, explaining why this is happening now.

We submitted a PR to fix the issue in that JIRA ticket ourselves: https://github.com/openshift/ocp-build-data/pull/49 and then the referenced commit essentially un-did the changes in this PR.

Comment 5 Douglas Smith 2019-04-09 14:03:34 UTC
Should be fixed in the build, per this merged PR: https://github.com/openshift/ocp-build-data/pull/77

Comment 9 zhaozhanqi 2019-04-11 02:08:39 UTC
Verified this bug on 4.0.0-0.nightly-2019-04-10-182914

Comment 11 errata-xmlrpc 2019-06-04 10:47:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758