Bug 1801537 - RHEL worker can not get ready and crio service is reporting "Unknown option --persist-dir"
Summary: RHEL worker can not get ready and crio service is reporting "Unknown option -...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.4.0
Assignee: Jindrich Novy
QA Contact: Gaoyun Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-11 06:26 UTC by Gaoyun Pei
Modified: 2020-05-04 11:36 UTC (History)
8 users (show)

Fixed In Version: conmon-2.0.9-1.rhaos4.4.el8
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-04 11:35:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:0581 0 None None None 2020-05-04 11:36:01 UTC

Description Gaoyun Pei 2020-02-11 06:26:44 UTC
Description of problem:
This should be the same issue as https://bugzilla.redhat.com/show_bug.cgi?id=1783846 reported in 4.3.
In latest 4.4 puddle, it's still conmon-2.0.3-1.el7.x86_64.rpm 


Scaling-up RHEL worker to an existing 4.4 cluster, but the playbook failed at:

TASK [openshift_node : Wait for nodes to report ready] *************************
Tuesday 11 February 2020  11:07:43 +0800 (0:00:11.434)       0:08:44.158 ****** 
FAILED - RETRYING: Wait for nodes to report ready (36 retries left).
FAILED - RETRYING: Wait for nodes to report ready (35 retries left).
FAILED - RETRYING: Wait for nodes to report ready (34 retries left).
FAILED - RETRYING: Wait for nodes to report ready (33 retries left).
...
FAILED - RETRYING: Wait for nodes to report ready (2 retries left).
FAILED - RETRYING: Wait for nodes to report ready (1 retries left).
failed: [ip-10-0-60-42.us-east-2.compute.internal -> localhost] (item=ip-10-0-60-42.us-east-2.compute.internal) => {"ansible_loop_var": "item", "attempts": 36, "changed": true, "cmd": ["oc", "get", "node", "ip-10-0-60-42.us-east-2.compute.internal", "--config=/tmp/installer-Yg5_Ef/auth/kubeconfig", "--output=jsonpath={.status.conditions[?(@.type==\"Ready\")].status}"], "delta": "0:00:01.243624", "end": "2020-02-11 11:10:57.864377", "item": "ip-10-0-60-42.us-east-2.compute.internal", "rc": 0, "start": "2020-02-11 11:10:56.620753", "stderr": "", "stderr_lines": [], "stdout": "False", "stdout_lines": ["False"]}


Log into the RHEL worker, check the cri-o status
[root@ip-10-0-60-42 ec2-user]# systemctl status crio
● crio.service - Open Container Initiative Daemon
   Loaded: loaded (/usr/lib/systemd/system/crio.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/crio.service.d
           └─10-default-env.conf
   Active: active (running) since Tue 2020-02-11 03:07:13 UTC; 2h 30min ago
     Docs: https://github.com/cri-o/cri-o
 Main PID: 2219 (crio)
   Memory: 72.6M
   CGroup: /system.slice/crio.service
           └─2219 /usr/bin/crio --enable-metrics=true --metrics-port=9537

Feb 11 05:37:23 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:26 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:26 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:29 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:30 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:36 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:38 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:38 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:43 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir
Feb 11 05:37:45 ip-10-0-60-42.us-east-2.compute.internal crio[2219]: conmon: option parsing failed: Unknown option --persist-dir



Version-Release number of selected component (if applicable):
cri-o-1.17.0-1.dev.rhaos4.4.git3afd333.el7.x86_64
conmon-2.0.3-1.el7.x86_64

How reproducible:
Always

Actual results:

Expected results:
RHEL worker is joined cluster successfully.

Additional info:
After updating conmon to conmon-2.0.8-2.el7.x86_64, cri-o service is running well and RHEL worker became Ready.

Comment 1 Peter Hunt 2020-02-11 14:06:38 UTC
Jindrich,

Can you package conmon 2.0.9 (the version of conmon we are testing against with cri-o 1.17) for 4.4?

Comment 2 Jindrich Novy 2020-02-12 14:48:24 UTC
conmon-2.0.9 is now built for RHEL8 rhaos4.4:

http://brew-task-repos.usersys.redhat.com/repos/scratch/jnovy/conmon/2.0.9/1.rhaos4.4.el8/

Comment 13 errata-xmlrpc 2020-05-04 11:35:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581


Note You need to log in before you can comment on or make changes to this bug.