| Summary: | Fail to start crio service on RHEL worker | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Gaoyun Pei <gpei> |
| Component: | Machine Config Operator | Assignee: | Peter Hunt <pehunt> |
| Status: | CLOSED ERRATA | QA Contact: | Michael Nguyen <mnguyen> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 4.3.0 | CC: | jialiu, kgarriso, wsun, yanyang |
| Target Milestone: | --- | Keywords: | TestBlocker |
| Target Release: | 4.3.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-01-23 11:18:15 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Bug Depends On: | 1782843 | ||
| Bug Blocks: | |||
@Peter this is relevant to the crio dirs I opened a PR on master to fix in MCO 4.3 backport linked Verified this bug with cri-o-1.16.1-6.dev.rhaos4.3.gitfa42e2f.el7.x86_64 and openshift-ansible-4.3.0-201912061055.git.177.a1caf57.el7.noarch, PASS.
$ cat /etc/crio/crio.conf
<--snip-->
hooks_dir = [
"/etc/containers/oci/hooks.d",
]
<--snip-->
crio service is started successfully.
# journalctl -f -u crio.service
-- Logs begin at Fri 2019-12-13 01:00:24 EST. --
Dec 13 01:00:33 xiyuan-1210-zxpm6-rhel-0 systemd[1]: Starting Open Container Initiative Daemon...
Dec 13 01:00:34 xiyuan-1210-zxpm6-rhel-0 systemd[1]: Started Open Container Initiative Daemon.
Dec 13 01:01:33 xiyuan-1210-zxpm6-rhel-0 crio[1859]: 2019-12-13T01:01:33-05:00 [verbose] Add: openshift-dns:dns-default-n2wcf:openshift-sdn:eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/proc/12600/ns/net"}],"ips":[{"version":"4","interface":0,"address":"10.130.2.3/23"}],"routes":[{"dst":"0.0.0.0/0","gw":"10.130.2.1"},{"dst":"224.0.0.0/4"},{"dst":"10.128.0.0/14"}],"dns":{}}
Dec 13 01:01:34 xiyuan-1210-zxpm6-rhel-0 crio[1859]: 2019-12-13T01:01:34-05:00 [verbose] Add: openshift-image-registry:node-ca-ntbzv:openshift-sdn:eth0 {"cniVersion":"0.3.1","interfaces":[{"name":"eth0","sandbox":"/proc/12608/ns/net"}],"ips":[{"version":"4","interface":0,"address":"10.130.2.2/23"}],"routes":[{"dst":"0.0.0.0/0","gw":"10.130.2.1"},{"dst":"224.0.0.0/4"},{"dst":"10.128.0.0/14"}],"dns":{}}
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062 |
Description of problem: With the new cri-o package(cri-o-1.16.1-2.dev.rhaos4.3.git7b04b62.el7.x86_64) installed on RHEL worker, crio.service failed to get started. -- Unit crio.service has begun starting up. Dec 09 00:49:57 ip-10-0-62-115.us-east-2.compute.internal crio[17896]: time="2019-12-09 00:49:57.366178213-05:00" level=fatal msg="runtime config: invalid hooks_dir: stat /usr/share/containers/oci/hooks.d: no such file or directory: stat /usr/share/containers/oci/hooks.d: no such file or directory" Dec 09 00:49:57 ip-10-0-62-115.us-east-2.compute.internal systemd[1]: crio.service: main process exited, code=exited, status=1/FAILURE Dec 09 00:49:57 ip-10-0-62-115.us-east-2.compute.internal systemd[1]: Failed to start Open Container Initiative Daemon. Compared to cri-o-1.14, there're two directories added in the hooks_dir [root@ip-10-0-62-115 ec2-user]# grep -A 3 hooks_dir /etc/crio/crio.conf hooks_dir = [ "/etc/containers/oci/hooks.d", "/usr/share/containers/oci/hooks.d", ] [root@ip-10-0-62-115 ec2-user]# ls /usr/share/containers/oci/hooks.d ls: cannot access /usr/share/containers/oci/hooks.d: No such file or directory On RHCOS worker, it's provided by oci-systemd-hook package. # rpm -qf /usr/share/containers/oci/hooks.d oci-systemd-hook-0.1.15-2.git2d0b8a3.module+el8+2769+577ad176.x86_64 Version-Release number of the following components: openshift-ansible.noarch 0:4.3.0-201912061055.git.177.a1caf57.el7 How reproducible: Always Steps to Reproduce: 1. 2. 3. Actual results: Please include the entire output from the last TASK line through the end of output if an error is generated Expected results: Additional info: Please attach logs from ansible-playbook with the -vvv flag