Bug 1529233

Summary: "Add iptables allow rules" task failed when running system container install with cri-o enabled.
Product: OpenShift Container Platform Reporter: Johnny Liu <jialiu>
Component: InstallerAssignee: Giuseppe Scrivano <gscrivan>
Status: CLOSED ERRATA QA Contact: Johnny Liu <jialiu>
Severity: high Docs Contact:
Priority: high    
Version: 3.7.0CC: aos-bugs, jialiu, jokerman, mmccomas, xtian
Target Milestone: ---Keywords: Regression, TestBlocker
Target Release: 3.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-05 09:34:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Johnny Liu 2017-12-27 08:32:01 UTC
Description of problem:
See the following details.

inventory file and installation log will be attached together in one file later.

Version-Release number of the following components:
openshift-ansible-3.7.16-1.git.0.906ed81.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1. Trigger a system container install, and enable cri-o
openshift_use_system_containers=true
openshift_use_crio=true
2.
3.

Actual results:
installation job exit at the following task:
TASK [docker : Add iptables allow rules] ***************************************
Wednesday 27 December 2017  07:36:27 +0000 (0:00:00.174)       0:01:26.058 **** 
failed: [host-8-241-53.host.centralci.eng.rdu2.redhat.com] (item={u'port': u'10010/tcp', u'service': u'crio'}) => {"changed": false, "item": {"port": "10010/tcp", "service": "crio"}, "module_stderr": "iptables: Bad rule (does a matching rule exist in that chain?).\niptables: No chain/target/match by that name.\nTraceback (most recent call last):\n  File \"/tmp/ansible_9cH_b1/ansible_module_os_firewall_manage_iptables.py\", line 283, in <module>\n    main()\n  File \"/tmp/ansible_9cH_b1/ansible_module_os_firewall_manage_iptables.py\", line 266, in main\n    iptables_manager.add_rule(port, protocol)\n  File \"/tmp/ansible_9cH_b1/ansible_module_os_firewall_manage_iptables.py\", line 88, in add_rule\n    self.verify_chain()\n  File \"/tmp/ansible_9cH_b1/ansible_module_os_firewall_manage_iptables.py\", line 81, in verify_chain\n    self.create_chain()\n  File \"/tmp/ansible_9cH_b1/ansible_module_os_firewall_manage_iptables.py\", line 207, in create_chain\n    self.save()\n  File \"/tmp/ansible_9cH_b1/ansible_module_os_firewall_manage_iptables.py\", line 73, in save\n    self.output.append(subprocess.check_output(self.save_cmd, stderr=subprocess.STDOUT))\n  File \"/usr/lib64/python2.7/subprocess.py\", line 568, in check_output\n    process = Popen(stdout=PIPE, *popenargs, **kwargs)\n  File \"/usr/lib64/python2.7/subprocess.py\", line 711, in __init__\n    errread, errwrite)\n  File \"/usr/lib64/python2.7/subprocess.py\", line 1327, in _execute_child\n    raise child_exception\nOSError: [Errno 2] No such file or directory\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}

Go to the failed node, /etc/sysconfig/iptables is not created yet at that time.


Expected results:
installation is passed.

Additional info:
when running a rpm install with cri-o enabled, no such issue.

Comment 2 Johnny Liu 2017-12-27 08:37:13 UTC
Created attachment 1372708 [details]
installation log with inventory embeded

Comment 5 Scott Dodson 2018-01-02 13:24:21 UTC
Giuseppe, do you mind looking at this? It's likely missing the iptables-services package installation which gets triggered by including the os_firewall role, we need to ensure that the playbook that configures crio includes that role prior to including the docker role or potentially just make it a dependency of the docker role.

Comment 6 Giuseppe Scrivano 2018-01-02 23:24:09 UTC
Proposed PR:

https://github.com/openshift/openshift-ansible/pull/6590

Comment 7 Scott Dodson 2018-01-18 21:37:32 UTC
Do you know if the same problem happens on 3.9? There's been significant refactoring there which would have addressed this problem.

Unfortunately the fix on 3.7, which i agree with, is running into some integration test issues.

Comment 8 Johnny Liu 2018-01-19 14:47:53 UTC
(In reply to Scott Dodson from comment #7)
> Do you know if the same problem happens on 3.9? There's been significant
> refactoring there which would have addressed this problem.
> 
> Unfortunately the fix on 3.7, which i agree with, is running into some
> integration test issues.

No, when I was reporting this bug on 3.7, 3.9 does not have such problem, just like what you said, 3.9 has a significant refactor.

Comment 10 Johnny Liu 2018-01-26 05:52:40 UTC
Verified this bug with openshift-ansible-3.7.26-1.git.0.f87f1af.el7.noarch, and PASS.


iptable service is installed in the beginning of installation.
<--snip-->

PLAY [Ensure that all non-node hosts are accessible] ***************************

TASK [Gathering Facts] *********************************************************
Friday 26 January 2018  05:18:23 +0000 (0:00:00.040)       0:00:00.738 ******** 
ok: [ec2-52-90-147-28.compute-1.amazonaws.com]

PLAY [Initialize host facts] ***************************************************

TASK [Gathering Facts] *********************************************************
Friday 26 January 2018  05:18:24 +0000 (0:00:01.026)       0:00:01.765 ******** 
ok: [ec2-52-90-147-28.compute-1.amazonaws.com]
ok: [ec2-52-202-142-111.compute-1.amazonaws.com]

TASK [os_firewall : Detecting Atomic Host Operating System] ********************
Friday 26 January 2018  05:18:25 +0000 (0:00:00.688)       0:00:02.453 ******** 
ok: [ec2-52-202-142-111.compute-1.amazonaws.com] => {"changed": false, "stat": {"exists": false}}
ok: [ec2-52-90-147-28.compute-1.amazonaws.com] => {"changed": false, "stat": {"exists": false}}

TASK [os_firewall : Set fact r_os_firewall_is_atomic] **************************
Friday 26 January 2018  05:18:25 +0000 (0:00:00.511)       0:00:02.965 ******** 
ok: [ec2-52-90-147-28.compute-1.amazonaws.com] => {"ansible_facts": {"r_os_firewall_is_atomic": false}, "changed": false}
ok: [ec2-52-202-142-111.compute-1.amazonaws.com] => {"ansible_facts": {"r_os_firewall_is_atomic": false}, "changed": false}
<--snip-->
TASK [os_firewall : Ensure firewalld service is not enabled] *******************
Friday 26 January 2018  05:18:26 +0000 (0:00:00.039)       0:00:03.306 ******** 
ok: [ec2-52-202-142-111.compute-1.amazonaws.com] => {"changed": false, "failed_when_result": false, "msg": "Could not find the requested service firewalld: host"}
ok: [ec2-52-90-147-28.compute-1.amazonaws.com] => {"changed": false, "failed_when_result": false, "msg": "Could not find the requested service firewalld: host"}

TASK [os_firewall : Wait 10 seconds after disabling firewalld] *****************
Friday 26 January 2018  05:18:26 +0000 (0:00:00.710)       0:00:04.016 ******** 
skipping: [ec2-52-90-147-28.compute-1.amazonaws.com] => {"changed": false, "skip_reason": "Conditional result was False"}

TASK [os_firewall : Install iptables packages] *********************************
Friday 26 January 2018  05:18:26 +0000 (0:00:00.034)       0:00:04.051 ******** 
ok: [ec2-52-90-147-28.compute-1.amazonaws.com] => (item=iptables) => {"attempts": 1, "changed": false, "item": "iptables", "msg": "", "rc": 0, "results": ["iptables-1.4.21-18.el7.x86_64 providing iptables is already installed"]}
ok: [ec2-52-202-142-111.compute-1.amazonaws.com] => (item=iptables) => {"attempts": 1, "changed": false, "item": "iptables", "msg": "", "rc": 0, "results": ["iptables-1.4.21-18.el7.x86_64 providing iptables is already installed"]}
changed: [ec2-52-90-147-28.compute-1.amazonaws.com] => (item=iptables-services) => {"attempts": 1, "changed": true, "item": "iptables-services", "msg": "", "rc": 0, "results": ["Loaded plugins: amazon-id, search-disabled-repos\nResolving Dependencies\n--> Running transaction check\n---> Package iptables-services.x86_64 0:1.4.21-18.2.el7_4 will be installed\n--> Processing Dependency: iptables = 1.4.21-18.2.el7_4 for package: iptables-services-1.4.21-18.2.el7_4.x86_64\n--> Running transaction check\n---> Package iptables.x86_64 0:1.4.21-18.el7 will be updated\n---> Package iptables.x86_64 0:1.4.21-18.2.el7_4 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package           Arch   Version           Repository                     Size\n================================================================================\nInstalling:\n iptables-services x86_64 1.4.21-18.2.el7_4 oso-rhui-rhel-server-releases  51 k\nUpdating for dependencies:\n iptables          x86_64 1.4.21-18.2.el7_4 oso-rhui-rhel-server-releases 428 k\n\nTransaction Summary\n================================================================================\nInstall  1 Package\nUpgrade             ( 1 Dependent package)\n\nTotal download size: 479 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal                                              1.0 MB/s | 479 kB  00:00     \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n  Updating   : iptables-1.4.21-18.2.el7_4.x86_64                            1/3 \n  Installing : iptables-services-1.4.21-18.2.el7_4.x86_64                   2/3 \n  Cleanup    : iptables-1.4.21-18.el7.x86_64                                3/3 \n  Verifying  : iptables-services-1.4.21-18.2.el7_4.x86_64                   1/3 \n  Verifying  : iptables-1.4.21-18.2.el7_4.x86_64                            2/3 \n  Verifying  : iptables-1.4.21-18.el7.x86_64                                3/3 \n\nInstalled:\n  iptables-services.x86_64 0:1.4.21-18.2.el7_4                                  \n\nDependency Updated:\n  iptables.x86_64 0:1.4.21-18.2.el7_4                                           \n\nComplete!\n"]}
changed: [ec2-52-202-142-111.compute-1.amazonaws.com] => (item=iptables-services) => {"attempts": 1, "changed": true, "item": "iptables-services", "msg": "", "rc": 0, "results": ["Loaded plugins: amazon-id, search-disabled-repos\nResolving Dependencies\n--> Running transaction check\n---> Package iptables-services.x86_64 0:1.4.21-18.2.el7_4 will be installed\n--> Processing Dependency: iptables = 1.4.21-18.2.el7_4 for package: iptables-services-1.4.21-18.2.el7_4.x86_64\n--> Running transaction check\n---> Package iptables.x86_64 0:1.4.21-18.el7 will be updated\n---> Package iptables.x86_64 0:1.4.21-18.2.el7_4 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package           Arch   Version           Repository                     Size\n================================================================================\nInstalling:\n iptables-services x86_64 1.4.21-18.2.el7_4 oso-rhui-rhel-server-releases  51 k\nUpdating for dependencies:\n iptables          x86_64 1.4.21-18.2.el7_4 oso-rhui-rhel-server-releases 428 k\n\nTransaction Summary\n================================================================================\nInstall  1 Package\nUpgrade             ( 1 Dependent package)\n\nTotal download size: 479 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal                                              1.0 MB/s | 479 kB  00:00     \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n  Updating   : iptables-1.4.21-18.2.el7_4.x86_64                            1/3 \n  Installing : iptables-services-1.4.21-18.2.el7_4.x86_64                   2/3 \n  Cleanup    : iptables-1.4.21-18.el7.x86_64                                3/3 \n  Verifying  : iptables-services-1.4.21-18.2.el7_4.x86_64                   1/3 \n  Verifying  : iptables-1.4.21-18.2.el7_4.x86_64                            2/3 \n  Verifying  : iptables-1.4.21-18.el7.x86_64                                3/3 \n\nInstalled:\n  iptables-services.x86_64 0:1.4.21-18.2.el7_4                                  \n\nDependency Updated:\n  iptables.x86_64 0:1.4.21-18.2.el7_4                                           \n\nComplete!\n"]}

TASK [os_firewall : Start and enable iptables service] *************************
Friday 26 January 2018  05:18:35 +0000 (0:00:09.014)       0:00:13.066 ******** 
changed: [ec2-52-90-147-28.compute-1.amazonaws.com -> ec2-52-90-147-28.compute-1.amazonaws.com] => (item=ec2-52-90-147-28.compute-1.amazonaws.com) => {"changed": true, "enabled": true, "item": "ec2-52-90-147-28.compute-1.amazonaws.com", "name": "iptables", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "systemd-journald.socket basic.target syslog.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network.service shutdown.target ip6tables.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "IPv4 firewall with iptables", "DevicePolicy": "auto", "Environment": "BOOTUP=serial CONSOLETYPE=serial", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/libexec/iptables/iptables.init ; argv[]=/usr/libexec/iptables/iptables.init reload ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/libexec/iptables/iptables.init ; argv[]=/usr/libexec/iptables/iptables.init start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/libexec/iptables/iptables.init ; argv[]=/usr/libexec/iptables/iptables.init stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/iptables.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "iptables.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "14236", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "14236", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "iptables.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "yes", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "syslog", "StandardInput": "null", "StandardOutput": "syslog", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "0", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "oneshot", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "disabled", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}
changed: [ec2-52-90-147-28.compute-1.amazonaws.com -> ec2-52-202-142-111.compute-1.amazonaws.com] => (item=ec2-52-202-142-111.compute-1.amazonaws.com) => {"changed": true, "enabled": true, "item": "ec2-52-202-142-111.compute-1.amazonaws.com", "name": "iptables", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice syslog.target systemd-journald.socket basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network.service ip6tables.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "IPv4 firewall with iptables", "DevicePolicy": "auto", "Environment": "BOOTUP=serial CONSOLETYPE=serial", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/libexec/iptables/iptables.init ; argv[]=/usr/libexec/iptables/iptables.init reload ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/libexec/iptables/iptables.init ; argv[]=/usr/libexec/iptables/iptables.init start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/libexec/iptables/iptables.init ; argv[]=/usr/libexec/iptables/iptables.init stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/iptables.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "iptables.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "14236", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "14236", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "iptables.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "yes", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "syslog", "StandardInput": "null", "StandardOutput": "syslog", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "0", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "oneshot", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "disabled", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}

TASK [os_firewall : need to pause here, otherwise the iptables service starting can sometimes cause ssh to fail] ***
Friday 26 January 2018  05:18:37 +0000 (0:00:01.475)       0:00:14.542 ******** 
Pausing for 10 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
ok: [ec2-52-90-147-28.compute-1.amazonaws.com] => {"changed": false, "delta": 10, "rc": 0, "start": "2018-01-26 05:18:37.403396", "stderr": "", "stdout": "Paused for 10.0 seconds", "stop": "2018-01-26 05:18:47.403617", "user_input": ""}

TASK [openshift_sanitize_inventory : Check for usage of deprecated variables] ***
<--snip-->
TASK [docker : Create the CRI-O configuration] *********************************
Friday 26 January 2018  05:21:00 +0000 (0:00:00.668)       0:02:37.358 ******** 
changed: [ec2-52-90-147-28.compute-1.amazonaws.com] => {"backup_file": "/etc/crio/crio.conf.4408.2018-01-26@00:21:01~", "changed": true, "checksum": "782ce165fac14cbd300bb8c415a81d91bea3b250", "dest": "/etc/crio/crio.conf", "gid": 0, "group": "root", "md5sum": "e0c701d5ac969d27fa5bf8a50a6c8640", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:container_config_t:s0", "size": 5781, "src": "/root/.ansible/tmp/ansible-tmp-1516944060.33-127364211185816/source", "state": "file", "uid": 0}

TASK [docker : Ensure CNI configuration directory exists] **********************
Friday 26 January 2018  05:21:01 +0000 (0:00:01.042)       0:02:38.401 ******** 
changed: [ec2-52-90-147-28.compute-1.amazonaws.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cni/net.d/", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 6, "state": "directory", "uid": 0}

TASK [docker : Add iptables allow rules] ***************************************
Friday 26 January 2018  05:21:01 +0000 (0:00:00.332)       0:02:38.733 ******** 
changed: [ec2-52-90-147-28.compute-1.amazonaws.com] => (item={u'port': u'10010/tcp', u'service': u'crio'}) => {"changed": true, "item": {"port": "10010/tcp", "service": "crio"}, "output": ["", "Successfully created chain OS_FIREWALL_ALLOW", "iptables: Saving firewall rules to /etc/sysconfig/iptables: [  OK  ]\r\n", "", "iptables: Saving firewall rules to /etc/sysconfig/iptables: [  OK  ]\r\n", "", "iptables: Saving firewall rules to /etc/sysconfig/iptables: [  OK  ]\r\n"]}

Comment 14 errata-xmlrpc 2018-04-05 09:34:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0636