Hide Forgot
Description of problem: NAME READY STATUS RESTARTS AGE po/dancer-mysql-persistent-1-build 0/1 Init:0/2 0 9m po/database-1-deploy 0/1 ContainerCreating 0 9m Version-Release number of selected component (if applicable): [root@free-stg-master-03fb6 ~]# oc version oc v3.7.4 kubernetes v1.7.6+a08f5eeb62 features: Basic-Auth GSSAPI Kerberos SPNEGO Server https://172.31.78.254:443 openshift v3.7.4 kubernetes v1.7.6+a08f5eeb62
The CNI plugin is jammed (54k occurences in the node log): 1304 cni.go:304] Error deleting network when building cni runtime conf: could not retrieve port mappings: checkpoint is corrupted. While the checkpoint file should not be corrupt, the docker shim should remove any corrupt checkpoint file, but it is not due to a bug. buildCNIRuntimeConf() is modifying the err from plugin.host.GetPodPortMappings() as it propagates to the caller. However, the caller checks the error against errors.CorruptCheckpointError to determine if the checkpoint file should be removed. This will never be true as buildCNIRuntimeConf() is modifying the error.
Sorry, meant to keep this one. Working on a fix.
Sorry for the delay. The corrupt checkpoint messages, while nasty, are not the cause of the delay in sandbox start. It is the vnid issue again. *** This bug has been marked as a duplicate of bug 1509799 ***
(In reply to Seth Jennings from comment #4) > While the checkpoint file should not be corrupt, the docker shim should > remove any corrupt checkpoint file, but it is not due to a bug. > > buildCNIRuntimeConf() is modifying the err from > plugin.host.GetPodPortMappings() as it propagates to the caller. However, > the caller checks the error against errors.CorruptCheckpointError to > determine if the checkpoint file should be removed. This will never be true > as buildCNIRuntimeConf() is modifying the error. The newly-added check in docker_sandbox.go is also too late: the CorruptCheckpointError we're getting isn't coming from StopContainer(), it's coming from TearDownPod() a few lines earlier (via buildCNIRuntimeConf() -> GetPodPortMappings() -> GetCheckpoint()).