Description of problem: Underlying cause of #1787581, opening BZ as per Colin's request here: https://github.com/openshift/machine-config-operator/pull/1361#issuecomment-572737805 SDN is currently saving state/identifiers related to transient things in persistent files ie: /var/lib/cni/networks/openshift-sdn/ /run should be used for this. #1787581 is a fix (explicitly removing this dir on reboot) but SDN should be modified to use /run not /var for this in the first place.
It's possible to move the directory, and it probably should be done. FWIW /var/lib/cni/networks is the default upstream default directory in CNI. Moving would require (1) updating hte mounted directories in the CNO, and (2) changing the openshift-sdn host-local IPAM CNI config to add the 'dataDir' option in the CNI config JSON to point to where we want it. That would be clearer than playing games with the container mounts. However, the big issue would be upgrades-without-reboot; when the new openshift-sdn starts it would no longer see existing pod IP leases because it would be looking at a new directory. So we'd have to figure out how to copy stuff over from the old dir to the new dir, or something like that.
Opened JIRA card for tracking this: https://issues.redhat.com/browse/SDN-781 Closing BZ.