Bug 1663376 - MachineConfigDaemon does not prune files that don't exist in the desiredConfig but existed before
Summary: MachineConfigDaemon does not prune files that don't exist in the desiredConfi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: RHCOS
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.1.0
Assignee: Steve Milner
QA Contact: Micah Abbott
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-04 06:36 UTC by Johnny Liu
Modified: 2019-06-04 10:41 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:41:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 0 None None None 2019-06-04 10:41:36 UTC

Description Johnny Liu 2019-01-04 06:36:27 UTC
Description of problem:
MachineConfigDaemon does not prune files that don't exist in the desiredConfig but existed before

Version-Release number of the following components:
# ./openshift-install version
./openshift-install v0.8.0-master-8-g713289e20bd6afccb06f2e4ff7ed89d2483fac9a

# oc get clusterversion
NAME      VERSION                           AVAILABLE   PROGRESSING   SINCE     STATUS
version   4.0.0-0.alpha-2019-01-03-031244   True        False         7m        Cluster version is 4.0.0-0.alpha-2019-01-03-031244

How reproducible:
Always

Steps to Reproduce:
1. Setup a cluster
2. Create a machineconfnig to write a file onto the machine
# cat customized_master.yaml 
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: master
  name: master-customization1
spec:
  config:
    ignition:
      config: {}
      security:
        tls: {}
      timeouts: {}
      version: 2.2.0
    networkd: {}
    passwd: {}
    storage:
      files:
      - contents:
          source: data:,testing1
          verification: {}
        filesystem: root
        mode: 420
        path: /etc/custom1.txt
  osImageURL: ""
# oc create -f customized_master.yaml
3. Check the new generated machineconfig take effect
# oc get machineconfig
NAME                                      AGE
00-master                                 1h
00-worker                                 1h
01-master-kubelet                         1h
01-worker-kubelet                         1h
master-0c99241e37b235e1dc1ae3fd98e1a329   1h
master-7f6162f8e56ab5a461301e10b588c640   13s
master-bc3342fb14b6afc253e7a0152d1a18bc   1h
master-customization1                     13s
worker-1b808daba3e3e583a13ee10f805adb16   1h
worker-df55614d2c4782d133a310c118d89fea   1h

#  oc describe node|grep machineconfig
                    machineconfiguration.openshift.io/currentConfig=worker-df55614d2c4782d133a310c118d89fea
                    machineconfiguration.openshift.io/desiredConfig=worker-df55614d2c4782d133a310c118d89fea
                    machineconfiguration.openshift.io/state=Done
                    machineconfiguration.openshift.io/currentConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/desiredConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/state=Done
                    machineconfiguration.openshift.io/currentConfig=worker-df55614d2c4782d133a310c118d89fea
                    machineconfiguration.openshift.io/desiredConfig=worker-df55614d2c4782d133a310c118d89fea
                    machineconfiguration.openshift.io/state=Done
                    machineconfiguration.openshift.io/currentConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/desiredConfig=master-7f6162f8e56ab5a461301e10b588c640
                    machineconfiguration.openshift.io/state=Working
                    machineconfiguration.openshift.io/currentConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/desiredConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/state=Done
4. From the above output, the newly generated machineconfig (master-7f6162f8e56ab5a461301e10b588c640) is being applied to replace old one (master-bc3342fb14b6afc253e7a0152d1a18bc).
5. Once done, go to master node, check /etc/custom1.txt is written to disk.
# cat /etc/custom1.txt 
testing1
6. Remove 'master-customization1' machineconfig, make sure the desiredconfig is changed back.
# oc delete machineconfig master-customization1
machineconfig.machineconfiguration.openshift.io "master-customization1" deleted
#  oc describe node|grep machineconfig
                    machineconfiguration.openshift.io/currentConfig=worker-df55614d2c4782d133a310c118d89fea
                    machineconfiguration.openshift.io/desiredConfig=worker-df55614d2c4782d133a310c118d89fea
                    machineconfiguration.openshift.io/state=Done
                    machineconfiguration.openshift.io/currentConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/desiredConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/state=Done
                    machineconfiguration.openshift.io/currentConfig=worker-df55614d2c4782d133a310c118d89fea
                    machineconfiguration.openshift.io/desiredConfig=worker-df55614d2c4782d133a310c118d89fea
                    machineconfiguration.openshift.io/state=Done
                    machineconfiguration.openshift.io/currentConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/desiredConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/state=Done
                    machineconfiguration.openshift.io/currentConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/desiredConfig=master-bc3342fb14b6afc253e7a0152d1a18bc
                    machineconfiguration.openshift.io/state=Done
7. get diff output between master-7f6162f8e56ab5a461301e10b588c640 and master-bc3342fb14b6afc253e7a0152d1a18bc, according to https://github.com/openshift/machine-config-operator/blob/master/docs/MachineConfigDaemon.md#directory--file-updates:
'''
The daemon should prune all the files and directories that don't exist in the desiredConfig but existed before. Diff the current config and desired config, then remove the nodes that were removed.
'''


Actual results:
Go to master node, check again, '/etc/custom1.txt' is still existing.
# cat /etc/custom1.txt 
testing1

Expected results:
'/etc/custom1.txt' should be pruned.

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Alex Crawford 2019-01-05 00:55:05 UTC
This is not an installer issue. You'll need to assign this to the RHCOS team. They maintain the Machine Config Operator.

Comment 2 Johnny Liu 2019-01-07 02:37:07 UTC
Good to know that, Changing to RHCOS component.

Comment 5 Steve Milner 2019-01-09 14:29:25 UTC
Second PR: https://github.com/openshift/machine-config-operator/pull/274

Both merged.

Comment 6 Johnny Liu 2019-01-15 10:53:31 UTC
Verified this bug with quay.io/openshift-release-dev/ocp-release:4.0.0-0.1, and PASS.

After customized machineconfig is removed, the created file is cleaned up.

$ cat /etc/custom1.txt 
cat: /etc/custom1.txt: No such file or directory

Comment 9 errata-xmlrpc 2019-06-04 10:41:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.