Red Hat Bugzilla – Bug 1217288
[enh] Configuration snapshots and rollbacks
Last modified: 2016-11-03 15:13:44 EDT
Various consumers of NetworkManager's API (like Cockpit) have requested snapshot/checkpoint capability for NM configuration before, to enable rollback if the machine fails post configuration tests. My initial thoughts on this are: - add a org.fdo.NM.Settings.Snapshot() -> (id: s), where we take a snapshot of all connectionsand serialize those somewhere in keyfile format in a tar.gz along with some metadata about what plugin they came from (if any, no plugin means temporary connection of course). - add a Commit(id: s) method; calling this method makes the changes since the given Snapshot() permanent, and deletes the .tar.gz of the snapshot origin - add a Rollback(id: s) method that resets that configuration to the backed up copy from Snapshot() by deleting any new connections, overwriting existing ones, and adding back missing ones. - if NM gets told to quit while there's an outstanding snapshot, it should probably roll everything back. Not 100% sure about this one though, but it's the safest bet. - NM should be determining what "successful" system boot or configuration actually is; that should be up to the thing that calls Snapshot() and Rollback(). We had discussions with the libvirt guys about this and they indicated that there are many ways to determine whether the config should be rolled back, that are higher level than just "did we get an IP". Pings to certain machines, "can I talk to known etcd", that kind of thing...
(In reply to Dan Williams from comment #0) > Various consumers of NetworkManager's API (like Cockpit) have requested > snapshot/checkpoint capability for NM configuration before, to enable > rollback if the machine fails post configuration tests. > > My initial thoughts on this are: > > - add a org.fdo.NM.Settings.Snapshot() -> (id: s), where we take a snapshot > of all connectionsand serialize those somewhere in keyfile format in a > tar.gz along with some metadata about what plugin they came from (if any, no > plugin means temporary connection of course). We could embed additional metadata inside keyfile such as [keyfile] setting-plugin=ifcfg-rh ifcfg-rh-filename=/etc/sysconfig/network-scripts/ifcfg-em1 Then either nm_keyfile_read() returns an additional hash of metadata, and nm_keyfile_writer() likewise accepts one. > - add a Commit(id: s) method; calling this method makes the changes since > the given Snapshot() permanent, and deletes the .tar.gz of the snapshot > origin > > - add a Rollback(id: s) method that resets that configuration to the backed > up copy from Snapshot() by deleting any new connections, overwriting > existing ones, and adding back missing ones. > > - if NM gets told to quit while there's an outstanding snapshot, it should > probably roll everything back. Not 100% sure about this one though, but > it's the safest bet. > > - NM should be determining what "successful" system boot or configuration > actually is; that should be up to the thing that calls Snapshot() and > Rollback(). We had discussions with the libvirt guys about this and they > indicated that there are many ways to determine whether the config should be > rolled back, that are higher level than just "did we get an IP". Pings to > certain machines, "can I talk to known etcd", that kind of thing... You want only one snapshot at a atime? We could have a list of snapshots with the following operations: SnapshotWrite() SnapshotLoad() SnapshotDelete() SnapshotList() Maybe instead of a tar file, concatenate them to one structured text file? That is then still human readable. Then they could be in one file like: /etc/NetworkManager/snapshots/<UUID>.snapshot Or even better: put separate keyfiles in a snapshot directory: /etc/NetworkManager/snapshots/<snapshot-UUID>/<connection-UUID>.keyfile The advantage of that is that keyfiles are very nicely to inspect and reuse. A user could do: cp /etc/NetworkManager/snapshots/<snapshot-UUID>/<connection-UUID>.keyfile \ /etc/NetworkMaanger/system-connections/my nmcli connection load /etc/NetworkMaanger/system-connections/my
/etc/NetworkManager/snapshots/<snapshot-UUID>/ or instead of using UUIDs, just use a timestamp? "%H%m%d-%H%M%S"? After all, we would not keep a cached version of this in memory. SnapshotList() would just iterate the filesystem. SnapshotDelete() would just delete a directory/file. SnapshotWrite() would create a new directory, and write the keyfiles there. SnapshotLoad(), would load all keyfiles from a directory in memory first, before proceeding with rollback. Anyway, the file name of the snapshot directory shouldn't matter.
This feature will not make it for rhel-7.2
A completely different solution would be RFE https://bugzilla.gnome.org/show_bug.cgi?id=757858 I think that would be more what cockpit requires.
Branch on review in upstream bug https://bugzilla.gnome.org/show_bug.cgi?id=757858
Branch merged, awaits next snapshot.
reverting changed configuration to saved state works well
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2581.html