Description of problem: OpenShift Operations has written a tool called oo-admin-regenerate-gear-metadata that can be used to recover a failed Node when all that is available is the gear data under /var/lib/openshift. Upstream commits: commit 00ec1ca76a8ac00c39d6e3851a884121828949fb Author: Thomas Wiest <twiest> Date: Wed Aug 13 13:05:04 2014 -0400 oo-admin-regenerate-gear-metadata: Changed to using oo_spawn and node cgroup libraries. Added --quiet and --no-accept-node options. commit 4c26b360d0285d69ab8be82c80243d16ca3b1a56 Author: Thomas Wiest <twiest> Date: Wed Aug 13 10:14:18 2014 -0400 Added oo-admin-regenerate-gear-metadata
Some bugs with this tool. 1), 2) 3) need to be fix. 4) we should discuss if provide this feature. 1) The tool shouldn't report error (return code 5) if no meta data is missing. [root@node2 limits.d]# oo-admin-regenerate-gear-metadata This script attempts to regenerate gear entries for: * /etc/passwd * /etc/shadow * /etc/group * /etc/cgrules.conf * /etc/cgconfig.conf * /etc/security/limits.d Proceed? [yes/NO]: yes Checking for passwd entry for 53fbfdfdfa838ef802000036... found, skipping. Checking for cgconfig.conf and cgrules.conf entries for 53fbfdfdfa838ef802000036... found, skipping. Checking for limits.d file for 53fbfdfdfa838ef802000036... found, skipping. Running oo-accept-node to check node consistency... INFO: using default accept-node extensions ---snip--- ---snip--- INFO: checking cartridge repository PASS Error: some problems weren't able to be fixed. [root@node2 limits.d]# echo $? 5 2) The fields 4,5,6 should be same as before in /etc/shadow. 53fbfd8efa838ef802000002:!!:16308:::::: 53fbfdc8fa838ef802000017:!!:16308:0:99999:7::: 53fbfe89fa838ef802000056:!!:16308:0:99999:7::: 53fbfec6fa838ef802000073:!!:16308:0:99999:7::: 53fbff68fa838ef802000088:!!:16308:0:99999:7::: 53fbff92fa838ef8020000a4:!!:16308:::::: 3)There isn't message when fixing /etc/security/limits.d/84-$uuid.conf 4) a limitation： If the group was delete while the user still exist, the tool couldn't fix the mismatching between /etc/passwd /etc/group and /etc/shadow.
Add positive comments： the main feature works well, it can fix the missed user/ cgroup /limit.d items, and those gears can be access after fix.
The problems from Comment #5 should be resolved with the latest build. In addition I added logic to oo-accept-node to detect if /etc/shadow entries were missing. If so the admin is given advice on how to resolve the problem.
Verified and pass on puddle-2-1-2014-08-27. oo-admin-regenerate-gear-metadata can regenerate gear entries. The issue in comment 5 was fixed.
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/1d5e9654268d01c9ae9262cdfa77e4b5e96817fc Bug 1131031 - improving /etc/group, /etc/shadow recovery
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-1183.html