Hide Forgot
Description of problem: The /var/run/resource-agents/ directory is created with the incorrect mode. During installation it is created correctly, however, with the advent of the tmpfs-backed /run directory, this does not persist across a reboot. This results in "rpm" verification checks, used by a number of system integrity scanning utilities, to fail similar to the following: # rpm -V resource-agents .M....... /var/run/resource-agents Version-Release number of selected component (if applicable): resource-agents-3.9.5-82.el7_3.11 How reproducible: Easily Steps to Reproduce: 1. Install resource-agents 2. Restart the system 3. Actual results: # rpm -V resource-agents .M....... /var/run/resource-agents Expected results: # rpm -V resource-agents #Nothing returned Additional info:
Kyle, Do you have any info on the change to a tmpfs backed /var/run? Is this a default in RHEL, do you know when this changed? Thanks, Chris
(In reply to Chris Feist from comment #2) > Kyle, > > Do you have any info on the change to a tmpfs backed /var/run? Is this a > default in RHEL, do you know when this changed? > Yep! This is the default for RHEL 7. It was implemented upstream in Fedora 15: https://docs.fedoraproject.org/en-US/Fedora/15/pdf/Release_Notes/Fedora-15-Release_Notes-en-US.pdf ~~~~ 3.2.2. /run directory Fedora 15 has a /run directory for storing runtime data. /run is now a tmpfs, and /var/run is bind mounted to it. /var/lock is bind mounted to /run/lock. Applications can use /run the same way as /var/run. Several programs including udev, dracut, mdadm, mount and initscripts used hidden directories under /dev for runtime data during early bootup before /var is mounted. However /dev/ is supposed to be used for only device nodes and there is consensus between major distributions to shift to using /run instead. Fedora 15 is leading this change. Details including the benefits are explained here26 . This change is compliant with the Filesystem Hierarchy Standard27, which allows distributions to create new directories in the root hierarchy as long as there is careful consideration of the consequences. Co-author of the latest FHS specification has expressed support28 for this change. Lennart Poettering has filed a request29 to update the FHS standard to include this change as well. ~~~ In RHEL 7, the /var/run directory is a symlink to the /run base tmpfs directory above: # ls -l /var/run lrwxrwxrwx. 1 root root 6 Jan 13 2016 /var/run -> ../runrp More information: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Migration_Planning_Guide/sect-Red_Hat_Enterprise_Linux-Migration_Planning_Guide-File_System_Layout.html In keeping with other packages solution to this type of behaviour, I would recommend adding a tmpfiles.d drop-in similar to the following: # cat /var/lib/tmpfiles.d/resource-agents.conf d /var/run/resource-agents 1755 root root That causes systemd-tmpfiles to create the directory with the correct permissions during each reboot iteration. - Kyle Walker
One thing to note here, pacemaker (lrmd or pacemaker_remoted, to be specific) will currently happily create that directory if it doesn't exist already (with mode 0755) on the startup of the respective daemon. That may be the primary reason of the clash observed with "rpm -V" as the resource-agents in their "make install" will create that directory with 1755 (note the restricted deletion flag) that then gets remembered. But what comes clear, that directory is a sort of an integration point between (clustered) resource manager and the resource-agents (i.e. implementation of some resources using settled "API"). On several occasions already, I've been toying with an idea of "ocf-standard" (or the like) separate package which would concentrate all such commons and that would be required by both resources-users and resources-providers. Besided common paths, it would also offer rpm macros for them (to be utilized in the spec files of both parties, assuming it is also build-required) so there would be simplification (no hardcoding), and tools like rpmlint would stop complaining about that. Such package might be also a good candidate to deliver the tmpfiles.d config if it's deemed a way forward.
@Jan, Just a heads-up. For whichever reason, pacemaker actually creates the directory with 0751 permissions. # ls -l /var/run/resource-agents -d ls: cannot access /var/run/resource-agents: No such file or directory # pcs cluster start # ls -l /var/run/resource-agents -d drwxr-x--x. 2 root root 40 Jun 26 08:37 /var/run/resource-agents Not sure it will cause any issues, but it does result in an unexpected permission set. - Kyle Walker
Kyle, I believe this is because of additional umask enforcement: lib/common/utils.c: crm_make_daemon: umask(0026) Although I checked that a difference in restricted deletion flag alone will provoke that complaint on rpm -V.
(In reply to Jan Pokorný from comment #8) > Kyle, I believe this is because of additional umask enforcement: > > lib/common/utils.c: crm_make_daemon: umask(0026) > > Although I checked that a difference in restricted deletion flag > alone will provoke that complaint on rpm -V. Ahhh, I missed that. That makes sense. I like the separate package approach with tmpfiles.d configuration file approach. Thanks for the feedback! - Kyle Walker
Agreed that we should add a tmpfiles.d configuration when systemd is in use. But I think it should stay in the resource-agents package, because the directory is specific to these agents and not the OCF standard. Other agents are free to depend on resource-agents and use the same directory if desired. For backward compatibility though, pacemaker should still create the directory if it doesn't exist.
(In reply to Ken Gaillot from comment #10) Makes sense to me, so I'll look into adding it to resource-agents.
> the directory is specific to these agents and not the OCF standard. > Other agents are free to depend on resource-agents and use the same > directory if desired. > > For backward compatibility though, pacemaker should still create the > directory if it doesn't exist. True, "RSCTMPDIR" is currently not a part of OCF standard... But I am the only one to see the internal conflict between resource-agents originally going thus far to make the resource managers (originating in heartbeat, it seems) create semi-standard directory for them ... and just for them (because again, OCF does not specify such a directory and the connected requirements on it)? General OCF adherent audience would appreciate such an optional benefit like having predestined directory to store tempory data, I believe. And there are systems that have no equivalent of systemd-tmpfiles (e.g. tmpwatch is just a cleaner) generaly handy. That would also alleviate the schizophrenia where some of RAs create that directory on their own -- but not all! Also note that restricted deletion flag makes little sense for anything running as root (which is a default mode of operation for RAs -- there is hardly a priviledge separation to enforce). So I see at least two other alternatives: 1. each agent is responsible for managing "RSCTMPDIR" on its own, there's no need for any sort of babysitting then (tmpfiles, OCF, not even in pacemaker + specfile of resource-agents should read: "%dir %verify(not mode) %{_var}/run/resource-agents") 2. OCF standard actually becomes the enforcing point for that directory, it's semantics and requirements imposed on it (for both providers and users)
Note that both 1. and 2. are more systemic than adding 4th(!) possible originator of that directory: - installation of resource-agents package (only until the next reboot if the FHS semantics are preserved) - pacemaker - agent itself - tmpfiles.d config
So I rechecked the Fedora packaging guidelines and related sources and slightly shifted my opinion: Initial considerations: - we are discussing RHEL 7, which branched from Fedora 18 - "/var/run and /var/lock on tmpfs" change introduced in Fedora 15 https://fedoraproject.org/wiki/Features/var-run-tmpfs - respective packaging guidelines: https://fedoraproject.org/wiki/Packaging:Tmpfiles.d Advantages of using tmpfiles.d mechanism: - it gets SELinux labels right out of the box (not that it currently matters, but could/should) - it's supposed to work correctly with RPM files tracking (it's unlikely it would ever run prior to systemd-tmpfiles kicking in) and listing of the such files/dirs is even encouraged Disadvantes: - tmpfiles.d mechanism is not generally available in containers (assuming there is an effort to make do without systemd in it when using pacemaker-remote?) ... but see the initscript equivalent idea bellow * * * So while in [comment 13] I opposed the idea of multi-sourced birth of the directory at hand (complexity is nasty, right?), tmpfiles.d config now looks as well-fitting in the wider system perspective, and we can perhaps plan on cutting other ones: - in pacemaker: as mentioned, that directory was never standardized (it's strange to embed a non-standard use-case specific support silently); if it gets (under OCF), it should be some equivalent of proposed ocf-standard package that will eventually deliver that tmpfiles.d config (and simple initscript for daemons like pacemaker to depend on with non-systemd systems) - in agents themselves: they should start taking existence of that directory for granted; at the same time, it should become a constant, not an externally influencable variable (precondition for the former to work reliably) (and we have bind mounts, chroots and containers for extravagant corner cases requiring a use of the different effective path)
https://github.com/ClusterLabs/resource-agents/pull/1031
Marking Verified in version resource-agents-3.9.5-116.el7.x86_64. /var/run/resource-agents is now present after reboot. 1) BEFORE fix - /var/run/resource-agents is missing after reboot ============= # rpm -q resource-agents resource-agents-3.9.5-105.el7_4.2.x86_64 # reboot # ls /var/run | grep resource-agents # rpm -V resource-agents missing /var/run/resource-agents 2) AFTER fix - /var/run/resource-agents is now present after reboot ============= # rpm -q resource-agents resource-agents-3.9.5-116.el7.x86_64 # reboot # ls /var/run | grep resource-agents resource-agents # rpm -V resource-agents
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0757