Kyle,

Do you have any info on the change to a tmpfs backed /var/run?  Is this a default in RHEL, do you know when this changed?

Thanks,
Chris

(In reply to Chris Feist from comment #2)
> Kyle,
> 
> Do you have any info on the change to a tmpfs backed /var/run?  Is this a
> default in RHEL, do you know when this changed?
> 

Yep! This is the default for RHEL 7. It was implemented upstream in Fedora 15:

https://docs.fedoraproject.org/en-US/Fedora/15/pdf/Release_Notes/Fedora-15-Release_Notes-en-US.pdf
~~~~
3.2.2. /run directory
Fedora 15 has a /run directory for storing runtime data. /run is now a tmpfs, and /var/run is bind
mounted to it. /var/lock is bind mounted to /run/lock. Applications can use /run the same
way as /var/run. Several programs including udev, dracut, mdadm, mount and initscripts
used hidden directories under /dev for runtime data during early bootup before /var is mounted.
However /dev/ is supposed to be used for only device nodes and there is consensus between major
distributions to shift to using /run instead. Fedora 15 is leading this change. Details including the
benefits are explained here26
.
This change is compliant with the Filesystem Hierarchy Standard27, which allows distributions
to create new directories in the root hierarchy as long as there is careful consideration of the
consequences. Co-author of the latest FHS specification has expressed support28 for this change.
Lennart Poettering has filed a request29 to update the FHS standard to include this change as well.
~~~

In RHEL 7, the /var/run directory is a symlink to the /run base tmpfs directory above:

  # ls -l /var/run
  lrwxrwxrwx. 1 root root 6 Jan 13  2016 /var/run -> ../runrp

More information: 
  https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Migration_Planning_Guide/sect-Red_Hat_Enterprise_Linux-Migration_Planning_Guide-File_System_Layout.html



In keeping with other packages solution to this type of behaviour, I would recommend adding a tmpfiles.d drop-in similar to the following:

   # cat /var/lib/tmpfiles.d/resource-agents.conf 
   d /var/run/resource-agents 1755 root root


That causes systemd-tmpfiles to create the directory with the correct permissions during each reboot iteration.

- Kyle Walker

One thing to note here, pacemaker (lrmd or pacemaker_remoted, to be
specific) will currently happily create that directory if it doesn't
exist already (with mode 0755) on the startup of the respective daemon.

That may be the primary reason of the clash observed with "rpm -V"
as the resource-agents in their "make install" will create that
directory with 1755 (note the restricted deletion flag) that then
gets remembered.


But what comes clear, that directory is a sort of an integration point
between (clustered) resource manager and the resource-agents (i.e.
implementation of some resources using settled "API").

On several occasions already, I've been toying with an idea of
"ocf-standard" (or the like) separate package which would concentrate
all such commons and that would be required by both resources-users
and resources-providers.  Besided common paths, it would also offer
rpm macros for them (to be utilized in the spec files of both parties,
assuming it is also build-required) so there would be simplification
(no hardcoding), and tools like rpmlint would stop complaining about
that.

Such package might be also a good candidate to deliver the tmpfiles.d
config if it's deemed a way forward.

@Jan,

Just a heads-up. For whichever reason, pacemaker actually creates the directory with 0751 permissions.

    # ls -l /var/run/resource-agents -d
    ls: cannot access /var/run/resource-agents: No such file or directory

    # pcs cluster start

    # ls -l /var/run/resource-agents -d
    drwxr-x--x. 2 root root 40 Jun 26 08:37 /var/run/resource-agents


Not sure it will cause any issues, but it does result in an unexpected permission set.

- Kyle Walker

Kyle, I believe this is because of additional umask enforcement:

lib/common/utils.c: crm_make_daemon: umask(0026)

Although I checked that a difference in restricted deletion flag
alone will provoke that complaint on rpm -V.

(In reply to Jan Pokorný from comment #8)
> Kyle, I believe this is because of additional umask enforcement:
> 
> lib/common/utils.c: crm_make_daemon: umask(0026)
> 
> Although I checked that a difference in restricted deletion flag
> alone will provoke that complaint on rpm -V.

Ahhh, I missed that. That makes sense. I like the separate package approach with tmpfiles.d configuration file approach.

Thanks for the feedback!

- Kyle Walker

Agreed that we should add a tmpfiles.d configuration when systemd is in use. But I think it should stay in the resource-agents package, because the directory is specific to these agents and not the OCF standard. Other agents are free to depend on resource-agents and use the same directory if desired.

For backward compatibility though, pacemaker should still create the directory if it doesn't exist.

(In reply to Ken Gaillot from comment #10)
Makes sense to me, so I'll look into adding it to resource-agents.

> the directory is specific to these agents and not the OCF standard.
> Other agents are free to depend on resource-agents and use the same
> directory if desired.
> 
> For backward compatibility though, pacemaker should still create the
> directory if it doesn't exist.

True, "RSCTMPDIR" is currently not a part of OCF standard...
But I am the only one to see the internal conflict between
resource-agents originally going thus far to make the resource
managers (originating in heartbeat, it seems) create 
semi-standard directory for them ... and just for them (because
again, OCF does not specify such a directory and the connected
requirements on it)?

General OCF adherent audience would appreciate such an optional
benefit like having predestined directory to store tempory data,
I believe.  And there are systems that have no equivalent of
systemd-tmpfiles (e.g. tmpwatch is just a cleaner) generaly handy.

That would also alleviate the schizophrenia where some of RAs
create that directory on their own -- but not all!

Also note that restricted deletion flag makes little sense for
anything running as root (which is a default mode of operation
for RAs -- there is hardly a priviledge separation to enforce).


So I see at least two other alternatives:

1. each agent is responsible for managing "RSCTMPDIR" on its own,
   there's no need for any sort of babysitting then
   (tmpfiles, OCF, not even in pacemaker + specfile of resource-agents
   should read: "%dir %verify(not mode) %{_var}/run/resource-agents")

2. OCF standard actually becomes the enforcing point for that
   directory, it's semantics and requirements imposed on it
   (for both providers and users)

Note that both 1. and 2. are more systemic than adding 4th(!) possible
originator of that directory:

- installation of resource-agents package (only until the next reboot
  if the FHS semantics are preserved)
- pacemaker
- agent itself
- tmpfiles.d config

So I rechecked the Fedora packaging guidelines and related sources
and slightly shifted my opinion:

Initial considerations:
- we are discussing RHEL 7, which branched from Fedora 18
- "/var/run and /var/lock on tmpfs" change introduced in Fedora 15
  https://fedoraproject.org/wiki/Features/var-run-tmpfs
- respective packaging guidelines:
  https://fedoraproject.org/wiki/Packaging:Tmpfiles.d

Advantages of using tmpfiles.d mechanism:
- it gets SELinux labels right out of the box (not that it currently
  matters, but could/should)
- it's supposed to work correctly with RPM files tracking
  (it's unlikely it would ever run prior to systemd-tmpfiles
   kicking in) and listing of the such files/dirs is even encouraged

Disadvantes:
- tmpfiles.d mechanism is not generally available in containers
  (assuming there is an effort to make do without systemd in it when
  using pacemaker-remote?) ... but see the initscript equivalent idea
  bellow

* * *

So while in [comment 13] I opposed the idea of multi-sourced
birth of the directory at hand (complexity is nasty, right?),
tmpfiles.d config now looks as well-fitting in the wider system
perspective, and we can perhaps plan on cutting other ones:

- in pacemaker:
  as mentioned, that directory was never standardized (it's strange
  to embed a non-standard use-case specific support silently);
  if it gets (under OCF), it should be some equivalent of proposed
  ocf-standard package that will eventually deliver that tmpfiles.d
  config (and simple initscript for daemons like pacemaker to depend
  on with non-systemd systems)

- in agents themselves:
  they should start taking existence of that directory for granted; at
  the same time, it should become a constant, not an externally
  influencable variable (precondition for the former to work reliably)
  (and we have bind mounts, chroots and containers for extravagant
  corner cases requiring a use of the different effective path)

https://github.com/ClusterLabs/resource-agents/pull/1031

Marking Verified in version resource-agents-3.9.5-116.el7.x86_64. /var/run/resource-agents is now present after reboot.


1) BEFORE fix - /var/run/resource-agents is missing after reboot
=============
# rpm -q resource-agents
resource-agents-3.9.5-105.el7_4.2.x86_64

# reboot

# ls /var/run | grep resource-agents

# rpm -V resource-agents
missing     /var/run/resource-agents


2) AFTER fix - /var/run/resource-agents is now present after reboot
=============
# rpm -q resource-agents
resource-agents-3.9.5-116.el7.x86_64

# reboot

# ls /var/run | grep resource-agents
resource-agents

# rpm -V resource-agents

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0757