Bug 1462802 - /var/run/resource-agents/ is created with the incorrect mode
Summary: /var/run/resource-agents/ is created with the incorrect mode
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: resource-agents
Version: 7.5
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: rc
: ---
Assignee: Oyvind Albrigtsen
QA Contact: cluster-qe@redhat.com
Steven J. Levine
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-19 14:53 UTC by Kyle Walker
Modified: 2018-05-09 06:09 UTC (History)
13 users (show)

Fixed In Version: resource-agents-3.9.5-106.el7
Doc Type: Bug Fix
Doc Text:
`/var/run/resource-agents` directory now persists across reboots Previously, the `/var/run/resource-agents` directory, created at installation of the `resource-agents` package, was not persistent across reboots. With this fix, the directory is now present after a reboot.
Clone Of:
Environment:
Last Closed: 2018-04-10 12:07:57 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0757 None None None 2018-04-10 12:08:52 UTC

Description Kyle Walker 2017-06-19 14:53:15 UTC
Description of problem:
 The /var/run/resource-agents/ directory is created with the incorrect mode. During installation it is created correctly, however, with the advent of the tmpfs-backed /run directory, this does not persist across a reboot. This results in "rpm" verification checks, used by a number of system integrity scanning utilities, to fail similar to the following:

    # rpm -V resource-agents
       .M.......    /var/run/resource-agents


Version-Release number of selected component (if applicable):
 resource-agents-3.9.5-82.el7_3.11

How reproducible:
 Easily


Steps to Reproduce:
1. Install resource-agents
2. Restart the system
3. 

Actual results:
 # rpm -V resource-agents
    .M.......    /var/run/resource-agents

Expected results:
 # rpm -V resource-agents  #Nothing returned

Additional info:

Comment 2 Chris Feist 2017-06-19 15:44:24 UTC
Kyle,

Do you have any info on the change to a tmpfs backed /var/run?  Is this a default in RHEL, do you know when this changed?

Thanks,
Chris

Comment 3 Kyle Walker 2017-06-19 15:54:01 UTC
(In reply to Chris Feist from comment #2)
> Kyle,
> 
> Do you have any info on the change to a tmpfs backed /var/run?  Is this a
> default in RHEL, do you know when this changed?
> 

Yep! This is the default for RHEL 7. It was implemented upstream in Fedora 15:

https://docs.fedoraproject.org/en-US/Fedora/15/pdf/Release_Notes/Fedora-15-Release_Notes-en-US.pdf
~~~~
3.2.2. /run directory
Fedora 15 has a /run directory for storing runtime data. /run is now a tmpfs, and /var/run is bind
mounted to it. /var/lock is bind mounted to /run/lock. Applications can use /run the same
way as /var/run. Several programs including udev, dracut, mdadm, mount and initscripts
used hidden directories under /dev for runtime data during early bootup before /var is mounted.
However /dev/ is supposed to be used for only device nodes and there is consensus between major
distributions to shift to using /run instead. Fedora 15 is leading this change. Details including the
benefits are explained here26
.
This change is compliant with the Filesystem Hierarchy Standard27, which allows distributions
to create new directories in the root hierarchy as long as there is careful consideration of the
consequences. Co-author of the latest FHS specification has expressed support28 for this change.
Lennart Poettering has filed a request29 to update the FHS standard to include this change as well.
~~~

In RHEL 7, the /var/run directory is a symlink to the /run base tmpfs directory above:

  # ls -l /var/run
  lrwxrwxrwx. 1 root root 6 Jan 13  2016 /var/run -> ../runrp

More information: 
  https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Migration_Planning_Guide/sect-Red_Hat_Enterprise_Linux-Migration_Planning_Guide-File_System_Layout.html



In keeping with other packages solution to this type of behaviour, I would recommend adding a tmpfiles.d drop-in similar to the following:

   # cat /var/lib/tmpfiles.d/resource-agents.conf 
   d /var/run/resource-agents 1755 root root


That causes systemd-tmpfiles to create the directory with the correct permissions during each reboot iteration.

- Kyle Walker

Comment 6 Jan Pokorný [poki] 2017-06-23 18:38:43 UTC
One thing to note here, pacemaker (lrmd or pacemaker_remoted, to be
specific) will currently happily create that directory if it doesn't
exist already (with mode 0755) on the startup of the respective daemon.

That may be the primary reason of the clash observed with "rpm -V"
as the resource-agents in their "make install" will create that
directory with 1755 (note the restricted deletion flag) that then
gets remembered.


But what comes clear, that directory is a sort of an integration point
between (clustered) resource manager and the resource-agents (i.e.
implementation of some resources using settled "API").

On several occasions already, I've been toying with an idea of
"ocf-standard" (or the like) separate package which would concentrate
all such commons and that would be required by both resources-users
and resources-providers.  Besided common paths, it would also offer
rpm macros for them (to be utilized in the spec files of both parties,
assuming it is also build-required) so there would be simplification
(no hardcoding), and tools like rpmlint would stop complaining about
that.

Such package might be also a good candidate to deliver the tmpfiles.d
config if it's deemed a way forward.

Comment 7 Kyle Walker 2017-06-26 13:07:00 UTC
@Jan,

Just a heads-up. For whichever reason, pacemaker actually creates the directory with 0751 permissions.

    # ls -l /var/run/resource-agents -d
    ls: cannot access /var/run/resource-agents: No such file or directory

    # pcs cluster start

    # ls -l /var/run/resource-agents -d
    drwxr-x--x. 2 root root 40 Jun 26 08:37 /var/run/resource-agents


Not sure it will cause any issues, but it does result in an unexpected permission set.

- Kyle Walker

Comment 8 Jan Pokorný [poki] 2017-06-28 08:40:37 UTC
Kyle, I believe this is because of additional umask enforcement:

lib/common/utils.c: crm_make_daemon: umask(0026)

Although I checked that a difference in restricted deletion flag
alone will provoke that complaint on rpm -V.

Comment 9 Kyle Walker 2017-06-28 14:34:47 UTC
(In reply to Jan Pokorný from comment #8)
> Kyle, I believe this is because of additional umask enforcement:
> 
> lib/common/utils.c: crm_make_daemon: umask(0026)
> 
> Although I checked that a difference in restricted deletion flag
> alone will provoke that complaint on rpm -V.

Ahhh, I missed that. That makes sense. I like the separate package approach with tmpfiles.d configuration file approach.

Thanks for the feedback!

- Kyle Walker

Comment 10 Ken Gaillot 2017-06-28 14:42:55 UTC
Agreed that we should add a tmpfiles.d configuration when systemd is in use. But I think it should stay in the resource-agents package, because the directory is specific to these agents and not the OCF standard. Other agents are free to depend on resource-agents and use the same directory if desired.

For backward compatibility though, pacemaker should still create the directory if it doesn't exist.

Comment 11 Oyvind Albrigtsen 2017-06-28 14:51:25 UTC
(In reply to Ken Gaillot from comment #10)
Makes sense to me, so I'll look into adding it to resource-agents.

Comment 12 Jan Pokorný [poki] 2017-06-28 17:26:02 UTC
> the directory is specific to these agents and not the OCF standard.
> Other agents are free to depend on resource-agents and use the same
> directory if desired.
> 
> For backward compatibility though, pacemaker should still create the
> directory if it doesn't exist.

True, "RSCTMPDIR" is currently not a part of OCF standard...
But I am the only one to see the internal conflict between
resource-agents originally going thus far to make the resource
managers (originating in heartbeat, it seems) create 
semi-standard directory for them ... and just for them (because
again, OCF does not specify such a directory and the connected
requirements on it)?

General OCF adherent audience would appreciate such an optional
benefit like having predestined directory to store tempory data,
I believe.  And there are systems that have no equivalent of
systemd-tmpfiles (e.g. tmpwatch is just a cleaner) generaly handy.

That would also alleviate the schizophrenia where some of RAs
create that directory on their own -- but not all!

Also note that restricted deletion flag makes little sense for
anything running as root (which is a default mode of operation
for RAs -- there is hardly a priviledge separation to enforce).


So I see at least two other alternatives:

1. each agent is responsible for managing "RSCTMPDIR" on its own,
   there's no need for any sort of babysitting then
   (tmpfiles, OCF, not even in pacemaker + specfile of resource-agents
   should read: "%dir %verify(not mode) %{_var}/run/resource-agents")

2. OCF standard actually becomes the enforcing point for that
   directory, it's semantics and requirements imposed on it
   (for both providers and users)

Comment 13 Jan Pokorný [poki] 2017-06-28 17:32:21 UTC
Note that both 1. and 2. are more systemic than adding 4th(!) possible
originator of that directory:

- installation of resource-agents package (only until the next reboot
  if the FHS semantics are preserved)
- pacemaker
- agent itself
- tmpfiles.d config

Comment 14 Jan Pokorný [poki] 2017-06-29 13:45:13 UTC
So I rechecked the Fedora packaging guidelines and related sources
and slightly shifted my opinion:

Initial considerations:
- we are discussing RHEL 7, which branched from Fedora 18
- "/var/run and /var/lock on tmpfs" change introduced in Fedora 15
  https://fedoraproject.org/wiki/Features/var-run-tmpfs
- respective packaging guidelines:
  https://fedoraproject.org/wiki/Packaging:Tmpfiles.d

Advantages of using tmpfiles.d mechanism:
- it gets SELinux labels right out of the box (not that it currently
  matters, but could/should)
- it's supposed to work correctly with RPM files tracking
  (it's unlikely it would ever run prior to systemd-tmpfiles
   kicking in) and listing of the such files/dirs is even encouraged

Disadvantes:
- tmpfiles.d mechanism is not generally available in containers
  (assuming there is an effort to make do without systemd in it when
  using pacemaker-remote?) ... but see the initscript equivalent idea
  bellow

* * *

So while in [comment 13] I opposed the idea of multi-sourced
birth of the directory at hand (complexity is nasty, right?),
tmpfiles.d config now looks as well-fitting in the wider system
perspective, and we can perhaps plan on cutting other ones:

- in pacemaker:
  as mentioned, that directory was never standardized (it's strange
  to embed a non-standard use-case specific support silently);
  if it gets (under OCF), it should be some equivalent of proposed
  ocf-standard package that will eventually deliver that tmpfiles.d
  config (and simple initscript for daemons like pacemaker to depend
  on with non-systemd systems)

- in agents themselves:
  they should start taking existence of that directory for granted; at
  the same time, it should become a constant, not an externally
  influencable variable (precondition for the former to work reliably)
  (and we have bind mounts, chroots and containers for extravagant
  corner cases requiring a use of the different effective path)

Comment 16 Oyvind Albrigtsen 2017-09-01 11:35:37 UTC
https://github.com/ClusterLabs/resource-agents/pull/1031

Comment 18 Monika Muzikovska 2017-12-05 11:27:39 UTC
Marking Verified in version resource-agents-3.9.5-116.el7.x86_64. /var/run/resource-agents is now present after reboot.


1) BEFORE fix - /var/run/resource-agents is missing after reboot
=============
# rpm -q resource-agents
resource-agents-3.9.5-105.el7_4.2.x86_64

# reboot

# ls /var/run | grep resource-agents

# rpm -V resource-agents
missing     /var/run/resource-agents


2) AFTER fix - /var/run/resource-agents is now present after reboot
=============
# rpm -q resource-agents
resource-agents-3.9.5-116.el7.x86_64

# reboot

# ls /var/run | grep resource-agents
resource-agents

# rpm -V resource-agents

Comment 21 errata-xmlrpc 2018-04-10 12:07:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0757


Note You need to log in before you can comment on or make changes to this bug.