Actually this is not a false negative, it's just that rpm never verified *any* aspects of %ghost files and as a side-effect of bug 1406611 it now does. Mode, which is the failing thing with ca-certificates, is one of the few things of %ghost files that *can* be meaningfully verified. It's ca-certificates whose 
permissions differ between the package and what gets extracted (this is from Fedora package so details might differ, the issue is the same):

[root@sopuli ~]# ls -l /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt-r--r--r--. 1 root root 261947 Feb 21 10:22 /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
[root@sopuli ~]# rpm -qlv ca-certificates|grep /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt|head -1
-rw-r--r--    1 root    root                        0 Feb  6 15:48 /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt

And that's why rpm now complains. The problem is that --noghost doesn't work so you can't easily filter it out.

See above, not a bug in rpm but intentional behavior change. So it's the affected packages that need fixing, reassigning to ca-certificates.

*** Bug 1583796 has been marked as a duplicate of this bug. ***

The differences aren't limited to the file permission.

The sizes differ, too, because we package an empty file. After installation, the file contents is dynamically created based on system configuration. So it's impossible to package the exact state the file will have on the configured system.

I believe you are asking for a fix to the file permission. Currently, one side has "user rw", and the other side has "user r" permissions.

I locally used chmod 644 to change the permissions of the file to the permissions that are used inside the package.

Afterwards, rpm --verify still complains that the file was modified.

Is the request of this bug that "rpm --verify ca-certificates" should return no differences? I think this cannot be achieved.

That's strange. I cannot duplicate your result:

bash# rpm --verify -q ca-certificates
.M.......  g /etc/pki/ca-trust/extracted/java/cacerts
.M.......  g /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
.M.......  g /etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
.M.......  g /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
.M.......  g /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
bash# chmod 0644 /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
bash# rpm --verify -q ca-certificates
.M.......  g /etc/pki/ca-trust/extracted/java/cacerts
.M.......  g /etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
.M.......  g /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
.M.......  g /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

Notice that it no longer complains about the file whose mode I fixed.
What do you see?

Regards,
Andy

Thanks for doublechecking. Yes indeed, the one fixed file is no longer being reported as modified, I hadn't looked carefully enough.

I can get that fixed in Rawhide soon. What's the priority to get this fixed elsewhere?

(In reply to Kai Engert (:kaie) from comment #5)
> The differences aren't limited to the file permission.
> 
> The sizes differ, too, because we package an empty file. After installation,
> the file contents is dynamically created based on system configuration. So
> it's impossible to package the exact state the file will have on the
> configured system.

Obviously actual content checks, including size, do not make sense for %ghost files and thus they are skipped automatically. However when the file is actually present, mode, user+group can sanely be verified even for %ghost's, and making sure they match between seems like a good thing to avoid eg security-related surprises.

Fixed for Fedora Rawhide 29:
  ca-certificates-2018.2.24-3.fc29
  https://koji.fedoraproject.org/koji/taskinfo?taskID=27414322

Experimental update for Fedora 28:
  ca-certificates-2018.2.24-1.1.fc28
  https://koji.fedoraproject.org/koji/taskinfo?taskID=27414698

*** Bug 1571236 has been marked as a duplicate of this bug. ***

*** Bug 1622251 has been marked as a duplicate of this bug. ***

FYI, this is still broken in 7.6:

sh-4.2# rpm -q ca-certificates
ca-certificates-2018.2.22-70.0.el7_5.noarch
sh-4.2# rpm --verify -q ca-certificates
.M.......  g /etc/pki/ca-trust/extracted/java/cacerts
.M.......  g /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
.M.......  g /etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
.M.......  g /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
.M.......  g /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
sh-4.2# rpm -qlv ca-certificates | awk '$5 == 0 && $9 ~ /^\/etc\/pki\/ca-trust\/extracted.*(\.pem|crt|cacerts)/'
-rw-r--r--    1 root    root                        0 May 16  2018 /etc/pki/ca-trust/extracted/java/cacerts
-rw-r--r--    1 root    root                        0 May 16  2018 /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
-rw-r--r--    1 root    root                        0 May 16  2018 /etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
-rw-r--r--    1 root    root                        0 May 16  2018 /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
-rw-r--r--    1 root    root                        0 May 16  2018 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
sh-4.2# rpm -qlv ca-certificates | awk '$5 == 0 && $9 ~ /^\/etc\/pki\/ca-trust\/extracted.*(\.pem|crt|cacerts)/ {print $9}' | xargs ls -ld
-r--r--r-- 1 root root 154279 May 19  2018 /etc/pki/ca-trust/extracted/java/cacerts
-r--r--r-- 1 root root 257889 May 19  2018 /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
-r--r--r-- 1 root root 177221 May 19  2018 /etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
-r--r--r-- 1 root root      0 May 19  2018 /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
-r--r--r-- 1 root root 211658 May 19  2018 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

Regards,
Andy

This issue was not selected to be included either in Red Hat Enterprise Linux 7.7 because it is seen either as low or moderate impact to a small amount of use-cases. The next release will be in Maintenance Support 1 Phase, which means that qualified Critical and Important Security errata advisories (RHSAs) and Urgent Priority Bug Fix errata advisories (RHBAs) may be released as they become available. We will now close this issue, but if you believe that it qualifies for the Maintenance Support 1 Phase, please re-open; otherwise we recommend moving the request to Red Hat Enterprise Linux 8 if applicable.

@Simo,

I'm re-opening this bug report, please feel free to reach out to me directly if you would like to discuss directly. The issue here is actually _not_ a low priority instance. The RPM attributes mismatching result in a High priority STIG compliance issue due to the following:

    https://rhel7stig.readthedocs.io/en/latest/high.html#v-71849-the-file-permissions-ownership-and-group-membership-of-system-files-and-commands-must-match-the-vendor-values-rhel-07-010010

    Warning: The above link is to a source that are not authored by Red Hat directly. As such, Red Hat support cannot verify its accuracy and content.

Any STIG secure environment will fail the above high priority check, which we ship in our oscap-security-guide validation mechanism.

(In reply to Kyle Walker from comment #21)
> @Simo,
> 
> I'm re-opening this bug report, please feel free to reach out to me directly
> if you would like to discuss directly. The issue here is actually _not_ a
> low priority instance. The RPM attributes mismatching result in a High
> priority STIG compliance issue due to the following:
> 
>    
> https://rhel7stig.readthedocs.io/en/latest/high.html#v-71849-the-file-
> permissions-ownership-and-group-membership-of-system-files-and-commands-must-
> match-the-vendor-values-rhel-07-010010
> 
>     Warning: The above link is to a source that are not authored by Red Hat
> directly. As such, Red Hat support cannot verify its accuracy and content.
> 
> Any STIG secure environment will fail the above high priority check, which
> we ship in our oscap-security-guide validation mechanism.

Speaking as someone who has been filing Bugzilla bugs and support requests over RPM attribute mismatches for several years now, for the exact same reason (violations of STIG ID RHEL-07-010010, which we are required to care about), I 100% agree.

But the problem is deeper than just a single package:

$ sudo stig-check -s STIG-RHEL7-V2R2 -v
RHEL-07-010010,CAT I,permissions/owner/group-owner of /etc/sssd/sssd.conf differ (.....UG..) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /run/lsm differ (.M....G..) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /run/lsm/ipc differ (.M....G..) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /usr/sbin/netreport differ (.M.......) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /usr/sbin/usernetctl differ (.M.......) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /var/lib/PackageKit/transactions.db differ (.M.......) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /var/lock/iscsi differ (.M.......) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /var/lock/iscsi/lock differ (.M.......) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /var/log/dmesg differ (.M.......) from RPM database
RHEL-07-010010,CAT I,permissions/owner/group-owner of /var/log/dmesg.old differ (.M.......) from RPM database

(stig-check is a utility we wrote internally, because we needed to be able to perform a scan-load-scan process and diff the results in a consistent and automated manner, and we couldn't figure out how to do that with oscap.)

Depending when in the point release cycle I file the Bugzilla or support request, it can take close to a year to get a mismatched package corrected.

Even worse, Red Hat seems to create new packages with mismatched attributes just as quickly (if not more so) than I can get known packages corrected. For example, sssd was not previously a violation, but RHEL 7.5 updated to sssd-1.16.0-19.el7, which repackaged certain files formerly owned by root:root to sssd:sssd, which will always violate RHEL-07-010010. (/etc/sssd/sssd.conf is packaged as sssd:sssd, but unless it is owned by root:root, sssd refuses to start!)

To give credit where credit is due, out of all Linux distributions, Red Hat Enterprise Linux is probably the best in terms of caring about customers who require SSH and/or STIG compliance. For that, we are grateful.

But there is still much room for improvement here. The fact that Red Hat keeps creating *new* packages with RPM attribute mismatches indicates clearly that either the package QA process doesn't check RPM attributes at all, or else the QA process does check, but no one cares.

Testing for RPM attribute mismatches needs to be a component of the package QA process, and a violation (e.g., your package fails "rpm --verify" once installed/activated on a host) must cause the candidate package to fail the QA process. Then (and only then) will packagers comply, and stop producing packages with RPM attribute mismatches.

Bob any chance we can include this bug in the ca-certificate rebases planned for Q3 ?

They aren't in my tree ready to push, but since we haven't got the acks on the required bugs, but I can certainly add them. (Except rhel 7.4 which is already pushed). It was pushed from rhel 7.7 because of QA resources.