Bug 2185874

The problem is this file is referenced by other packages, which do not implement any kind of choosing either vanilla shipped file or modified file. That file IS variable file and the location for it is correct.

What is in /var/lib/unbound/root.key is after the installation is DNSSEC trust anchor provided by the package. But unbound-anchor.service implements RFC 5011 algorithm to keep this file updated with any updated keys even if the package update will not arrive. It implements secure way to update to the new key automatically. All you need to do is using DNSSEC enabled DNS servers.

unbound-anchor.service is enabled automatically after the installation and is recommended by unbound-libs. It were required part of unbound-libs before, I made it possible to not install it. I have tried to address it in bug #2132103. That root.key is replaced on the first successful start of unbound-anchor with a cached data, where it maintains timestamps.
I have made the /etc/unbound/dnssec-root.key stay in /etc, so even the packaged default stays somewhere.

But the initial link in /var/lib/unbound/root.key is NOT an error. It is intended as a compromise for packages like libreswan, which can use unbound-libs ability to validate DNSSEC. It should always contain working DNSSEC anchor, either maintained or link to the static one. Can you suggest where else should be maintained content stored than in /var?

That file is also used by ldns-utils utilities like drill.

I guess you can remove that file during the container build if it cannot cope with that path. I don't know a better way to have it prepared for command line utilities and also managed by unbound.service or unbound-anchor.service. If it does not exist, unbound-anchor.service will create it. But CLI tools should not depend on manual running of unbound-anchor.service, se we want it present for normal distribution as it is. "drill -S" command needs that for example.

The only solution I found is creating that link from %post, so it is created on non-containerized distributions only. Is that what you would like?

If I use fedora-coreos:stable and rpm-ostree install unbound-utils ldns-utils, I am able to use for example this:

# unbound-host -D getfedora.org
[1685743242] libunbound[267:0] error: error opening file /var/lib/unbound/root.key: No such file or directory
[1685743242] libunbound[267:0] error: error reading trust-anchor-file: /var/lib/unbound/root.key
[1685743242] libunbound[267:0] error: validator: error in trustanchors config
[1685743242] libunbound[267:0] error: validator: could not apply configuration settings.
[1685743242] libunbound[267:0] error: module init for module validator failed
resolve error: initialization failure

# drill -S getfedora.org
;; Chasing: getfedora.org. A
Warning: No trusted keys specified


DNSSEC Trust tree:
getfedora.org. (A)
|---getfedora.org. (DNSKEY keytag: 10147 alg: 14 flags: 256)
    |---getfedora.org. (DNSKEY keytag: 23753 alg: 14 flags: 257)
    |---getfedora.org. (DS keytag: 23753 digest type: 1)
    |   |---org. (DNSKEY keytag: 33369 alg: 8 flags: 256)
    |       |---org. (DNSKEY keytag: 26974 alg: 8 flags: 257)
    |       |---org. (DS keytag: 26974 digest type: 2)
    |           |---. (DNSKEY keytag: 60955 alg: 8 flags: 256)
    |               |---. (DNSKEY keytag: 20326 alg: 8 flags: 257)
    |---getfedora.org. (DS keytag: 23753 digest type: 2)
        |---org. (DNSKEY keytag: 33369 alg: 8 flags: 256)
            |---org. (DNSKEY keytag: 26974 alg: 8 flags: 257)
            |---org. (DS keytag: 26974 digest type: 2)
                |---. (DNSKEY keytag: 60955 alg: 8 flags: 256)
                    |---. (DNSKEY keytag: 20326 alg: 8 flags: 257)
You have not provided any trusted keys.
;; Chase successful

The above happens when I move /var/lib/unbound/root.key away.
If I move it back:

# drill -S getfedora.org
;; Number of trusted keys: 1
;; Chasing: getfedora.org. A


DNSSEC Trust tree:
getfedora.org. (A)
|---getfedora.org. (DNSKEY keytag: 10147 alg: 14 flags: 256)
    |---getfedora.org. (DNSKEY keytag: 23753 alg: 14 flags: 257)
    |---getfedora.org. (DS keytag: 23753 digest type: 1)
    |   |---org. (DNSKEY keytag: 33369 alg: 8 flags: 256)
    |       |---org. (DNSKEY keytag: 26974 alg: 8 flags: 257)
    |       |---org. (DS keytag: 26974 digest type: 2)
    |           |---. (DNSKEY keytag: 60955 alg: 8 flags: 256)
    |               |---. (DNSKEY keytag: 20326 alg: 8 flags: 257)
    |---getfedora.org. (DS keytag: 23753 digest type: 2)
        |---org. (DNSKEY keytag: 33369 alg: 8 flags: 256)
            |---org. (DNSKEY keytag: 26974 alg: 8 flags: 257)
            |---org. (DS keytag: 26974 digest type: 2)
                |---. (DNSKEY keytag: 60955 alg: 8 flags: 256)
                    |---. (DNSKEY keytag: 20326 alg: 8 flags: 257)
;; Chase successful

# unbound-host -D getfedora.org
getfedora.org has address 18.192.40.85
getfedora.org has address 18.133.140.134
getfedora.org has address 18.159.254.57
getfedora.org has address 38.145.60.20
getfedora.org has address 152.19.134.142
getfedora.org has address 85.236.55.6
getfedora.org has address 38.145.60.21
getfedora.org has address 185.141.165.254
getfedora.org has address 152.19.134.198
getfedora.org has IPv6 address 2605:bc80:3010:600:dead:beef:cafe:fed9
getfedora.org has IPv6 address 2001:4178:2:1269::fed2
getfedora.org has IPv6 address 2600:2701:4000:5211:dead:beef:fe:fed3
getfedora.org has IPv6 address 2604:1580:fe00:0:dead:beef:cafe:fed1
getfedora.org has IPv6 address 2a05:d014:10:7803:f774:4d7c:e277:a457
getfedora.org has IPv6 address 2a05:d01c:c6a:cc01:269:da52:9ae1:43e6
getfedora.org mail is handled by 10 mx1.redhat.com.
getfedora.org mail is handled by 20 mx2.redhat.com.

So it can be used even on CoreOS and has some reason to remain there. Can you propose a solution, which would not break this in order to make "ostree container commit" pass without error? Is there some ignore list, where that file could be added to be accepted or just ignored?

Thank you to have deepened. I'm not aware of any ignore method to skip /var checking in the building process.  Unfortunately I don't have suggestion about the issue. 

In addition to those listed (cockpit-*), many other packages like NetworkManage-ovs make modify root.key file and thanks to your details it's reasonable.

But In the context of containerized distributions I don't have any solution to propose.

Due to recurrence of the issue in many other packages I adopted a simple workaround removing or moving file before "ostree container commit" to guarantee compatibility with /var content on commit.

Workaround example:

# omitted ...
RUN set -x; PACKAGES_INSTALL="NetworkManager-ovs"; \
      rpm-ostree install $PACKAGES_INSTALL \
      && rpm-ostree cleanup -m \
      && rm -rf /var/lib/unbound/root.key \
      && ostree container commit


But of course it's a workaround and not a solution

> In addition to those listed (cockpit-*), many other packages like
> NetworkManage-ovs make modify root.key file and thanks to your details it's
> reasonable.

Other packages should not be modifying root.key. Only unbound-anchor or unbound daemon itself should overwrite that file.
If another package is doing that, please file a bug

Closing the bug. I have not found any way to make it more correct and still working. Please reopen with some proposal, how should it be fixed better.