Hi,

Petr is on holiday this and next week, so he won't be able to answer for some time.

I have looked into the fix - it changes arguments of functions which are public (dst_key_t * -> const dst_key_t *), so IMO it may break API - so IMO in any case this will require coordination with at least with users of the library, so no simple merge/build would do the trick in my humble opinion.

noted, thx.

will push temporary fixes here to dial down log levels on production.
not ideal for monitoring, but better than the massive logs.

Oh, this should be fixed by 9.18.38, recently released. Would be fixed by the rebase together with it, there are just minimal changes in 9.18 version anyway.

Although the problem is not too hard to backport, it just exports dns_keymgr_key_may_be_purged previously just static function and uses it also in zone.c to skip already processed keys. With const part adding avoidable, if needed.

Have not tested, but might be avoided by using different directory for storing keys on different views. Can you try key-directory in view {} block pointing to separate directory? Is it expected to reuse the same keys for different views?

Also we do not care for bind API anymore and changes in function signature is not a problem. Bind libraries are considered implementation detail since 9.16.

> Can you try key-directory in view {} block pointing to separate directory? Is it expected to reuse the same keys for different views?

i _already_ have here

	view "external" {
		allow-new-zones true;
		match-clients {
			...
		};

		key-directory "keys/dnssec"; // fallback

as well as in each zone, e.g.,

	zone "example.com" IN {
		type master; file "namedb/primary/example.com.zone";
		dnssec-policy "lan01ns";
		inline-signing yes;
		parental-agents { "parental_agents"; };
		key-directory "keys/dnssec/example.com"; // per-zone
		update-policy {
			grant local-ddns zonesub any;
			grant xxx-rndc-key zonesub txt;
		};
	};


where

	ls -1 keys/dnssec/example.com/
		Kexample.com.+013+11111.key
		Kexample.com.+013+11111.private
		Kexample.com.+013+11111.state
		Kexample.com.+013+22222.key
		Kexample.com.+013+22222.private
		Kexample.com.+013+22222.state
		Kexample.com.+013+33333.key
		Kexample.com.+013+33333.private
		Kexample.com.+013+33333.state


> Oh, this should be fixed by 9.18.38, recently released. Would be fixed by the rebase together with it

do you have a rough ETA for those builds @ Fedora 42 pkgs?

FEDORA-2025-0db3384d26 (bind-9.18.38-1.fc44 and bind-dyndb-ldap-11.11-6.fc44) has been submitted as an update to Fedora 44.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-0db3384d26

FEDORA-2025-0db3384d26 (bind-9.18.38-1.fc44 and bind-dyndb-ldap-11.11-6.fc44) has been pushed to the Fedora 44 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-2025-b223648ed1 (bind-9.18.38-1.fc43 and bind-dyndb-ldap-11.11-6.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-b223648ed1

FEDORA-2025-470bc2dd8a (bind-9.18.38-1.fc42 and bind-dyndb-ldap-11.11-5.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-470bc2dd8a

FEDORA-2025-b223648ed1 (bind-9.18.38-1.fc43 and bind-dyndb-ldap-11.11-6.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.

> FEDORA-2025-470bc2dd8a (bind-9.18.38-1.fc42 and bind-dyndb-ldap-11.11-5.fc42) has been submitted as an update to Fedora 42.
> https://bodhi.fedoraproject.org/updates/FEDORA-2025-470bc2dd8a

with
```
rpm -qa bind
	bind-9.18.38-1.fc42.x86_64
```

from https://koji.fedoraproject.org/koji/buildinfo?buildID=2804029

and
```
cat named.conf
	logging {
	  channel loglevel_critical { severity critical; syslog; print-category yes; print-severity yes; print-time yes; };
	...
	  channel loglevel_notice   { severity notice;   syslog; print-category yes; print-severity yes; print-time yes; };

-	  category dnssec          { loglevel_critical; };
+	  category dnssec          { loglevel_notice;   };
```

logs still _stream_
```
...
025-08-20T07:34:57.578146-04:00 test named[49058]: 20-Aug-2025 07:34:57.577 dnssec: error: zone example.com/IN/external (signed): zone_rekey:zone_verifykeys failed: some key files are missing
2025-08-20T07:34:57.578765-04:00 test named[49058]: 20-Aug-2025 07:34:57.577 dnssec: error: zone example.com/IN/external (signed): zone_rekey:zone_verifykeys failed: some key files are missing
2025-08-20T07:34:57.579396-04:00 test named[49058]: 20-Aug-2025 07:34:57.578 dnssec: error: zone example.com/IN/external (signed): zone_rekey:zone_verifykeys failed: some key files are missing
2025-08-20T07:34:57.580031-04:00 test named[49058]: 20-Aug-2025 07:34:57.579 dnssec: error: zone example.com/IN/external (signed): zone_rekey:zone_verifykeys failed: some key files are missing
2025-08-20T07:34:57.580651-04:00 test named[49058]: 20-Aug-2025 07:34:57.579 dnssec: error: zone example.com/IN/external (signed): zone_rekey:zone_verifykeys failed: some key files are missing
2025-08-20T07:34:57.581290-04:00 test named[49058]: 20-Aug-2025 07:34:57.580 dnssec: error: zone ```

back to
```
grep "category dnssec" named.conf
-	  category dnssec          { loglevel_notice;   };
+	  category dnssec          { loglevel_critical; };
```

all's good

Ah, so the upstream fix in linked issue is not enough for your issue? I verified twice it were included in this release.

Could you please change notice to dynamic severity:

channel loglevel_notice   { severity dynamic; syslog; print-category yes; print-severity yes; print-time yes; };

or just adding debug output to dnssec category too.

category dnssec          { loglevel_notice; default_debug; };

Then try "rndc trace 1" command enabled for a while, and so whatever it takes to make these verifykeys messages spawn.
/var/named/data/named.run is default_debug target file log.

Then disable it by "rndc notrace" command again.

That should at least share what key it thinks is missing. Should emit some debug(1) lines with messages like: verifykeys: key %s - not available

Then use dnssec-settime -pall <keyfile> to display, what problem it might have with it.

It might help if you had those keys separated. Like:

		key-directory "keys/dnssec/external"; // fallback

...

	zone "example.com" IN {
		type master; file "namedb/primary/example.com.zone";
		dnssec-policy "lan01ns";
		inline-signing yes;
		parental-agents { "parental_agents"; };
		key-directory "keys/dnssec/external/example.com"; // per-zone
		update-policy {
			grant local-ddns zonesub any;
			grant xxx-rndc-key zonesub txt;
		};
	};

The disadvantage would be the need to have twice DS records at parent zone. One for external, one for internal view. But should keep those files separated and do not clash and log this way.

I do not see anything obvious, why it should complain. Or why upstream fix did not help.

Another workaround might be to set purge-keys 0; in your dnssec-policy. It would stash old and unused hidden keys, but should prevent these logs.

Maybe it just prevented the non-consistent state creation, but does not fix the keys when this state already happened. Although it seems to be just internal state of named.service and just restart should make it well defined.

the "rndc trace 1" showed _all_ my zones reporting the same problem, with the new installed rpm.

looking at the upstream fix, iiuc, it should do the trick.

noting that i'd simply intalled this updated rpm over existing/current pkgs, i UNINSTALLED all, purged caches, etc, and RE-INSTALLED the updated from my COPR build of the latest rawhide src, for f42.

and, NOW, it works.  log spam appears to be gone.

FEDORA-2025-470bc2dd8a has been pushed to the Fedora 42 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-470bc2dd8a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-470bc2dd8a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

so does it fix your issue in the end? Did you delete also any of dnssec keys manually or anything in /var/named explicitly? The rawhide and f42 builds should differ only in missing mass F43 rebuild spec bump. There should not be any need to purge caches if you had already 9.18.38 version, whether from f42 or rawhide branch. It should behave the same way.

"rndc trace 1" should report all missing keys, which are not yet ready to be purged. I expect unless they were removed by some external tool on the filesystem, it should not happen at all.

Default for purge-keys value is P90D, meaning 90 days. Do you use other value?

This issue might have happened if you used shorter period and changed it to longer value afterwards. But keys already deleted should not be read again on named restart. It might cause some errors logged if value changed only after "rndc reconfig" command. Then the missing keys could be still in memory, but with different period for pruning execution.

Uninstallation and fresh installation should have not changed anything. These are changes made by named into files the package does not touch or modify. The only thing which may affect it would be no proper restart after the package upgrade. Did you change your configuration somehow between reinstallation?

Can you please try upgrade to updates-testing package and check named.service is restarted after the package installation? Scriptlets seem to be okay. Do you use bind-chroot package maybe?

> so does it fix your issue in the end?

yes. specifically, i no longer see the log spam with

-	category dnssec          { loglevel_critical;  };
+	category dnssec          { loglevel_notice;   };


> Did you delete also any of dnssec keys manually or anything in /var/named explicitly?

no.  i did not touch any of the keys. nor in /var/named

> "rndc trace 1" should report all missing keys, which are not yet ready to be
purged. I expect unless they were removed by some external tool on the
filesystem, it should not happen at all.

before the cleanup -- uninstall & reinstall -- it streamed log entries for all my zones' keys.

and, toggling between `loglevel_critical` and `loglevel_notice` for dnssec loglevel would control that ...

after, the reinstall, same test -- no streaming of these errors at either loglevel.

> Default for purge-keys value is P90D, meaning 90 days. Do you use other value?

yes.

grep purge-keys named.conf
        purge-keys                 P30D;

> This issue might have happened if you used shorter period and changed it to
longer value afterwards.

fwiw, i did not change that value.

> only thing which may affect it would be no proper restart after the package upgrade.

i definitely did a manual stop/start of named after the install.

> Did you change your configuration somehow between reinstallation?

nope. not at all.  well, other than toggling the loglevel back and forth ...

> Do you use bind-chroot package maybe?

no.

> Can you please try upgrade to updates-testing package and check named.service
is restarted after the package installation?

with my current 'clean reinstall' from my copr repo,

rpm -qa | grep ^bind | sort
  bind-9.18.38-1.fc42.x86_64
  bind-chroot-9.18.38-1.fc42.x86_64
  bind-devel-9.18.38-1.fc42.x86_64
  bind-dnssec-utils-9.18.38-1.fc42.x86_64
  bind-libs-9.18.38-1.fc42.x86_64
  bind-utils-9.18.38-1.fc42.x86_64

ps ax | grep named
  56366 ?        Ssl    0:20 /usr/sbin/named -n 2 -S 1024 -u named -c /usr/local/etc/named/etc/named.conf

all is good again.

swapping in the release from `updates-testing`

  https://bodhi.fedoraproject.org/updates/FEDORA-2025-470bc2dd8a

instead, and restarting, there's no change.  still good.

_something_ in that cleanup/reinstall procedure made a difference.
i'll be darned if i can figure out _what_ :-/

i'll likely double-check with 9.20x in a bit.  i assume that's on its way to pkg release sometime soon ...

It might be suppressed because expired keys have been already purged. Then after the restart it won't print anything, even if the race would remain. It would have to wait until another key file is purged by named just from one view, IMO.

Unless there are already missing key files for signed zones in general.

You use unusual path to main configuration file. Are you sure that path did not change with different version of the package? Do you have all local modifications to named.service in /etc/systemd/systemd/named.service* ? There were no recent changes to service file, but might affect you somehow. Unit files displayed by systemctl cat named would rewrite any changes done in /usr/lib/systemd.

> You use unusual path to main configuration file.

it's simplyexplicitly specified:

cat /etc/systemd/system/named.service
	...
	ExecStart=/usr/sbin/named -n 2 -S 1024 -u named -c /usr/local/etc/named/etc/named.conf
	...

> Are you sure that path did not change with different version of the package?

yes.  again, it's explicitly specified.

if the path weren't recognized by any running instance, then _nothing_ would work as intended.
instead, every works as intended/expected.  the only issue was the (prior) log spam.

> Do you have all local modifications to named.service in /etc/systemd/systemd/named.service*

yes. it's a FULL service file, not an override

anyway, working now.

FEDORA-2025-470bc2dd8a (bind-9.18.38-1.fc42 and bind-dyndb-ldap-11.11-5.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.