+++ This bug was initially created as a clone of Bug #1705979 +++ Description of problem: The /var/lib/tftpboot/boot is growing a lot, it adds a lot of new files: - The old files with the old naming convention are still kept, should those not be deleted if not used anymore? - The files with the same content are not hardlinked taking up a lot of space. Example: # du -h /var/lib/tftpboot/boot 1.4G /var/lib/tftpboot/boot # ls -l /var/lib/tftpboot/boot total 1365152 -rw-r--r--. 1 foreman-proxy foreman-proxy 41039717 Nov 9 16:20 RedHat-6.10-x86_64-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 4316752 Nov 9 16:20 RedHat-6.10-x86_64-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 41411274 Nov 10 03:02 RedHat-6.9-x86_64-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 4276496 Nov 10 03:02 RedHat-6.9-x86_64-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 44701632 Nov 10 03:12 RedHat-7.3-x86_64-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5391264 Nov 10 03:12 RedHat-7.3-x86_64-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 49763300 Nov 9 17:11 RedHat-7.4-x86_64-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5875184 Nov 9 17:11 RedHat-7.4-x86_64-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54525200 Nov 9 17:24 RedHat-7.5-x86_64-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6381872 Nov 9 17:24 RedHat-7.5-x86_64-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 RedHat-7.6-x86_64-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 RedHat-7.6-x86_64-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 41039717 Nov 9 16:20 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-534-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 4316752 Nov 9 16:20 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-534-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 41039717 Nov 9 16:20 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-556-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 4316752 Nov 9 16:20 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-556-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 41039717 Nov 9 16:20 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-583-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 4316752 Nov 9 16:20 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-583-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 44701632 Nov 10 03:12 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-616-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5391264 Nov 10 03:12 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-616-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 44701632 Nov 10 03:12 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-652-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5391264 Nov 10 03:12 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-652-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 44701632 Nov 10 03:12 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-688-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5391264 Nov 10 03:12 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-688-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 49763300 Nov 9 17:11 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-4-795-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5875184 Nov 9 17:11 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-4-795-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54525200 Nov 9 17:24 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-5-891-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6381872 Nov 9 17:24 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-5-891-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1027-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1027-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1060-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1060-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1096-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1096-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1148-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1148-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-13132-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-13132-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1318-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1318-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-4300-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-4300-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-5674-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-5674-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-923-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-923-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-963-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-963-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-989-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Nov 10 03:14 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-989-vmlinuz # md5sum /var/lib/tftpboot/boot/* | sort 1ff03802bb4b852a95f191a5c0247df7 /var/lib/tftpboot/boot/RedHat-7.5-x86_64-initrd.img 1ff03802bb4b852a95f191a5c0247df7 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-5-891-initrd.img 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/RedHat-7.6-x86_64-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1027-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1060-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1096-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1148-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-13132-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1318-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-4300-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-5674-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-923-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-963-vmlinuz 36eff2be463ab8e70518c802647a96c5 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-989-vmlinuz 431c9c6e39fce34ffc3eb93df21e865a /var/lib/tftpboot/boot/RedHat-7.5-x86_64-vmlinuz 431c9c6e39fce34ffc3eb93df21e865a /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-5-891-vmlinuz ad945bbd0365fcab439a8ab65a133881 /var/lib/tftpboot/boot/RedHat-6.10-x86_64-vmlinuz ad945bbd0365fcab439a8ab65a133881 /var/lib/tftpboot/boot/red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-534-vmlinuz ad945bbd0365fcab439a8ab65a133881 /var/lib/tftpboot/boot/red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-556-vmlinuz ad945bbd0365fcab439a8ab65a133881 /var/lib/tftpboot/boot/red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-583-vmlinuz b113220e7b9f392f79a11ade59c3ba52 /var/lib/tftpboot/boot/RedHat-6.10-x86_64-initrd.img b113220e7b9f392f79a11ade59c3ba52 /var/lib/tftpboot/boot/red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-534-initrd.img b113220e7b9f392f79a11ade59c3ba52 /var/lib/tftpboot/boot/red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-556-initrd.img b113220e7b9f392f79a11ade59c3ba52 /var/lib/tftpboot/boot/red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-583-initrd.img c48b15aa91c08801a72063a54788686f /var/lib/tftpboot/boot/RedHat-6.9-x86_64-vmlinuz cd156a477a7584d108569673df46de64 /var/lib/tftpboot/boot/RedHat-6.9-x86_64-initrd.img e0342f5226d7f35b2b12fb5f5506c13f /var/lib/tftpboot/boot/RedHat-7.3-x86_64-initrd.img e0342f5226d7f35b2b12fb5f5506c13f /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-616-initrd.img e0342f5226d7f35b2b12fb5f5506c13f /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-652-initrd.img e0342f5226d7f35b2b12fb5f5506c13f /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-688-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/RedHat-7.6-x86_64-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1027-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1060-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1096-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1148-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-13132-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1318-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-4300-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-5674-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-923-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-963-initrd.img e15958ee629978a716218bc3595bcfef /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-989-initrd.img f0bc285eb44f9e363d1327f99d28d976 /var/lib/tftpboot/boot/RedHat-7.4-x86_64-initrd.img f0bc285eb44f9e363d1327f99d28d976 /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-4-795-initrd.img f15a0fb05249b3d1daa46ec179e9928e /var/lib/tftpboot/boot/RedHat-7.4-x86_64-vmlinuz f15a0fb05249b3d1daa46ec179e9928e /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-4-795-vmlinuz f3558ce1e32775a86d9b367a11c58ccc /var/lib/tftpboot/boot/RedHat-7.3-x86_64-vmlinuz f3558ce1e32775a86d9b367a11c58ccc /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-616-vmlinuz f3558ce1e32775a86d9b367a11c58ccc /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-652-vmlinuz f3558ce1e32775a86d9b367a11c58ccc /var/lib/tftpboot/boot/red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-688-vmlinuz Version-Release number of selected component (if applicable): 6.5 How reproducible: Always Steps to Reproduce: 1. Use deployment via TFTP 2. Check tftp/boot Actual results: Space wasted. Expected results: If there is no process to track if these files are still needed or not, there should be at least a process that does a hardlink run from time to time. Is there a manual process that one can trigger to get rid of old files there?
As suggested over in the other BZ, I'm opening this one in order to address/add the functionality to clean out old files in tftp/boot with satellite-maintain.
To give you little bit more context: Implement a command that checks if there are any /var/lib/tftpboot/boot/*-{vmlinuz,initrd.img} files older than two weeks (configurable via option). Provide a resolution (fix) - delete those files. Make this check to be part of the standard suite of checks. Note this directory can be missing if TFTP module is not set up - skip the check in that case.
Lukas, Deleting everything older than 2 weeks cannot be done, at least not what i quickly see as an user: -------- [crash/LI] root@li-lc-2222:/var/lib/tftpboot/boot# ls -lrt total 2411476 -rw-r--r--. 1 foreman-proxy foreman-proxy 4316752 Jun 18 2019 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-415-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 41039717 Jun 18 2019 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-415-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 4316752 Jun 18 2019 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-389-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 41039717 Jun 18 2019 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-389-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 4316752 Jun 18 2019 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-361-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 41039717 Jun 18 2019 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-361-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5391264 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-520-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 44701632 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-520-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5391264 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-484-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 44701632 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-484-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 5391264 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-448-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 44701632 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-3-448-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-990-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-990-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-9713-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-9713-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-960-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-960-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-900-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-900-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-840-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-840-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-750-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-750-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-718-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-718-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-682-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-682-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-646-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-646-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-612-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-612-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-582-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-582-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-552-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-552-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6635920 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1073-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 54799220 Jun 18 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-6-1073-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9641-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9641-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9595-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9595-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9206-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9206-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9167-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9167-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9106-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9106-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9067-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9067-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9037-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-9037-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-32557-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-32557-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-32461-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-32461-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-24252-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-24252-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-19531-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-19531-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-18625-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Aug 14 2019 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-18625-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 8106848 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-27021-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-27021-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 8106848 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26974-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26974-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 8106848 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26941-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26941-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 8106848 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26250-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26250-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 8106848 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26216-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26216-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6762800 Apr 2 13:44 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-45614-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57894232 Apr 2 13:44 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-45614-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6762800 Apr 2 13:44 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-44873-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57894232 Apr 2 13:44 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-44873-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6762800 Apr 2 13:44 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-44829-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57894232 Apr 2 13:44 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-44829-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6762800 Apr 2 13:44 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-43766-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57894232 Apr 2 13:44 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-43766-initrd.img [crash/LI] root@li-lc-2222: ------------- If you implement what you mentioned above to delete files older than 2 weeks then All files will be deleted and then any TFTP is broken until a build-pxe-default is executed and the boot images are recreated. Peter
And secondly the original issue is not only partly solved by cleanup of obsolete images. Because after cleanup of all files (2.4GB) and then call to build-pxe-default to rebuild all TFTP images i still have 1.4GB of images: ---- [crash/LI] root@li-lc-2222:/var/lib/tftpboot# sudo du -ms /var/lib/tftpboot/boot 1383 /var/lib/tftpboot/boot ---- Why do i have 5x the same image for red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-* without using e.g. hardlinking (the ls -l below shows '1' for files that have the same hash in column 2)? ----- [crash/LI] root@li-lc-2222:/var/lib/tftpboot/boot# ls -l red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-*-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26216-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26250-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26941-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26974-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 62248424 Nov 8 13:38 red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-27021-initrd.img [crash/LI] root@li-lc-2222:/var/lib/tftpboot/boot# sha1sum red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-*-initrd.img 24506ad7a7de868e5e991491d5cad102a5e1d1ec red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26216-initrd.img 24506ad7a7de868e5e991491d5cad102a5e1d1ec red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26250-initrd.img 24506ad7a7de868e5e991491d5cad102a5e1d1ec red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26941-initrd.img 24506ad7a7de868e5e991491d5cad102a5e1d1ec red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-26974-initrd.img 24506ad7a7de868e5e991491d5cad102a5e1d1ec red-hat-enterprise-linux-8-for-x86_64-baseos-kickstart-8-1-27021-initrd.img [crash/LI] root@li-lc-2222:/var/lib/tftpboot/boot# -----
The reason why this is so clunky is because PXE handling is done by Foreman core while content handling (Content Views) is done by Katello plugin and this is not integrated deeply. Ideally, we would calculate SHA1 sum of the content and name files accordingly, problem is filenames are created by Foreman not Smart Proxy and without access to the content. Foreman would need to download them, calculate SHA1 sum thus filename and pass this to smart proxy. This seems to be too complicated. Thus the proper solution is to completely redesign this. One partial solution could be that we keep everything (the API) as is but proxy would download and store files under their content hash name and create only symlinks pointing to these files. But then it's just a different problem - you would end up with many symlinks. Granted it wouldn't consume too much space then but with high rate of promotions you would still end up with many symlinks. A cleanup process would still be needed. Is your bigger concern the size or amount of files? Now, you are right that I haven't considered hosts which are in build mode. But hosts are only valid for given amount of time (token duration) and they expire and the only way to get out of that is to exit and enter build mode which is a process which re-downloads TFTP files. Therefore I am going to update instructions for the foreman maintain team: - Connect to Satellite API or DB and find token_duration setting (amount of minutes host provisioning tokens are valid). - Implement a maintain command that checks if there are any /var/lib/tftpboot/boot/*-{vmlinuz,initrd.img} files older than N minutes (token duration setting found above). - Provide a resolution (fix) - delete those files. - Make this check to be part of the standard suite of checks. - Skip this check if: - the TFTP directory is missing - token_duration is set to 0 (tokens are disabled) I hope this creates better experience until we find time to redesign TFTP.
Created redmine issue https://projects.theforeman.org/issues/29727 from this bug
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/29727 has been resolved.
FailedQA Tested with: - Satellite 6.8.0 snap 11 - rubygem-foreman_maintain-0.6.8-1.el7sat.noarch Test steps: - Run "foreman-maintain health check --label check-tftp-storage" Observations/Issues found: - Check deleted all files older than token_duration setting from /var/lib/tftpboot/boot/ instead of only deleting those matching /var/lib/tftpboot/boot/*-{vmlinuz,initrd.img}(Ref: comment#8) - It also deleted fdi-image subdirectory. Additional info: # ll /var/lib/tftpboot/boot/ total 4.0K drwxr-xr-x. 2 foreman-proxy root 4.0K Aug 5 06:37 fdi-image # ll /var/lib/tftpboot/boot/fdi-image/ total 227M -rw-r--r--. 1 foreman-proxy root 220M Aug 5 06:37 foreman-discovery-image-3.6.1-2.iso-img -rw-r--r--. 1 foreman-proxy root 6.4M Aug 5 06:37 foreman-discovery-image-3.6.1-2.iso-vmlinuz lrwxrwxrwx. 1 foreman-proxy root 39 Aug 5 06:37 initrd0.img -> foreman-discovery-image-3.6.1-2.iso-img lrwxrwxrwx. 1 foreman-proxy root 43 Aug 5 06:37 vmlinuz0 -> foreman-discovery-image-3.6.1-2.iso-vmlinuz # foreman-maintain health check --label check-tftp-storage Running ForemanMaintain::Scenario::FilteredScenario ================================================================================ Clean old Kernel and initramfs files from tftp-boot: [FAIL] There are old initrd and vmlinuz files present in tftp -------------------------------------------------------------------------------- Continue with step [Remove the files]?, [y(yes), n(no), q(quit)] y Remove the files: [OK] -------------------------------------------------------------------------------- Rerunning the check after fix procedure Clean old Kernel and initramfs files from tftp-boot: [OK] -------------------------------------------------------------------------------- # ll /var/lib/tftpboot/boot/ total 0 # ll /var/lib/tftpboot/boot/fdi-image/ ls: cannot access /var/lib/tftpboot/boot/fdi-image/: No such file or directory
Verified: Verified with: - Satellite 6.8.0 snap 15 - rubygem-foreman_maintain-0.6.10-1.el7sat.noarch Test steps: - Have some *-{vmlinuz,initrd.img} files in /var/lib/tftpboot/boot/ directory which are older than token_duration settings. - Run "foreman-maintain health check --label check-tftp-storage" Observation: - check-tftp-storage check deletes files older than token_duration time, matching *-{vmlinuz,initrd.img} pattern from /var/lib/tftpboot/boot/ directory. [root@qe-sat6-feature-rhel7 ~]# ll /var/lib/tftpboot/boot/ total 107272 drwxr-xr-x. 2 foreman-proxy root 4096 Sep 10 17:24 fdi-image -rw-r--r--. 1 root root 0 Sep 11 04:56 red-hat-enterprise-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 41039717 Sep 10 17:56 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-28-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 4316752 Sep 10 17:56 red-hat-enterprise-linux-6-server-kickstart-x86_64-6-10-28-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 57746484 Sep 10 17:58 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-19-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6730032 Sep 10 17:57 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-7-19-vmlinuz -rw-r--r--. 1 root root 0 Sep 11 04:56 red-hat-enterprise-vmlinuz -rw-r--r--. 1 root root 0 Sep 11 04:55 test.yml [root@qe-sat6-feature-rhel7 ~]# hammer settings list --search token_duration ---------------|----------------|-------|--------------------------------------------------------------------------------- NAME | FULL NAME | VALUE | DESCRIPTION ---------------|----------------|-------|--------------------------------------------------------------------------------- token_duration | Token duration | 360 | Time in minutes installation tokens should be valid for, 0 to disable token g... ---------------|----------------|-------|--------------------------------------------------------------------------------- [root@qe-sat6-feature-rhel7 ~]# date Fri Sep 11 04:58:47 EDT 2020 [root@qe-sat6-feature-rhel7 ~]# foreman-maintain health check --label check-tftp-storage Running ForemanMaintain::Scenario::FilteredScenario ================================================================================ Clean old Kernel and initramfs files from tftp-boot: [FAIL] There are old initrd and vmlinuz files present in tftp -------------------------------------------------------------------------------- Continue with step [Remove the files]?, [y(yes), n(no), q(quit)] y Remove the files: [OK] -------------------------------------------------------------------------------- Rerunning the check after fix procedure Clean old Kernel and initramfs files from tftp-boot: [OK] -------------------------------------------------------------------------------- [root@qe-sat6-feature-rhel7 ~]# ll /var/lib/tftpboot/boot/ total 4 drwxr-xr-x. 2 foreman-proxy root 4096 Sep 10 17:24 fdi-image -rw-r--r--. 1 root root 0 Sep 11 04:56 red-hat-enterprise-initrd.img -rw-r--r--. 1 root root 0 Sep 11 04:56 red-hat-enterprise-vmlinuz -rw-r--r--. 1 root root 0 Sep 11 04:55 test.yml [root@qe-sat6-feature-rhel7 ~]# foreman-maintain health check --label check-tftp-storage Running ForemanMaintain::Scenario::FilteredScenario ================================================================================ Clean old Kernel and initramfs files from tftp-boot: [OK] --------------------------------------------------------------------------------
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Satellite 6.8 Satellite Maintenance Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4365