Description of problem: Environment: Beaker server and Beaker controllers X86_64, beaker version 24 Systems under test are Ampere eMAG aarch64. Currently have ~200 systems in beaker, and adding capacity for another 100+ Added a new beaker-controller, added Distros to the controller, executed beaker-repo-update on the beaker server and following errors generated. Been approx 2-3 weeks since last beaker-repo-update command on the beaker server [root@beaker yum.repos.d]# beaker-repo-update 2018-09-04 13:53:13,511 bkr.server.tools.repo_update WARNING failure: python-twisted-core-12.2.0-4.el7.aarch64.rpm from http:--beaker-project.org-yum-harness-RedHatEnterpriseLinux7-: [Errno 256] No more mirrors to try. http://beaker-project.org/yum/harness/RedHatEnterpriseLinux7/python-twisted-core-12.2.0-4.el7.aarch64.rpm: [Errno 14] HTTPS Error 416 - Requested Range Not Satisfiable 2018-09-04 13:53:25,786 bkr.server.tools.repo_update WARNING failure: acpica-tools-debuginfo-20140926-1.el7.ppc64.rpm from http:--beaker-project.org-yum-harness-CentOS7-: [Errno 256] No more mirrors to try. http://beaker-project.org/yum/harness/CentOS7/acpica-tools-debuginfo-20140926-1.el7.ppc64.rpm: [Errno 14] HTTPS Error 416 - Requested Range Not Satisfiable - Able to ping beaker-project.org from beaker server - able to wget both files from the beaker server Version-Release number of selected component (if applicable): 24 How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Additional info.. Beaker system will now not start a job following provision/reboot - Executed an inventory job on a previously working system, rack, beaker-controller - System under test (in this case an Osprey) boots successfully - Osprey ethernet has good IP address, can ping the beaker server, beaker controller, and google.com - Beaker Controller: verified all services are running - beaker-proxy, beaker-provision, beaker-watchdog, dhcpd, httpd, tftp - Beaker Server: verified all services are running: beakerd, ntpd, mariadb The beaker system stopped working following the failed beaker-repo-update Entire system was fully operational prior to the beaker-repo-update command fail
Dear Ed, beaker-repo-update received maintenance updates in our latest 25.6 release. The corresponding bug is: Bug 1619969. What you're describing looks like what Dan has just spend time on, but I'm not 100% sure. Check Bug 1619969 comment 1 which has a workaround if the repositories are in a bit of a messed up state in case you do not want to upgrade to 25.6. Please let me know if that has solved the problem for you and your harness repositories.
Roman, Applied the workaround from 1619969 and it resolved the issue, lab is back speed. Will need to schedule time at some point to upgrade the server. Thanks!
Will mark this as a duplicate of bug 1619969. Just note that the workaround is not a very good one -- it is not only the repodata that can be inconsistent but the packages themselves could be corrupted, as I later discovered. The corrupted packages will be fixed by running beaker-repo-update after Beaker is upgraded to 25.6. So it would be best to upgrade ASAP. However, if the packages can be successfully installed on the system under test then it means they aren't corrupted so in that case there is no rush to upgrade I guess. Just something to bear in mind. *** This bug has been marked as a duplicate of bug 1619969 ***