Bug 2213791
Summary: | Everything/{x86_64,aarch64}/repodata/repomd.xml partially out of sync with other xml | ||
---|---|---|---|
Product: | [Fedora] Fedora EPEL | Reporter: | Owen LaGarde <olagarde> |
Component: | epel-release | Assignee: | Kevin Fenzi <kevin> |
Status: | NEW --- | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | medium | Docs Contact: | |
Priority: | unspecified | ||
Version: | epel8 | CC: | carl, kevin, mastahnke, smooge, tdawson |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | Type: | Bug | |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Owen LaGarde
2023-06-09 12:05:35 UTC
To validate the check and rsync status outside of EPEL I've run it against the other 12 mirrors we maintain. These are mirrored using the same method/scripting/filesystems/os/etc, and do not have this issue. Examples: 08:08:59 root:~ # ./check_repomd_xml_locations /var/repo/rocky-linux/8/BaseOS/x86_64/os ok: repodata/d023aea410537b1caa0df7c8b7d17bea8145302b59cf7adc3a9b92119a4f9654-primary.xml.gz ok: repodata/e4c6fc6e80303cf9c27c499b913bd7165e1f542fa622096004b180b8e9f1c603-filelists.xml.gz ok: repodata/a69ab1b6236b04d262a29b1c007d37e94ef5e2a6f7e576f6ce501842198bf424-other.xml.gz ok: repodata/b810761f905a608cd074aac3371aa4952f5bd30d2e77d368c28548572e950a72-primary.sqlite.xz ok: repodata/3e5e8447237bf7a5033ca0a8409d9e88ac6b208711713889a682165c578087bc-filelists.sqlite.xz ok: repodata/476f80e14610eec29d247f070f12e86955c92897e66e21565ca471493231c7f9-other.sqlite.xz ok: repodata/dae7e104812099a2f632ea4c5ef2769aca18ca1205abdd2c3ba6d171e319df3d-comps-BaseOS.x86_64.xml ok: repodata/741d3c80487757df624285f4a107f925abfac16115210486760a3920d8724e38-comps-BaseOS.x86_64.xml.xz ok: repodata/09fba4ce8e1dd769dd48d4f9e347119d1b30826e2710ea7fbfe21e91cbdd8523-updateinfo.xml.gz 08:09:02 root:~ # ./check_repomd_xml_locations /var/repo/rocky-linux/9/AppStream/aarch64/os ok: repodata/b0b7187b-86e0-4eaf-bec8-3802bb794f8a-PRIMARY.xml.gz ok: repodata/6871de8c-4eef-46bc-9f24-929877be2d45-FILELISTS.xml.gz ok: repodata/6871de8c-4eef-46bc-9f24-929877be2d45-OTHER.xml.gz ok: repodata/35e6cafd805abc07279320cdb95e0c8e25c75541214de7c04d84426c99ecc12a-primary.sqlite.gz ok: repodata/7a8c0814c9884a628f9df4e16d4e4a0611e377dfb21ba96e32d0a126084d2bc2-filelists.sqlite.gz ok: repodata/eb304fe6e6a780203743447f13d0e39b5fed2b36fc37e85fec512095848949b5-other.sqlite.gz ok: repodata/6871de8c-4eef-46bc-9f24-929877be2d45-GROUPS.xml ok: repodata/6871de8c-4eef-46bc-9f24-929877be2d45-GROUPS.xml.gz ok: repodata/6871de8c-4eef-46bc-9f24-929877be2d45-MODULES.yaml.gz ok: repodata/351f5b1c-967c-4392-8471-bb7aa0006c07-UPDATEINFO.xml.gz 08:14:31 root:~ # ./check_repomd_xml_locations /var/repo/nvidia/public/repo/mlnx_ofed/latest/rhel8/aarch64/ ok: repodata/c4d462655d6a9f742195959d82e97e92fff7be53c3b861b6dae585f1c4d71615-primary.xml.gz ok: repodata/afd7597806478bfa4f6756facd45d6117d464aef4e735ed2472bb4ef6a89ea8b-filelists.xml.gz ok: repodata/f1d5da69cc0ab7cdc7c7e59a349620177a3632ac6dc06f247010cc34542727bd-other.xml.gz ok: repodata/40aaa7ee67185156190c38c105e07a0a643b3790938ebdff28527678d0e2ea91-primary.sqlite.bz2 ok: repodata/e211035c3d60438d7dc5ef0ae2bf24c27792078d54e15dbbbc1459834cf5fda4-filelists.sqlite.bz2 ok: repodata/a3bb180fb5cee3177db6f29018ad5c246261e8b1ecfcf36401811cc7836dbb95-other.sqlite.bz2 08:14:37 root:~ # ./check_repomd_xml_locations /var/repo/auristor/filesystem/repo/v2021.05/8/aarch64/ ok: repodata/141a9197587b852e8419b106a4f79ae709415533d55f84ef8e880b75af366e15-primary.xml.gz ok: repodata/ae264358436d389155debd25f40faa4d85aa72683b7835d28de68b9bb846ae07-filelists.xml.gz ok: repodata/8c220794c7b653351cbe601231ed9ff660a55d63b5164fa8b1a56abe0aa63606-other.xml.gz ok: repodata/abd2c1e77b2bf4cfacfba4464126fc1b2b40114ce1045b40a7265f37040b1e4b-primary.sqlite.bz2 ok: repodata/0fda89927127594c9027e2e5ceabf74713a0256b8dcc01c0453ffb9aead3b41a-filelists.sqlite.bz2 ok: repodata/69faa91b61d27d829a745d4af62ecde859179296b6c01cceab893fb200126790-other.sqlite.bz2 I don't *think* I'm stepping in ongoing changes to the upstream sources... the results here are uniform through 4 successful rsyncs across as many days. But just to check I've gone through the verbose rsync output for the usual indications (permission denied for "Packages/{letter}/.{name}", presence of "Packages/{letter}/.*.rpm.{tmpstring}", etc. None have been reported in the last week. I think this is due to EPEL no longer doing modularity but am not sure. I have opened https://pagure.io/releng/issue/11470 for Fedora release engineering to look at it as this is more of a IT issue than a bug issue. Please note that the problem exists for epel/8/Everything/{arch} and epel/9/Everything/{arch} only. There is no problem with epel/8/Modular/{arch}/repodata/repomd.xml. Also, note that I'm *not* running createrepo, or referring to any locally produced repo metadata. All our EPEL repo metadata is from upstream via rsync. Understood. I am wrong about the repomd.xml file being from modularity. I went and looked at the upstream mirror you were using and it looks like it is an incomplete mirror for some reason. On https://dl.fedoraproject.org/pub/epel/8/Everything/aarch64/repodata there is a ``` -rw-r--r--. 1 263 263 1756628 2023-06-09 01:44 29755d3376d443154401a0a030d14873c6147c494d503a6eecdfff5dae24644f-other.sqlite.xz -rw-r--r--. 1 263 263 2903975 2023-06-09 01:43 4c4f92104a1abdd848086877c187f83c0fae4f63a929e56fd98f10c6af2fe191-primary.xml.gz -rw-r--r--. 1 263 263 4426556 2023-06-09 01:44 55e1359cbdce67ee6c617ef97640c40ad6726cf692d74a5e18322f59a034528f-primary.sqlite.xz -rw-r--r--. 1 263 263 25584 2023-06-09 01:43 9734d400fe2c45922f15a3db02c3f372ac76fc478d3b2a54da4ec702bd69c40a-comps-Everything.aarch64.xml.xz -rw-r--r--. 1 263 263 12052509 2023-06-09 01:43 a032b1c806230cb2459b560619c336701b1fdfff664ab368c56309c5bf9292fc-filelists.xml.gz -rw-r--r--. 1 263 263 132413 2023-06-09 01:43 b31dded4034511935721017022689f916dee4f46902ec03b200ea455b6c26e1c-comps-Everything.aarch64.xml -rw-r--r--. 1 263 263 1622493 2023-06-09 01:43 b8f3b279dfac835813fc5cef13ff5d62a22ac7a160f85c0b7a45d9732581015c-other.xml.gz -rw-r--r--. 1 263 263 10501984 2023-06-09 01:44 c618dc5b68d6311f3cf1e43fdc23e13b7ae35314c367e6e47f8572f9ac3ef688-filelists.sqlite.xz -rw-r--r--. 1 263 263 776 2023-06-09 01:44 f25d45e0aaf63956851c221a41843beb36b17ee4d29bf600b30f3a03ab34bf2a-prestodelta.xml.xz -rw-r--r--. 1 263 263 1072131 2023-06-09 01:47 f6504da72d77acc6b31db0ec0077bc580b424d3e5599f4e94bd2eb59dfd9684a-updateinfo.xml.bz2 -rw-r--r--. 1 263 263 4848 2023-06-09 01:47 repomd.xml ``` but none of those files are in rsync://mirror.umd.edu/fedora/epel/ except for the updated repomd.xml. Could you try a `rsync -n -rvlptDSH --exclude="*.~tmp~" --delete-delay --delay-updates rsync://dl.fedoraproject.org/fedora/epel/ /var/repo/epel/ >> "$logfile" 2>&1` I put in a `-n` to not change things but see what would change. Done, full upstream list, two noteworthy items: minor: testing/8/Everything/s390x/Packages/p/.plasma-workspace-wallpapers-5.24.6-1.el8.noarch.rpm.r5w18X -- orphaned from a borked build? major: a grep of the verbose log for "repodata" shows only the "repodata/" directories themselves (looks like they are all there for everything I'd expect), and these actual metadata files: testing/8/Modular/ppc64le/drpms/os/repodata/1e4079d1ccee120287c4091454fdc22d8dd696d07d276d1d0705aaf5a93b9e59-filelists.sqlite.bz2 testing/8/Modular/ppc64le/drpms/os/repodata/8497f7e78cd216ec0d03a478f149f4794e314f1c2404bde3d3808a6a0a020e16-other.sqlite.bz2 testing/8/Modular/ppc64le/drpms/os/repodata/ad8858cc8267b17abcfdd8236965a877433f54a29bb75cd8dc4569a5725eeb3f-primary.sqlite.bz2 testing/8/Modular/ppc64le/drpms/os/repodata/repomd.xml No other repodata/* files were listed. Upstream server was http://mirror.umd.edu/. Same grep in the previous log shows the expected repodata/*xml entries being downloaded in a pass ending Fri 09 Jun 2023 06:55:50 AM EDT. |