Bug 2233878
| Summary: | rpm python interface fails to expand fontpkgname1 macro | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Jerry James <loganjerry> |
| Component: | rpm | Assignee: | Panu Matilainen <pmatilai> |
| Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 39 | CC: | cqi, ffesti, igor.raits, jonathan.biegert, lsedlar, mdomonko, mjg, onosek, packaging-team-maint, philip.wyett, pmatilai, sergio |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2024-11-07 14:03:32 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This isn't a fedpkg problem. It simply invokes "spectool -l": $ spectool -l musescore.spec Source0: https://github.com/musescore/MuseScore/archive/v4.1.1/MuseScore-4.1.1.tar.gz Source1: 65-%{fontpkgname1}.conf Source2: 65-%{fontpkgname2}.conf Source3: 65-%{fontpkgname3}.conf Source4: 65-%{fontpkgname4}.conf Source5: 65-%{fontpkgname5}.conf Source6: 65-%{fontpkgname6}.conf Source7: 65-%{fontpkgname7}.conf Source8: 65-%{fontpkgname8}.conf Source9: 65-%{fontpkgname9}.conf Patch0: musescore-fix-qmake-path.patch Patch1: musescore-unbundle-libs.patch Patch2: musescore-unbundle-fonts.patch Patch3: musescore-splashscreen.patch Patch4: musescore-ffmpeg.patch Patch5: musescore-appdata.patch Patch6: musescore-neon.patch And spectool invokes the rpm python interface: $ python3 Python 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import rpm >>> spec = rpm.spec('musescore.spec') >>> spec.sources [('musescore-neon.patch', 6, 2), ('musescore-appdata.patch', 5, 2), ('musescore-ffmpeg.patch', 4, 2), ('musescore-splashscreen.patch', 3, 2), ('musescore-unbundle-fonts.patch', 2, 2), ('musescore-unbundle-libs.patch', 1, 2), ('musescore-fix-qmake-path.patch', 0, 2), ('65-%{fontpkgname9}.conf', 9, 1), ('65-%{fontpkgname8}.conf', 8, 1), ('65-%{fontpkgname7}.conf', 7, 1), ('65-%{fontpkgname6}.conf', 6, 1), ('65-%{fontpkgname5}.conf', 5, 1), ('65-%{fontpkgname4}.conf', 4, 1), ('65-%{fontpkgname3}.conf', 3, 1), ('65-%{fontpkgname2}.conf', 2, 1), ('65-%{fontpkgname1}.conf', 1, 1), ('https://github.com/musescore/MuseScore/archive/v4.1.1/MuseScore-4.1.1.tar.gz', 0, 1)] >>> hdr = spec.sourceHeader >>> hdr[rpm.RPMTAG_SOURCE] ['65-%{fontpkgname9}.conf', '65-%{fontpkgname8}.conf', '65-%{fontpkgname7}.conf', '65-%{fontpkgname6}.conf', '65-%{fontpkgname5}.conf', '65-%{fontpkgname4}.conf', '65-%{fontpkgname3}.conf', '65-%{fontpkgname2}.conf', '65-%{fontpkgname1}.conf', 'MuseScore-4.1.1.tar.gz'] I notice that the fontpkgnameX macros are Lua macros. Would the rpm maintainers chime in on whether this is expected behavior of the rpm python interface? If it is, what should spectool do to get fully macro-expanded source file names? Oh my. Frankly, I don't know how these packages manage to build *at all* (at least with rpm >= 4.15), yet somehow they apparently do. The issue is these source lines refer to macros that ARE undefined at the point they come up in the spec, and only become defined after the call to '%fontpkg -a'. Which happens much later in the spec. Moving the %fontpkg call before the Source lines makes them defined at that point, but *then* the spec fails to build. (I'm looking at makemusic-finale-fonts it's a more lightweight to work with than musescore, but the issue is the same) Thanks for looking at this Panu. Yes, moving %fontpkg before the SourceX lines leads to %{SOURCE0} and %{SOURCE1} somehow being undefined. I admit that I don't understand the %fontpkg macro. Do you have any suggestions on how we can move forward?
Note that this is not necessarily an issue in the spec of multi font packages. `spectool -l` works for https://src.fedoraproject.org/rpms/sil-gentium-basic-fonts and the adf fonts, for example. I couldn't spot any relevant differences (and checked that many are irrelevant ...). [Maybe lack of a main package/package0, or the numbering).] I also added `-v` to `%fontpkg -a` which gives debug output from the font macros. In all cases ("working" and "non-working"), macros use forward references. Also, if I edit the spec file to say ``` %global fontconfs1 %{SOURCE2} %global fontpkgname1 finale-ash-fonts ``` then `spectool -l` will happily resolve the forward reference! So it's not about `SOURCE` being forward referenced at all. Also, apparantly, the macros work fine with rpm, just not with spectool/bindings. "Forward declarations" are at the heart of the issue, really. The differences come from some users re-expanding the value (rpmbuild) and some not (bindings).
This is basically what happens:
[pmatilai🎩︎localhost ~]$ rpm --eval "%global fu %{bar}" --eval "%{fu}" --eval "%global bar 123" --eval "%{fu}"
%{bar}
123
During rpmbuild, in %prep, whatever sources will have already been defined via SourceX lines, so it expands to meaningful values. But at the time of declaration, which is the data spectool will see, those unresolvable macros come through as-is.
I think this changed behavior in commit https://github.com/rpm-software-management/rpm/commit/93604e2c3b8ed8a2b1ac77c4c43856e68b395c49 despite the commit saying otherwise. It was discussed back then and the consensus was basically that the spec is an ordered construct and you need to define before you can use. This thing here does fall into a certain "undefined behavior" black hole that's not very productive for anybody though: if its not legit then it should just fail. And if it isn't, it shouldn't have side-effects like this. Here's a potential upstream fix for the issue: https://github.com/rpm-software-management/rpm/pull/3067 Thank you, Panu. It will be great to have the pre-push hook work again. Oh, looks I forgot to comment on this: contrary to my suspicion in comment #6, the linked commit did not change this behavior although it's closely related. This case has always been broken. Seems we've forgotten this bug in a POST limbo despite actually fixing it. Fedora 39 is going EOL shortly but it's currently fixed in rawhide and Fedora 41, and this looks like something that could be included in the next 4.19.x bugfix when we get around to that. Thanks for the report! |
When attempting to push to a font package repository with multiple fonts, I invariably get a rejection from the pre-push hook, like this: Source file '65-%{fontpkgname1}.conf' was neither listed in the 'sources' file nor tracked in git. Push operation was cancelled Hint: this check (.git/hooks/pre-push script) can be bypassed by adding the argument '--no-verify' argument to the push command. The %{fontpkgname1} macro was not expanded, and so the filenames don't match. As noted in this thread [1], others have encountered this problem as well, and there is some suspicion that this bug may have contributed to many font packages not being rebuilt during the mass rebuild. [1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/7IIZPEC6B4BEOOOV5YFEUGOONNOH5LZO/ Reproducible: Always Steps to Reproduce: 1. Pick a font package such as fontawesome-fonts or makemusic-finale-fonts 2. Make some trivial change 3. Try to do a git push Actual Results: The git push is prevented by the pre-push hook, due to an unexpanded RPM macro. Expected Results: The macro should be expanded, and the pre-push hook should permit the push.