Bug 2004853
| Summary: | Module yaml error: Unexpected key in data: static_context | |||
|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Michal <michallinuxstuff> | |
| Component: | libdnf | Assignee: | amatej | |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | |
| Severity: | high | Docs Contact: | ||
| Priority: | urgent | |||
| Version: | 34 | CC: | amatej, bill.wood, dmach, jberan, jmracek, jrohel, mblaha, mhatina, nphilipp, packaging-team-maint, patrick_van_alphen, pkratoch, ppisar, rpm-software-management, sgallagh, user-cont-team+packit-fas, vmukhame | |
| Target Milestone: | --- | Keywords: | Triaged | |
| Target Release: | --- | |||
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | libdnf-0.66.0-1.fc37 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2007166 (view as bug list) | Environment: | ||
| Last Closed: | 2022-03-14 13:09:32 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
|
Description
Michal
2021-09-16 08:59:31 UTC
May I ask you for a version of libmodulemd? (rpm -q libmodulemd) (In reply to Jaroslav Mracek from comment #1) > May I ask you for a version of libmodulemd? (rpm -q libmodulemd) Sure, here it is: # modulemd-validator --version modulemd-validator 2.9.4 # rpm -q libmodulemd libmodulemd-2.9.4-3.fc33.x86_64 This is the version that's shipped inside the https://archives.fedoraproject.org/pub/fedora/linux/releases/33/Cloud/x86_64/images/Fedora-Cloud-Base-33-1.2.x86_64.qcow2. I can confirm that if I bump it to the latest one currently available for f33 (2.13.0), everything works as expected (errors from the initial report are gone as well). Interestingly, inside the https://archives.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.qcow2 libmodulemd is at 2.9.1 which works from the very get-go. So I am guessing something suddenly started triggering a regression introduced somewhere between 2.9.1<->2.9.4? Or is something else the culprit here? In any case, updating the package seems to be a feasible workaround on f33. :) The issue is valid for libmodulemd-2.9.4-3.fc33.x86_64 and can be resolved by upgrade libmodulemd (`dnf upgrade libmodulemd`). The issue is fixed in libmodulemd-2.13.0-1.fc33. What happen? libmodulemd-2.9.4-3.fc33.x86_64 detected unknown field `static_context [line 9 col 3]` and it reports it as a problem. It resulted in visibility of all packages (modular and not modular) in dnf. Then dnf rejects installation of modular packages without metadata. I am not 100% sure but I guess that libmodulemd-2.9.4-3.fc33.x86_64 skips all modules with static_context. This is a critical issue!!! Not only for Fedora but also for RHEL. Because the issue is in libmodulemd, changing component. Please do not close the issue even if the upgrade of libmodulemd helped! Noted, appreciate the feedback. :) This is caused by DNF which explicitly strictly validates modulemd-v2 documents before using them. Since the old libmodulemd 2.9.4 does not recognize static_context field, it reports that the document is not strictly valid and DNF will discard it. Because DNF discards all documents in a modular repository, it errors with "No available modular metadata for modular package". DNF needs stop strictly validating the documents. Without strict validation, old libmodulemd will ignore unknown fields and DNF will happily consume them. You can work around the DNF issues by upgrading libmodulemd first. Then subsequent invocations of DNF will recognize the same documents as valid. Because you use DNF for updating libmodulemd (and dnf, once fixed), you need temporarily disable modular repositories "dnf --disable-repo 'fedora-modular*' upgrade libmodulemd". To Michal's question what has changed: Yesterday first module builds with static_context field were pushed into stable repositories of Fedora 33. Your system probably has not enabled fedora-updates-testing-modular, thus you noticed this issue (= upgrading a nonupdated system) on pushing the modules into fedora-updates-modular repository. A support for the static_context field first appeared in libmodulemd-2.11.0. The only Fedora with older libmodulemd at G.A. is Fedora 33. Fedora 34 started with libmodulemd-2.11.2. Therefore upgrading from G.A. installation is affected only in Fedora 33. We cannot change G.A. installation media. But I believe Fedora updates installation media periodically. libmodulemd-2.11.0-1.fc33 was pushed to stable updated repository on 2020-12-16. I will check whether current installation media contain updated libmodulemd. Otherwise, the only fix would be retracting all the modules with static_context field from Fedora 33 repositories. Fedora 33 will be supported until 4 weeks after releasing Fedora 35 (not yet happened). (In reply to Petr Pisar from comment #6) > To Michal's question what has changed: Yesterday first module builds with > static_context field were pushed into stable repositories of Fedora 33. Your > system probably has not enabled fedora-updates-testing-modular, thus you > noticed this issue (= upgrading a nonupdated system) on pushing the modules > into fedora-updates-modular repository. Petr, Thanks for the detailed info. Indeed, fedora-updates-testing-modular was disabled as per the default setting shipped inside the .qcow2. Considering that, would you kindly advise if it's a good idea to enable it prior installing|upgrading any packages? Asking since the way how we provision our VMs doesn't really touch repos|dnf configuration (except some minor stuff like proxy, etc.) so we mainly depend on the defaults while installing some of the required stuff via dnf. (In reply to Michal from comment #8) > Indeed, fedora-updates-testing-modular > was disabled as per the default setting shipped inside the .qcow2. > Considering that, would you kindly advise if it's a good idea to enable it > prior installing|upgrading any packages? Enabling the testing repository won't help you in resolving this issue. It's too late. Enabling that repository would only help you spotting any issues sooner and provide a feedback to Fedora. And based on that Fedora could resolve the issues before they reached the stable updates repository. For instance, if you reported this issue before 15th September, we could avoid it. > Asking since the way how we > provision our VMs doesn't really touch repos|dnf configuration (except some > minor stuff like proxy, etc.) so we mainly depend on the defaults while > installing some of the required stuff via dnf. Your installation procedure is fine. Fedora supports upgrading from G.A. installation. But we failed to verify the upgrade and broke it. The problem that retracting updated from a stable updated repository is a non-standard procedure and I don't know how quickly or if Fedora will be able to fix it. I also has a bad news. My assumption that Fedora updates installation media was false. Hence this issue cannot be resolved by releasing a new DNF or libmodulemd version. For DNF maintainers: This Python code shows a difference between strict and nonstrict validation:
#!/usr/bin/python3
import gi
gi.require_version("Modulemd", "2.0")
from gi.repository import Modulemd
from gi.repository.Modulemd import ModuleIndex
import sys
module = '''
document: modulemd
version: 2
data:
name: foo
stream: bar
summary: text
description: text
version: 1
x-unknown: A
context: A
license:
module:
- MIT
'''
index = ModuleIndex.new()
result, failures = index.update_from_string(module,
False # ignore unknown fields
#True # error on an unknown field
)
if result:
print("Successfully loaded:")
for stream in index.search_streams():
print("{}:{}".format(stream.get_module_name(),
stream.get_stream_name()))
sys.exit(0)
else:
print("Loading failed:")
for failure in failures:
print(failure.get_gerror())
sys.exit(1)
I could not find in DNF code where modulemd files are loaded. I hope it will help you in improving DNF. The update_from...() method should be called with False argument.
Michal, I reproduced your issue and I have another work around for you, which actually I would recommend to do anytime: After a fresh installation, perform an upgrade, and then install additional packages: # dnf upgrade # dnf install autoconf And don't forget rebooting the virtual machine to take in effect the updated kernel and other daemons and libraries. Otherwise you will use up-to-date autoconf, but other software will contain known bugs, possibly critical security vulnerabilites. I asked relengs to evaluate the possibility of retracting the affected modules from Fedora 33 updates repository <https://pagure.io/releng/issue/10306>. Currently there are only two module builds with the static_context field in F33 repositories: perl:5.32:3320210903081051:94c872f1 in updates, swig:4.0:3320210916145835:601d93de in testing. I will rebuild the perl and in a week it should reach updates repository and replace the faulty one. I asked a swig maintainer to unpush it from testing. FEDORA-MODULAR-2021-8320b27b17 has been submitted as an update to Fedora 33 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2021-8320b27b17 I made a PR that switches off the strict validation for dnf (it is done through libdnf) https://github.com/rpm-software-management/libdnf/pull/1346. Great. Does it make sense to move this bug report to libdnf component? We can but I think it's not hugely important. FEDORA-MODULAR-2021-8320b27b17 has been pushed to the Fedora 33 Modular testing repository. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2021-8320b27b17 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates. FEDORA-MODULAR-2021-8320b27b17 has been pushed to the Fedora 33 Modular stable repository. If problem still persists, please make note of it in this bug report. Michal, the corrected perl:5.32 module reached Fedora 33 updates repository and you should not experience the reported problem any more when using Fedora 33 installation image. This message is a reminder that Fedora 33 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '33'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 33 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. This message is a reminder that Fedora 33 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '33'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 33 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. This message is a reminder that Fedora 33 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '33'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 33 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. FEDORA-2022-e119fcc7d5 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2022-e119fcc7d5 FEDORA-2022-e119fcc7d5 has been pushed to the Fedora 37 stable repository. If problem still persists, please make note of it in this bug report. RHEL 8.4 (EUS) - SAP Development / Test / Demo System
Release Set 8.4 for SAP support requirement
libmodulemd-2.9.4-2.el8.x86_64
I am having this exact same issue in RHEL 8.4 with EUS.
--> Module yaml error: Unexpected key in data: static_context [line 9 col 3]
How do I update libmodulemd with R=8.4 (EUS)? The upgrade is not available for this RHEL version. I can not convert to 8.5 or to 9.0 because of SAP support.
An update with this fix does not exist in RHEL 8.4. I recommend you opening a new bug against libdnf of RHEL 8.4. This current bug report is for Fedora. If you don't know how to od it, please contact Red Hat support <https://access.redhat.com/support>. I also have an RHEL 8.4 for SAP system with this issue. It's probably not supported however the Centos bug fixes end up in RHEL eventually. So you might use the Centos libmodulemd https://centos.pkgs.org/8-stream/centos-baseos-x86_64/libmodulemd-2.13.0-1.el8.x86_64.rpm.html. Download and rpm -Uvh libmodulemd-2.13.0-1.el8.x86_64.rpm . Issue disappeared after this. |