Bug 1943206
That seems more like a bug in dnf, it should be returning that as a dnf exception. Hello, apologies for the delay. dnf in RHEL 8.4 (dnf-4.4.2-11.el8 if I'm looking it up correctly) will raise an instance of dnf.exceptions.CompsError instead of Python's ValueError. However, lorax only catches dnf.exceptions.MarkingError (looking into /usr/lib/python3.9/site-packages/pylorax/api/projects.py on my Fedora 33). So it would appear lorax still needs to handle the CompsError that is being thrown here. Reassigning back, sorry for the inconvenience this is causing. Here's a fix for this, but it's only going to work with 8.3 and dnf-4.2.23-4.el8.noarch -- on 8.4 with dnf-4.4.2-2.el8.noarch dnf doesn't appear to be raising any errors for missing groups. https://github.com/weldr/lorax/pull/1133 In RHEL 8.3 when using dnf-4.2.23-4 the error that dnf raises has changed to a ValueError, the patch in comment 8 works around this. In RHEL 8.4 and later, using dnf dnf-4.4.2-2 (or -11) dnf has stopped raising any errors so livemedia-creator cannot detect the problem. I have filed bug #1947958 against dnf for this. There is nothing lmc can do on 8.4 and later. Created attachment 1770733 [details]
lorax-28.14.55-3.el8_3.src.rpm
Created attachment 1770734 [details]
composer-cli-28.14.55-3.el8_3.aarch64.rpm
Created attachment 1770735 [details]
lorax-28.14.55-3.el8_3.aarch64.rpm
Created attachment 1770736 [details]
lorax-composer-28.14.55-3.el8_3.aarch64.rpm
Created attachment 1770737 [details]
lorax-docs-28.14.55-3.el8_3.aarch64.rpm
Created attachment 1770738 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.aarch64.rpm
Created attachment 1770739 [details]
lorax-lmc-virt-28.14.55-3.el8_3.aarch64.rpm
Created attachment 1770740 [details]
lorax-templates-generic-28.14.55-3.el8_3.aarch64.rpm
Created attachment 1770741 [details]
composer-cli-28.14.55-3.el8_3.i686.rpm
composer-cli-28.14.55-3.el8_3.i686.rpm
Created attachment 1770742 [details]
lorax-28.14.55-3.el8_3.i686.rpm
lorax-28.14.55-3.el8_3.i686.rpm
Created attachment 1770743 [details]
lorax-composer-28.14.55-3.el8_3.i686.rpm
lorax-composer-28.14.55-3.el8_3.i686.rpm
Created attachment 1770744 [details]
lorax-docs-28.14.55-3.el8_3.i686.rpm
lorax-docs-28.14.55-3.el8_3.i686.rpm
Created attachment 1770745 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.i686.rpm
lorax-lmc-novirt-28.14.55-3.el8_3.i686.rpm
Created attachment 1770746 [details]
lorax-lmc-virt-28.14.55-3.el8_3.i686.rpm
lorax-lmc-virt-28.14.55-3.el8_3.i686.rpm
Created attachment 1770747 [details]
lorax-templates-generic-28.14.55-3.el8_3.i686.rpm
lorax-templates-generic-28.14.55-3.el8_3.i686.rpm
Created attachment 1770748 [details]
lorax-28.14.55-3.el8_3.ppc64le.rpm
lorax-28.14.55-3.el8_3.ppc64le.rpm
Created attachment 1770749 [details]
lorax-docs-28.14.55-3.el8_3.ppc64le.rpm
lorax-docs-28.14.55-3.el8_3.ppc64le.rpm
Created attachment 1770750 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.ppc64le.rpm
lorax-lmc-novirt-28.14.55-3.el8_3.ppc64le.rpm
Created attachment 1770751 [details]
lorax-lmc-virt-28.14.55-3.el8_3.ppc64le.rpm
lorax-lmc-virt-28.14.55-3.el8_3.ppc64le.rpm
Created attachment 1770752 [details]
lorax-templates-generic-28.14.55-3.el8_3.ppc64le.rpm
lorax-templates-generic-28.14.55-3.el8_3.ppc64le.rpm
Created attachment 1770753 [details]
composer-cli-28.14.55-3.el8_3.ppc64le.rpm
Created attachment 1770754 [details]
lorax-composer-28.14.55-3.el8_3.ppc64le.rpm
Created attachment 1770755 [details]
composer-cli-28.14.55-3.el8_3.s390x.rpm
composer-cli-28.14.55-3.el8_3.s390x.rpm
Created attachment 1770756 [details]
lorax-28.14.55-3.el8_3.s390x.rpm
lorax-28.14.55-3.el8_3.s390x.rpm
Created attachment 1770757 [details]
lorax-composer-28.14.55-3.el8_3.s390x.rpm
lorax-composer-28.14.55-3.el8_3.s390x.rpm
Created attachment 1770758 [details]
lorax-docs-28.14.55-3.el8_3.s390x.rpm
lorax-docs-28.14.55-3.el8_3.s390x.rpm
Created attachment 1770759 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.s390x.rpm
lorax-lmc-novirt-28.14.55-3.el8_3.s390x.rpm
Created attachment 1770760 [details]
lorax-lmc-virt-28.14.55-3.el8_3.s390x.rpm
lorax-lmc-virt-28.14.55-3.el8_3.s390x.rpm
Created attachment 1770761 [details]
lorax-templates-generic-28.14.55-3.el8_3.s390x.rpm
lorax-templates-generic-28.14.55-3.el8_3.s390x.rpm
Created attachment 1770762 [details]
composer-cli-28.14.55-3.el8_3.x86_64.rpm
composer-cli-28.14.55-3.el8_3.x86_64.rpm
Created attachment 1770763 [details]
lorax-28.14.55-3.el8_3.x86_64.rpm
lorax-28.14.55-3.el8_3.x86_64.rpm
Created attachment 1770764 [details]
lorax-composer-28.14.55-3.el8_3.x86_64.rpm
lorax-composer-28.14.55-3.el8_3.x86_64.rpm
Created attachment 1770765 [details]
lorax-docs-28.14.55-3.el8_3.x86_64.rpm
lorax-docs-28.14.55-3.el8_3.x86_64.rpm
Created attachment 1770766 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.x86_64.rpm
lorax-lmc-novirt-28.14.55-3.el8_3.x86_64.rpm
Created attachment 1770767 [details]
lorax-lmc-virt-28.14.55-3.el8_3.x86_64.rpm
lorax-lmc-virt-28.14.55-3.el8_3.x86_64.rpm
Created attachment 1770768 [details]
lorax-templates-generic-28.14.55-3.el8_3.x86_64.rpm
lorax-templates-generic-28.14.55-3.el8_3.x86_64.rpm
Customer also tried the 8.4 beta / rc, and he's wondering why the behaviour differs. Is it expected composer-cli doesn't report any error on unexisting group? It can be seen only in /var/log/lorax-composer/dnf.log: """ Group id 'this-doesnt-exist' does not exist, skipping. """ I think it should behave like the lorax el8.3 patched version or osbuild. That's what I was talking about in comment 12, DNF changed again. Also, lorax-composer is deprecated in 8.3 and has been replaced by osbuild-composer in 8.4 so they really should switch to using that as soon as they can. Brian, I've commented on dnf not raising an exception in the bug #1947958 you've created. Again, apologies for the inconvenience. |
Description of problem: Customer reported an issue with the lorax backend when an unexisting group is defined in a blueprint. Version-Release number of selected component (if applicable): lorax-composer-28.14.55-2.el8.x86_64 How reproducible: 100% Steps to Reproduce: 1. Create a minimal blueprint ~~~ name = "test" description = "Bug reproduction" version = "0.0.2" [[modules]] name = "nginx" version = "*" [[packages]] name = "openssh-server" version = "*" [[groups]] name = "this-doesnt-exist" ~~~ 2. composer-cli blueprints push test.toml Actual results: ~~~ # composer-cli blueprints freeze test 2021-03-25 09:58:45,154: Expecting value: line 1 column 1 (char 0) ~~~ Expected results: ~~~ # composer-cli blueprints freeze test 2021-03-25 10:57:54,858: test: The following package(s) had problems: Group this-doesnt-exist (L’identifiant de groupe « this-doesnt-exist » n’existe pas.) ~~~ Additional info: * While stracing the behaviour of the backend, we can see an unhandled python trace occurred: ~~~ [2021-03-25 09:59:16,736] ERROR in app: Exception on /api/v0/blueprints/freeze/test [GET] Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise raise value File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/lib/python3.6/site-packages/pylorax/api/crossdomain.py", line 48, in wrapped_function resp = make_response(f(*args, **kwargs)) File "/usr/lib/python3.6/site-packages/pylorax/api/checkparams.py", line 40, in wrapped_function return f(*args, **kwargs) File "/usr/lib/python3.6/site-packages/pylorax/api/v0.py", line 1437, in v0_blueprints_freeze deps = projects_depsolve(api.config["DNFLOCK"].dbo, projects, blueprint.group_names) File "/usr/lib/python3.6/site-packages/pylorax/api/projects.py", line 274, in projects_depsolve _depsolve(dbo, projects, groups) File "/usr/lib/python3.6/site-packages/pylorax/api/projects.py", line 237, in _depsolve dbo.group_install(name, ["mandatory", "default"]) File "/usr/lib/python3.6/site-packages/dnf/base.py", line 1627, in group_install strict) File "/usr/lib/python3.6/site-packages/dnf/comps.py", line 102, in install_or_skip return install_fnc(grp_or_env_id, types, exclude, strict, exclude_groups) File "/usr/lib/python3.6/site-packages/dnf/comps.py", line 666, in _group_install raise ValueError(_("Group_id %s does not exist.") % ucd(group_id)) ValueError: Group_id this-doesnt-exist does not exist. ~~~ * The following patch fixes the issue for me: ~~~ --- /usr/lib/python3.6/site-packages/pylorax/api/projects.py.orig 2021-03-25 10:54:48.282825076 -0400 +++ /usr/lib/python3.6/site-packages/pylorax/api/projects.py 2021-03-25 10:55:37.752743710 -0400 @@ -237,6 +237,8 @@ dbo.group_install(name, ["mandatory", "default"]) except dnf.exceptions.MarkingError as e: install_errors.append(("Group %s" % (name), str(e))) + except ValueError as e: + install_errors.append(("Group %s" % (name), str(e))) for name, version in projects: # Find the best package matching the name + version glob ~~~ * The customer cannot move to osbuild for now as their product is currently based on Lorax. The issue is properly handled with the new backend: ~~~ # composer-cli blueprints freeze test 2021-03-25 09:27:11,162: test: DNF error occured: MarkingErrors: Error occurred when marking packages for installation: missing groups or modules : @this-doesnt-exist ~~~