RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1943206 - lorax is broken if an unexisting group is present in the blueprint
Summary: lorax is broken if an unexisting group is present in the blueprint
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: lorax
Version: 8.3
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Brian Lane
QA Contact: Release Test Team
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-03-25 15:05 UTC by Christophe Besson
Modified: 2021-08-17 15:52 UTC (History)
11 users (show)

Fixed In Version: lorax-28.14.55-3.el8_3
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-08-17 15:52:36 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
lorax-28.14.55-3.el8_3.src.rpm (8.92 MB, application/x-rpm)
2021-04-09 17:37 UTC, Brian Lane 		no flags Details
composer-cli-28.14.55-3.el8_3.aarch64.rpm (84.67 KB, application/x-rpm)
2021-04-09 17:40 UTC, Brian Lane 		no flags Details
lorax-28.14.55-3.el8_3.aarch64.rpm (315.93 KB, application/x-rpm)
2021-04-09 17:40 UTC, Brian Lane 		no flags Details
lorax-composer-28.14.55-3.el8_3.aarch64.rpm (158.25 KB, application/x-rpm)
2021-04-09 17:40 UTC, Brian Lane 		no flags Details
lorax-docs-28.14.55-3.el8_3.aarch64.rpm (5.82 MB, application/x-rpm)
2021-04-09 17:40 UTC, Brian Lane 		no flags Details
lorax-lmc-novirt-28.14.55-3.el8_3.aarch64.rpm (48.48 KB, application/x-rpm)
2021-04-09 17:40 UTC, Brian Lane 		no flags Details
lorax-lmc-virt-28.14.55-3.el8_3.aarch64.rpm (48.39 KB, application/x-rpm)
2021-04-09 17:40 UTC, Brian Lane 		no flags Details
lorax-templates-generic-28.14.55-3.el8_3.aarch64.rpm (81.53 KB, application/x-rpm)
2021-04-09 17:40 UTC, Brian Lane 		no flags Details
composer-cli-28.14.55-3.el8_3.i686.rpm (84.73 KB, application/x-rpm)
2021-04-09 17:41 UTC, Brian Lane 		no flags Details
lorax-28.14.55-3.el8_3.i686.rpm (316.02 KB, application/x-rpm)
2021-04-09 17:41 UTC, Brian Lane 		no flags Details
lorax-composer-28.14.55-3.el8_3.i686.rpm (158.31 KB, application/x-rpm)
2021-04-09 17:41 UTC, Brian Lane 		no flags Details
lorax-docs-28.14.55-3.el8_3.i686.rpm (5.82 MB, application/x-rpm)
2021-04-09 17:41 UTC, Brian Lane 		no flags Details
lorax-lmc-novirt-28.14.55-3.el8_3.i686.rpm (48.54 KB, application/x-rpm)
2021-04-09 17:41 UTC, Brian Lane 		no flags Details
lorax-lmc-virt-28.14.55-3.el8_3.i686.rpm (48.46 KB, application/x-rpm)
2021-04-09 17:41 UTC, Brian Lane 		no flags Details
lorax-templates-generic-28.14.55-3.el8_3.i686.rpm (81.59 KB, application/x-rpm)
2021-04-09 17:41 UTC, Brian Lane 		no flags Details
lorax-28.14.55-3.el8_3.ppc64le.rpm (315.97 KB, application/x-rpm)
2021-04-09 17:42 UTC, Brian Lane 		no flags Details
lorax-docs-28.14.55-3.el8_3.ppc64le.rpm (5.82 MB, application/x-rpm)
2021-04-09 17:42 UTC, Brian Lane 		no flags Details
lorax-lmc-novirt-28.14.55-3.el8_3.ppc64le.rpm (48.49 KB, application/x-rpm)
2021-04-09 17:42 UTC, Brian Lane 		no flags Details
lorax-lmc-virt-28.14.55-3.el8_3.ppc64le.rpm (48.40 KB, application/x-rpm)
2021-04-09 17:42 UTC, Brian Lane 		no flags Details
lorax-templates-generic-28.14.55-3.el8_3.ppc64le.rpm (81.55 KB, application/x-rpm)
2021-04-09 17:42 UTC, Brian Lane 		no flags Details
composer-cli-28.14.55-3.el8_3.ppc64le.rpm (84.69 KB, application/x-rpm)
2021-04-09 17:43 UTC, Brian Lane 		no flags Details
lorax-composer-28.14.55-3.el8_3.ppc64le.rpm (158.26 KB, application/x-rpm)
2021-04-09 17:44 UTC, Brian Lane 		no flags Details
composer-cli-28.14.55-3.el8_3.s390x.rpm (84.69 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-28.14.55-3.el8_3.s390x.rpm (315.96 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-composer-28.14.55-3.el8_3.s390x.rpm (158.27 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-docs-28.14.55-3.el8_3.s390x.rpm (5.82 MB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-lmc-novirt-28.14.55-3.el8_3.s390x.rpm (48.50 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-lmc-virt-28.14.55-3.el8_3.s390x.rpm (48.41 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-templates-generic-28.14.55-3.el8_3.s390x.rpm (81.55 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
composer-cli-28.14.55-3.el8_3.x86_64.rpm (84.70 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-28.14.55-3.el8_3.x86_64.rpm (315.98 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-composer-28.14.55-3.el8_3.x86_64.rpm (158.28 KB, application/x-rpm)
2021-04-09 17:45 UTC, Brian Lane 		no flags Details
lorax-docs-28.14.55-3.el8_3.x86_64.rpm (5.82 MB, application/x-rpm)
2021-04-09 17:46 UTC, Brian Lane 		no flags Details
lorax-lmc-novirt-28.14.55-3.el8_3.x86_64.rpm (48.51 KB, application/x-rpm)
2021-04-09 17:46 UTC, Brian Lane 		no flags Details
lorax-lmc-virt-28.14.55-3.el8_3.x86_64.rpm (48.43 KB, application/x-rpm)
2021-04-09 17:46 UTC, Brian Lane 		no flags Details
lorax-templates-generic-28.14.55-3.el8_3.x86_64.rpm (81.56 KB, application/x-rpm)
2021-04-09 17:46 UTC, Brian Lane 		no flags Details
View All

Description Christophe Besson 2021-03-25 15:05:10 UTC 
Description of problem:
Customer reported an issue with the lorax backend when an unexisting group is defined in a blueprint.

Version-Release number of selected component (if applicable):
lorax-composer-28.14.55-2.el8.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create a minimal blueprint
~~~
name = "test"
description = "Bug reproduction"
version = "0.0.2"
 
[[modules]]
name = "nginx"
version = "*"
 
[[packages]]
name = "openssh-server"
version = "*"
 
[[groups]]
name = "this-doesnt-exist"
~~~
2. composer-cli blueprints push test.toml

Actual results:
~~~
# composer-cli blueprints freeze test
2021-03-25 09:58:45,154: Expecting value: line 1 column 1 (char 0)
~~~

Expected results:
~~~
# composer-cli blueprints freeze test
2021-03-25 10:57:54,858: test: The following package(s) had problems: Group this-doesnt-exist (L’identifiant de groupe « this-doesnt-exist » n’existe pas.)
~~~

Additional info:
* While stracing the behaviour of the backend, we can see an unhandled python trace occurred:
~~~
[2021-03-25 09:59:16,736] ERROR in app: Exception on /api/v0/blueprints/freeze/test [GET]
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib/python3.6/site-packages/pylorax/api/crossdomain.py", line 48, in wrapped_function
    resp = make_response(f(*args, **kwargs))
  File "/usr/lib/python3.6/site-packages/pylorax/api/checkparams.py", line 40, in wrapped_function
    return f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/pylorax/api/v0.py", line 1437, in v0_blueprints_freeze
    deps = projects_depsolve(api.config["DNFLOCK"].dbo, projects, blueprint.group_names)
  File "/usr/lib/python3.6/site-packages/pylorax/api/projects.py", line 274, in projects_depsolve
    _depsolve(dbo, projects, groups)
  File "/usr/lib/python3.6/site-packages/pylorax/api/projects.py", line 237, in _depsolve
    dbo.group_install(name, ["mandatory", "default"])
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 1627, in group_install
    strict)
  File "/usr/lib/python3.6/site-packages/dnf/comps.py", line 102, in install_or_skip
    return install_fnc(grp_or_env_id, types, exclude, strict, exclude_groups)
  File "/usr/lib/python3.6/site-packages/dnf/comps.py", line 666, in _group_install
    raise ValueError(_("Group_id %s does not exist.") % ucd(group_id))
ValueError: Group_id this-doesnt-exist does not exist.
~~~

* The following patch fixes the issue for me:
~~~
--- /usr/lib/python3.6/site-packages/pylorax/api/projects.py.orig	2021-03-25 10:54:48.282825076 -0400
+++ /usr/lib/python3.6/site-packages/pylorax/api/projects.py	2021-03-25 10:55:37.752743710 -0400
@@ -237,6 +237,8 @@
             dbo.group_install(name, ["mandatory", "default"])
         except dnf.exceptions.MarkingError as e:
             install_errors.append(("Group %s" % (name), str(e)))
+        except ValueError as e:
+            install_errors.append(("Group %s" % (name), str(e)))
 
     for name, version in projects:
         # Find the best package matching the name + version glob
~~~

* The customer cannot move to osbuild for now as their product is currently based on Lorax. The issue is properly handled with the new backend:
~~~
# composer-cli blueprints freeze test
2021-03-25 09:27:11,162: test: DNF error occured: MarkingErrors: Error occurred when marking packages for installation: 
missing groups or modules : @this-doesnt-exist
~~~
Comment 1 Brian Lane 2021-03-25 16:00:24 UTC 
That seems more like a bug in dnf, it should be returning that as a dnf exception.
Comment 4 Lukáš Hrázký 2021-04-01 15:21:44 UTC 
Hello, apologies for the delay.

dnf in RHEL 8.4 (dnf-4.4.2-11.el8 if I'm looking it up correctly) will raise an instance of dnf.exceptions.CompsError instead of Python's ValueError. However, lorax only catches dnf.exceptions.MarkingError (looking into /usr/lib/python3.9/site-packages/pylorax/api/projects.py on my Fedora 33). So it would appear lorax still needs to handle the CompsError that is being thrown here.

Reassigning back, sorry for the inconvenience this is causing.
Comment 8 Brian Lane 2021-04-08 23:47:44 UTC 
Here's a fix for this, but it's only going to work with 8.3 and dnf-4.2.23-4.el8.noarch -- on 8.4 with dnf-4.4.2-2.el8.noarch dnf doesn't appear to be raising any errors for missing groups.

https://github.com/weldr/lorax/pull/1133
Comment 12 Brian Lane 2021-04-09 15:40:05 UTC 
In RHEL 8.3 when using dnf-4.2.23-4 the error that dnf raises has changed to a ValueError, the patch in comment 8 works around this.

In RHEL 8.4 and later, using dnf dnf-4.4.2-2 (or -11) dnf has stopped raising any errors so livemedia-creator cannot detect the problem. I have filed bug #1947958 against dnf for this. There is nothing lmc can do on 8.4 and later.
Comment 13 Brian Lane 2021-04-09 17:37:09 UTC 
Created attachment 1770733 [details]
lorax-28.14.55-3.el8_3.src.rpm
Comment 14 Brian Lane 2021-04-09 17:40:03 UTC 
Created attachment 1770734 [details]
composer-cli-28.14.55-3.el8_3.aarch64.rpm
Comment 15 Brian Lane 2021-04-09 17:40:08 UTC 
Created attachment 1770735 [details]
lorax-28.14.55-3.el8_3.aarch64.rpm
Comment 16 Brian Lane 2021-04-09 17:40:12 UTC 
Created attachment 1770736 [details]
lorax-composer-28.14.55-3.el8_3.aarch64.rpm
Comment 17 Brian Lane 2021-04-09 17:40:22 UTC 
Created attachment 1770737 [details]
lorax-docs-28.14.55-3.el8_3.aarch64.rpm
Comment 18 Brian Lane 2021-04-09 17:40:26 UTC 
Created attachment 1770738 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.aarch64.rpm
Comment 19 Brian Lane 2021-04-09 17:40:28 UTC 
Created attachment 1770739 [details]
lorax-lmc-virt-28.14.55-3.el8_3.aarch64.rpm
Comment 20 Brian Lane 2021-04-09 17:40:32 UTC 
Created attachment 1770740 [details]
lorax-templates-generic-28.14.55-3.el8_3.aarch64.rpm
Comment 21 Brian Lane 2021-04-09 17:41:12 UTC 
Created attachment 1770741 [details]
composer-cli-28.14.55-3.el8_3.i686.rpm

composer-cli-28.14.55-3.el8_3.i686.rpm
Comment 22 Brian Lane 2021-04-09 17:41:16 UTC 
Created attachment 1770742 [details]
lorax-28.14.55-3.el8_3.i686.rpm

lorax-28.14.55-3.el8_3.i686.rpm
Comment 23 Brian Lane 2021-04-09 17:41:18 UTC 
Created attachment 1770743 [details]
lorax-composer-28.14.55-3.el8_3.i686.rpm

lorax-composer-28.14.55-3.el8_3.i686.rpm
Comment 24 Brian Lane 2021-04-09 17:41:28 UTC 
Created attachment 1770744 [details]
lorax-docs-28.14.55-3.el8_3.i686.rpm

lorax-docs-28.14.55-3.el8_3.i686.rpm
Comment 25 Brian Lane 2021-04-09 17:41:30 UTC 
Created attachment 1770745 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.i686.rpm

lorax-lmc-novirt-28.14.55-3.el8_3.i686.rpm
Comment 26 Brian Lane 2021-04-09 17:41:32 UTC 
Created attachment 1770746 [details]
lorax-lmc-virt-28.14.55-3.el8_3.i686.rpm

lorax-lmc-virt-28.14.55-3.el8_3.i686.rpm
Comment 27 Brian Lane 2021-04-09 17:41:34 UTC 
Created attachment 1770747 [details]
lorax-templates-generic-28.14.55-3.el8_3.i686.rpm

lorax-templates-generic-28.14.55-3.el8_3.i686.rpm
Comment 28 Brian Lane 2021-04-09 17:42:05 UTC 
Created attachment 1770748 [details]
lorax-28.14.55-3.el8_3.ppc64le.rpm

lorax-28.14.55-3.el8_3.ppc64le.rpm
Comment 29 Brian Lane 2021-04-09 17:42:18 UTC 
Created attachment 1770749 [details]
lorax-docs-28.14.55-3.el8_3.ppc64le.rpm

lorax-docs-28.14.55-3.el8_3.ppc64le.rpm
Comment 30 Brian Lane 2021-04-09 17:42:21 UTC 
Created attachment 1770750 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.ppc64le.rpm

lorax-lmc-novirt-28.14.55-3.el8_3.ppc64le.rpm
Comment 31 Brian Lane 2021-04-09 17:42:22 UTC 
Created attachment 1770751 [details]
lorax-lmc-virt-28.14.55-3.el8_3.ppc64le.rpm

lorax-lmc-virt-28.14.55-3.el8_3.ppc64le.rpm
Comment 32 Brian Lane 2021-04-09 17:42:25 UTC 
Created attachment 1770752 [details]
lorax-templates-generic-28.14.55-3.el8_3.ppc64le.rpm

lorax-templates-generic-28.14.55-3.el8_3.ppc64le.rpm
Comment 33 Brian Lane 2021-04-09 17:43:39 UTC 
Created attachment 1770753 [details]
composer-cli-28.14.55-3.el8_3.ppc64le.rpm
Comment 34 Brian Lane 2021-04-09 17:44:00 UTC 
Created attachment 1770754 [details]
lorax-composer-28.14.55-3.el8_3.ppc64le.rpm
Comment 35 Brian Lane 2021-04-09 17:45:17 UTC 
Created attachment 1770755 [details]
composer-cli-28.14.55-3.el8_3.s390x.rpm

composer-cli-28.14.55-3.el8_3.s390x.rpm
Comment 36 Brian Lane 2021-04-09 17:45:21 UTC 
Created attachment 1770756 [details]
lorax-28.14.55-3.el8_3.s390x.rpm

lorax-28.14.55-3.el8_3.s390x.rpm
Comment 37 Brian Lane 2021-04-09 17:45:23 UTC 
Created attachment 1770757 [details]
lorax-composer-28.14.55-3.el8_3.s390x.rpm

lorax-composer-28.14.55-3.el8_3.s390x.rpm
Comment 38 Brian Lane 2021-04-09 17:45:33 UTC 
Created attachment 1770758 [details]
lorax-docs-28.14.55-3.el8_3.s390x.rpm

lorax-docs-28.14.55-3.el8_3.s390x.rpm
Comment 39 Brian Lane 2021-04-09 17:45:36 UTC 
Created attachment 1770759 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.s390x.rpm

lorax-lmc-novirt-28.14.55-3.el8_3.s390x.rpm
Comment 40 Brian Lane 2021-04-09 17:45:38 UTC 
Created attachment 1770760 [details]
lorax-lmc-virt-28.14.55-3.el8_3.s390x.rpm

lorax-lmc-virt-28.14.55-3.el8_3.s390x.rpm
Comment 41 Brian Lane 2021-04-09 17:45:40 UTC 
Created attachment 1770761 [details]
lorax-templates-generic-28.14.55-3.el8_3.s390x.rpm

lorax-templates-generic-28.14.55-3.el8_3.s390x.rpm
Comment 42 Brian Lane 2021-04-09 17:45:53 UTC 
Created attachment 1770762 [details]
composer-cli-28.14.55-3.el8_3.x86_64.rpm

composer-cli-28.14.55-3.el8_3.x86_64.rpm
Comment 43 Brian Lane 2021-04-09 17:45:56 UTC 
Created attachment 1770763 [details]
lorax-28.14.55-3.el8_3.x86_64.rpm

lorax-28.14.55-3.el8_3.x86_64.rpm
Comment 44 Brian Lane 2021-04-09 17:45:59 UTC 
Created attachment 1770764 [details]
lorax-composer-28.14.55-3.el8_3.x86_64.rpm

lorax-composer-28.14.55-3.el8_3.x86_64.rpm
Comment 45 Brian Lane 2021-04-09 17:46:09 UTC 
Created attachment 1770765 [details]
lorax-docs-28.14.55-3.el8_3.x86_64.rpm

lorax-docs-28.14.55-3.el8_3.x86_64.rpm
Comment 46 Brian Lane 2021-04-09 17:46:12 UTC 
Created attachment 1770766 [details]
lorax-lmc-novirt-28.14.55-3.el8_3.x86_64.rpm

lorax-lmc-novirt-28.14.55-3.el8_3.x86_64.rpm
Comment 47 Brian Lane 2021-04-09 17:46:13 UTC 
Created attachment 1770767 [details]
lorax-lmc-virt-28.14.55-3.el8_3.x86_64.rpm

lorax-lmc-virt-28.14.55-3.el8_3.x86_64.rpm
Comment 48 Brian Lane 2021-04-09 17:46:16 UTC 
Created attachment 1770768 [details]
lorax-templates-generic-28.14.55-3.el8_3.x86_64.rpm

lorax-templates-generic-28.14.55-3.el8_3.x86_64.rpm
Comment 50 Christophe Besson 2021-04-12 13:21:55 UTC 
Customer also tried the 8.4 beta / rc, and he's wondering why the behaviour differs.
Is it expected composer-cli doesn't report any error on unexisting group?

It can be seen only in /var/log/lorax-composer/dnf.log:
"""
Group id 'this-doesnt-exist' does not exist, skipping.
"""

I think it should behave like the lorax el8.3 patched version or osbuild.
Comment 51 Brian Lane 2021-04-12 16:23:24 UTC 
That's what I was talking about in comment 12, DNF changed again. Also, lorax-composer is deprecated in 8.3 and has been replaced by osbuild-composer in 8.4 so they really should switch to using that as soon as they can.
Comment 53 Lukáš Hrázký 2021-04-13 13:58:22 UTC 
Brian, I've commented on dnf not raising an exception in the bug #1947958 you've created. Again, apologies for the inconvenience.
Note You need to log in before you can comment on or make changes to this bug.