RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2150920 - Image Builder fails with some customizations
Summary: Image Builder fails with some customizations
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: osbuild-composer
Version: 8.7
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Brian Lane
QA Contact: Release Test Team
Eliane Ramos Pereira
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-12-05 16:21 UTC by Christophe Besson
Modified: 2023-11-14 16:09 UTC (History)
9 users (show)

Fixed In Version: osbuild-composer-85-1.el8
Doc Type: Bug Fix
Doc Text:
Cause: The customizations.firewall example in the documentation at - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/composing_a_customized_rhel_system_image/index#image-customizations_creating-system-images-with-composer-command-line-interface is incorrect. It should be 'ports = ["PORTS"]' not 'port = ["PORTS"]' Consequence: Users construct blueprints that will not work correctly. Fix: Update the docs. Result: Happy Customers!
Clone Of:
Environment:
Last Closed: 2023-11-14 15:25:24 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-141330 0 None None None 2022-12-05 16:37:55 UTC
Red Hat Knowledge Base (Solution) 7020550 0 None None None 2023-06-21 09:19:13 UTC
Red Hat Product Errata RHEA-2023:6906 0 None None None 2023-11-14 15:25:32 UTC

Description Christophe Besson 2022-12-05 16:21:46 UTC 
Description of problem:
The below customizations were working on RHEL 8.6: services, firewall.services, firewall. After upgrading to RHEL 8.7, the below error is printed:

# composer-cli blueprints show notworking
ERROR: Show Error: Get "http://localhost/api/v1/blueprints/info/notworking?format=toml": EOF


Version-Release number of selected component (if applicable):
osbuild-composer-62-3.el8_7

How reproducible:
Always

Steps to Reproduce:
1. Create a blueprint with
[customizations.services]
enabled = ["sshd.service", "firewalld.service"]

[customizations.firewall]
port = ["22"]

[customizations.firewall.services]
enabled = ["ssh"]

2. Push it and show it


Actual results:
golang runtime error:
Dec 05 16:21:44 localhost.localdomain osbuild-composer[1169]: 2022/12/05 16:21:44 POST /api/v1/blueprints/new
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: 2022/12/05 16:21:47 GET /api/v1/blueprints/info/vmware-min
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: 2022/12/05 16:21:47 http: panic serving @: runtime error: comparing uncomparable type blueprint.ServicesCustomization
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: goroutine 93 [running]:
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: net/http.(*conn).serve.func1()
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /usr/lib/golang/src/net/http/server.go:1825 +0xbf
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: panic({0x559fb061f720, 0xc000131290})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /usr/lib/golang/src/runtime/panic.go:844 +0x258
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).safeEncode.func1()
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:152 +0x78
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: panic({0x559fb061f720, 0xc000131290})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /usr/lib/golang/src/runtime/panic.go:838 +0x207
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.isEmpty({0x559fb0674420?, 0xc000b82660?, 0x0?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:656 +0xf9
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).eStruct.func2({0xc00000e228?, 0x1, 0x559fb0746560?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:507 +0x23e
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).eStruct(0xc000b93518, {0xc000131090, 0x1, 0x1}, {0x559fb0746560?, 0xc0003ee160?, 0x1?}, 0x0)
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:529 +0x266
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).eMapOrStruct(0xc000b93518?, {0xc000131090?, 0x1?, 0x0?}, {0x559fb0746560?, 0xc0003ee160?, 0x559fb0746560?}, 0x0?)
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:370 +0x4e
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).eTable(0xc000b93518, {0xc000131090, 0x1, 0x1}, {0x559fb0746560?, 0xc0003ee160?, 0x559fb05c2c80?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:362 +0x1a5
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).encode(0x559fb05c5411?, {0xc000131090, 0x1, 0x1}, {0x559fb0746560?, 0xc0003ee160?, 0x559fb05c5411?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:202 +0x399
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).eStruct.func2({0xc00000e0f0?, 0x1, 0x559fb0729bc0?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:520 +0x425
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).eStruct(0xc000b93518, {0x559fb0f1ad78, 0x0, 0x0}, {0x559fb0729bc0?, 0xc0001ae160?, 0x2000?}, 0x0)
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:529 +0x266
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).eMapOrStruct(0xc000b93180?, {0x559fb0f1ad78?, 0x559fb062f400?, 0x559fb0729bc0?}, {0x559fb0729bc0?, 0xc0001ae160?, 0x559fb0729bc0?}, 0xc0?)
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:370 +0x4e
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).eTable(0xc000b93518, {0x559fb0f1ad78, 0x0, 0x0}, {0x559fb0729bc0?, 0xc0001ae160?, 0x1?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:362 +0x1a5
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).encode(0xc0001ae160?, {0x559fb0f1ad78, 0x0, 0x0}, {0x559fb0729bc0?, 0xc0001ae160?, 0x7fac23ee55b8?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:202 +0x399
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).safeEncode(0x559fb0729bc0?, {0x559fb0f1ad78?, 0xc000b93380?, 0x559faf654f4b?}, {0x559fb0729bc0?, 0xc0001ae160?, 0xc0001ae0b0?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:155 +0x77
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/BurntSushi/toml.(*Encoder).Encode(0xc000b93518, {0x559fb0729bc0?, 0xc0001ae160?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/BurntSushi/toml/encode.go:139 +0xc8
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/osbuild/osbuild-composer/internal/weldr.(*API).blueprintsInfoHandler(0xc00014d760, {0x559fb078e6d8?, 0xc0001f4000}, 0xc0005d0100, {0xc0005c4180?, 0x2, 0xc00037801b?})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/internal/weldr/api.go:1462 +0xbc5
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/julienschmidt/httprouter.(*Router).ServeHTTP(0xc00054eba0, {0x559fb078e6d8, 0xc0001f4000}, 0xc0005d0100)
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/vendor/github.com/julienschmidt/httprouter/router.go:387 +0x82b
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: github.com/osbuild/osbuild-composer/internal/weldr.(*API).ServeHTTP(0xc00014d760, {0x559fb078e6d8, 0xc0001f4000}, 0xc0005d0100)
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /builddir/build/BUILD/osbuild-composer-62/_build/src/github.com/osbuild/osbuild-composer/internal/weldr/api.go:299 +0x16a
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: net/http.serverHandler.ServeHTTP({0xc000134330?}, {0x559fb078e6d8, 0xc0001f4000}, 0xc0005d0100)
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /usr/lib/golang/src/net/http/server.go:2916 +0x43b
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: net/http.(*conn).serve(0xc0003fb680, {0x559fb078f5b0, 0xc00030e090})
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /usr/lib/golang/src/net/http/server.go:1966 +0x5d7
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]: created by net/http.(*Server).Serve
Dec 05 16:21:47 localhost.localdomain osbuild-composer[1169]:         /usr/lib/golang/src/net/http/server.go:3071 +0x4db

Additional info:
- the workaround could be downgrading, but I just tried and got the following error:
Dec 05 16:39:00 localhost.localdomain osbuild-composer[2195]: cannot read state: error reading db file state: unexpected target name
Comment 2 Brian Lane 2022-12-06 17:58:31 UTC 
Two unrelated issues here, the first is that you should use 'ports = ["22"]' not 'port'. That won't fix the crash though. Until that is fixed you can work around the crash by viewing the blueprint as JSON with 'composer-cli --json blueprints show notworking'.

This needs a toml library fix backported from upstream.
Comment 3 Brian Lane 2022-12-06 19:35:10 UTC 
https://github.com/osbuild/osbuild-composer/pull/3170
Comment 4 Christophe Besson 2022-12-07 11:13:52 UTC 
Ah yep, port in singular was a typo.
At a first glance I thought these directives were not taken into account anymore, but it's almost cosmetic.
Hence reducing the severity to medium.

Thanks for your prompt feedback!
Comment 5 Christophe Besson 2023-06-20 10:52:29 UTC 
Late note.

The doc needs to be amended, as it mentions "port" in singular whereas the code expects it in plural (as a list).
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/composing_a_customized_rhel_system_image/index#image-customizations_creating-system-images-with-composer-command-line-interface
Comment 6 Christophe Besson 2023-06-21 08:37:55 UTC 
Sorry to bother you again but the doc is also wrong on the expected value...

It tells:
"""
To enable lists, you can use numeric ports, or their names from the /etc/services file.
"""

Intuitively the user can define ports with ["22", "80"] or ["22/tcp", "80/tcp"] but both forms won't work, leading to a FAILED compose without any log explaining what happened ("log is empty").
This is problematic for the support as we need to rely on strace to figure out the problem:

/tmp/worker.strace:9697  15:06:08.743339 write(1<pipe:[164644]>, "{\"type\": \"https://osbuild.org/validation-error\", \"title\": \"JSON Schema validation failed\", \"success\": false, \"errors\": [{\"message\": \"'80/tcp' does not match '.:(tcp|udp|sctp|dccp)$'\", \"path\": [\"pipelines\", 1, \"stages\", 7, \"options\", \"ports\", 0]}]}\n", 248) = 248 <0.000030>

As per /usr/lib/osbuild/stages/org.osbuild.firewall, the expected pattern seems to be a numeric port or a port range followed by a dash and the protocol (which is mandatory).
 34 SCHEMA = """
 35 "additionalProperties": false,
 36 "properties": {
 37   "ports": {
 38     "description": "Ports (or port ranges) to open",
 39     "type": "array",
 40     "items": {
 41       "type": "string",
 42       "description": "A port or port range: 'portid[-portid]:protocol'",
 43       "pattern": ".:(tcp|udp|sctp|dccp)$"
 44     }
 45   },

Thanks!
Comment 7 Brian Lane 2023-06-21 15:30:04 UTC 
The documentation for blueprints is here - https://www.osbuild.org/guides/image-builder-on-premises/blueprint-reference.html#firewall
I'm not sure how we ended up with things getting out of sync.
Comment 8 Ondřej Budai 2023-07-12 09:35:51 UTC 
@elpereir This seems like an issue in the documentation. May I ask you to take a look? The upstream docs should be correct about the service customization. Many thanks! :)
Comment 9 Ondřej Budai 2023-07-12 09:44:35 UTC 
FTR, the crash was fixed upstream in https://github.com/osbuild/osbuild-composer/pull/3099.
Comment 18 errata-xmlrpc 2023-11-14 15:25:24 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (osbuild, osbuild-composer, and cockpit-composer bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:6906
Note You need to log in before you can comment on or make changes to this bug.