RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1942029 - [Edge] RHEL for Edge Container Commit fails silently when Repo URL doesn't have a trailing slash
Summary: [Edge] RHEL for Edge Container Commit fails silently when Repo URL doesn't ha...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: osbuild-composer
Version: 8.4
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Achilleas Koutsou
QA Contact: Xiaofeng Wang
URL:
Whiteboard:
Depends On:
Blocks: 1951192
TreeView+ depends on / blocked
 
Reported: 2021-03-23 13:21 UTC by Ben Breard
Modified: 2021-11-10 01:16 UTC (History)
8 users (show)

Fixed In Version: osbuild-composer-31-1.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1951192 (view as bug list)
Environment:
Last Closed: 2021-11-09 18:46:58 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2021:4273 0 None None None 2021-11-09 18:47:14 UTC

Description Ben Breard 2021-03-23 13:21:24 UTC
Description of problem:
RHEL for Edge Container Commit fails silently when Repo URL doesn't have a trailing slash

Version-Release number of selected component (if applicable):
$ rpm -qa |grep composer
osbuild-composer-core-28.2-1.el8.x86_64
osbuild-composer-worker-28.2-1.el8.x86_64
composer-cli-28.14.57-1.el8.x86_64
osbuild-composer-28.2-1.el8.x86_64
cockpit-composer-29-1.el8.noarch


How reproducible:

100%

Steps to Reproduce:
1. Make an RFE commit available over http
2. Create an empty blueprint
3. create an image -> select RHEL for Edge Container 
4. In Repository URL don't add a trailing slash [http://x.x.x.x/repo]

Actual results:
The UI says it's added to the Queue and never shows up and no indication of a failure happens.

Expected results:
It should work w/o the slash, or worst case provide a helpful error message.

Additional info:

Comment 1 Yi He 2021-03-24 09:58:23 UTC
Tested in my env:

[root@builder osbuild-composer]# rpm -qa|grep compose
cockpit-composer-29-1.el8.noarch
osbuild-composer-core-28.2-1.el8.x86_64
osbuild-composer-28.2-1.el8.x86_64
composer-cli-28.14.58-1.el8.x86_64
osbuild-composer-worker-28.2-1.el8.x86_64

My steps:
1. Create an empty blueprint
2. create an image -> select RHEL for Edge Container 
3. In Repository URL don't add a trailing slash [http://x.x.x.x/repo]
4. UI shows an image is added to queue.
5. I checked the image queue, there is no image there, and I checked with command line, it also shows there is no image in process.
6. Try again to create image,in Repository URL add trailing slash [http://x.x.x.x/repo/]
7. UI shows a image is added to queue.
8. I checked the image queue, it shows a image is in process of building, also find this image in command line.

Conclusion:
If user input repository url without trailing slash, no image will be added to queue, although the UI says so.

Further investigation:
To figure out why, I tried CLI to create image without trailing slash, and CLI reject the command immediately. After I added trailing slash, the CLI accepted the command, and an image is in process of building.
[root@builder test]# composer-cli compose start-ostree emtpy rhel-edge-container --url http://192.168.100.1/repo
2021-03-24 16:18:44,505: encoding/hex: invalid byte: U+003C '<'
[root@builder test]# composer-cli compose start-ostree emtpy rhel-edge-container --url http://192.168.100.1/repo/
Compose 7f62706c-397a-42b9-9923-ee147734bc48 added to the queue

Solution:
UI should return clear message to tell customer that the URL is not correct and no image will be built. And also, the CLI should return clear message too, but not error like this "ncoding/hex: invalid byte: U+003C '<'"

Comment 2 Jacob 2021-03-24 11:29:53 UTC
In order the fix this the issue, the UI will check if the last character is a slash. For RHEL 8.4 we can show an error icon and prevent image creation if there is no trailing slash. But, we cannot display an error message since we are past the translations deadlines. In future RHEL versions, the validation of the URL will contain help text explaining why a url is invalid.

Comment 3 Brian Lane 2021-03-24 16:05:08 UTC
This error is coming from the osbuild-composer API, it should be fixed there, not in the UI.

Comment 4 Achilleas Koutsou 2021-03-24 16:19:57 UTC
The URL handling could be improved in the API, indeed.  At least the error message for invalid or unavailable URLs should be improved.

Comment 5 Yi He 2021-03-29 10:14:47 UTC
Pull request https://github.com/osbuild/osbuild-composer/pull/1329 is to fix this bug. I tested this pull request, and observed different actions:

1. For location not ending in / (http://example.org/repo):
This works as expected, with or without '/', cli can handle it. Although the command failed but this is because I didn't provide a repo there, and actually the cli handled the parameter correctly.
[root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo/
2021-03-29 05:53:21,263: ostree repository "http://localhost/repo/refs/heads/rhel/8/x86_64/edge" returned status: 404 Not
Found
[root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo
2021-03-29 05:53:23,084: ostree repository "http://localhost/repo/refs/heads/rhel/8/x86_64/edge" returned status: 404 Not
Found

2. For ref begins with / (location: http://example.org/repo/, ref: /ref):
This does not work, if the ref parameter is '/ref' or 'ref/', cli returns error message:
root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo --ref /ref
2021-03-29 05:59:48,317: Invalid ostree ref
[root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo --ref ref/
2021-03-29 06:03:39,732: Invalid ostree ref

3. For ref is a full URL (location: http://example.org/repo/, ref: http://example.com):
This does not work, cli returns error message
[root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo --ref
http://ref.com
2021-03-29 06:00:16,265: Invalid ostree ref

My env:
[root@localhost html]# rpm -qa|grep compose
osbuild-composer-worker-29-1.20210325git09904d4.el8.x86_64
osbuild-composer-29-1.20210325git09904d4.el8.x86_64
composer-cli-28.14.58-1.el8.x86_64
osbuild-composer-core-29-1.20210325git09904d4.el8.x86_64

My repo:
[osbuild-composer]
name=osbuild-composer 09904d4
baseurl=http://osbuild-composer-repos.s3-website.us-east-2.amazonaws.com/osbuild-composer/rhel-8.4/x86_64/09904d43c6d2da2bd54aa429953ea26fb9bd947b
enabled=1
gpgcheck=0
priority=5
[osbuild]
name=osbuild 5de2d3f96bd4841738846d5e4f1e0e4ed01e7a2a
baseurl=http://osbuild-composer-repos.s3-website.us-east-2.amazonaws.com/osbuild/rhel-8.4/x86_64/5de2d3f96bd4841738846d5e4f1e0e4ed01e7a2a
enabled=1
gpgcheck=0
priority=10

Comment 6 Yi He 2021-03-30 07:32:02 UTC
Pre-verified this bug with build, works as expected.

[osbuild-composer]
name=osbuild-composer 09904d4
baseurl=http://osbuild-composer-repos.s3-website.us-east-2.amazonaws.com/osbuild-composer/rhel-8.4/x86_64/09904d43c6d2da2bd54aa429953ea26fb9bd947b
enabled=1
gpgcheck=0
priority=5
[osbuild]
name=osbuild 5de2d3f96bd4841738846d5e4f1e0e4ed01e7a2a
baseurl=http://osbuild-composer-repos.s3-website.us-east-2.amazonaws.com/osbuild/rhel-8.4/x86_64/5de2d3f96bd4841738846d5e4f1e0e4ed01e7a2a
enabled=1
gpgcheck=0
priority=10

Comment 7 Achilleas Koutsou 2021-03-30 11:28:52 UTC
> 2. For ref begins with / (location: http://example.org/repo/, ref: /ref):
> This does not work, if the ref parameter is '/ref' or 'ref/', cli returns
> error message:
> root@localhost html]# composer-cli compose start-ostree ostree
> rhel-edge-container --url http://localhost/repo --ref /ref
> 2021-03-29 05:59:48,317: Invalid ostree ref
> [root@localhost html]# composer-cli compose start-ostree ostree
> rhel-edge-container --url http://localhost/repo --ref ref/
> 2021-03-29 06:03:39,732: Invalid ostree ref
> 
> 3. For ref is a full URL (location: http://example.org/repo/, ref:
> http://example.com):
> This does not work, cli returns error message
> [root@localhost html]# composer-cli compose start-ostree ostree
> rhel-edge-container --url http://localhost/repo --ref
> http://ref.com
> 2021-03-29 06:00:16,265: Invalid ostree ref


This is correct now since ostree refs can't start with a / and can't contain :. The URL parser doesn't check for this, but the API validates the ref before parsing.

The validation is based on the ostree source: https://github.com/ostreedev/ostree/blob/47bf29fed3b99d72c153d3c1581bf9a19c7a9b6d/src/libostree/ostree-core.c#L151-L152

Comment 8 Yi He 2021-03-31 01:20:20 UTC
@akoutsou Will this fix be included in RHEL8.4?

Comment 9 Achilleas Koutsou 2021-03-31 12:42:58 UTC
This wont be fixed for 8.4.

Comment 16 Xiaofeng Wang 2021-07-20 14:14:23 UTC
I've marked this bug as [Verified:Tested] with scratch/mock build. I still need a RHEL 8.5 osbuild/osbuild-composer to verify this bug finally.

Comment 17 Alexander Todorov 2021-08-24 07:42:43 UTC
(In reply to Xiaofeng Wang from comment #16)
> I've marked this bug as [Verified:Tested] with scratch/mock build. I still
> need a RHEL 8.5 osbuild/osbuild-composer to verify this bug finally.

@Xiaofeng, @Yi,
can you verify with the latest official builds in a nightly/devel tree ?

Comment 18 Yi He 2021-08-26 09:54:49 UTC
Verified this bug with nightly build, the osbuild-composer command line can accept url with or without a trailing slash.

Verify steps:
1. build edge installer iso image with trailing slash
[root@localhost workspace]# composer-cli compose start-ostree installer edge-installer --url http://192.168.100.1/repo/ --ref rhel/8/x86_64/edge
Compose d1ec10fd-922c-443b-b4f4-5c0ed4e52dc1 added to the queue
2. build edge installer iso image without trailing slash.
[root@localhost workspace]# composer-cli compose start-ostree installer edge-installer --url http://192.168.100.1/repo --ref rhel/8/x86_64/edge
Compose 4cc4f519-99b1-4317-8094-0530f0c26e0d added to the queue

My env:
[root@localhost osbuild-composer]# cat /etc/os-release 
NAME="Red Hat Enterprise Linux"
VERSION="8.5 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.5"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.5 Beta (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.5
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.5 Beta"


[root@localhost osbuild-composer]# rpm -qi osbuild-composer
Name        : osbuild-composer
Version     : 31
Release     : 1.el8
Architecture: x86_64
Install Date: Thu 26 Aug 2021 05:33:03 AM EDT
Group       : Unspecified
Size        : 17297
License     : ASL 2.0
Signature   : RSA/SHA256, Thu 19 Aug 2021 03:21:33 PM EDT, Key ID 199e2f91fd431d51
Source RPM  : osbuild-composer-31-1.el8.src.rpm
Build Date  : Tue 17 Aug 2021 06:52:44 AM EDT
Build Host  : x86-vm-54.build.eng.bos.redhat.com
Relocations : (not relocatable)
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor      : Red Hat, Inc.
URL         : https://github.com/osbuild/osbuild-composer
Summary     : An image building service based on osbuild

Comment 19 Xiaofeng Wang 2021-09-01 04:29:17 UTC
It's verified already. Just to clear Needinfo.

Comment 21 errata-xmlrpc 2021-11-09 18:46:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (osbuild bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4273


Note You need to log in before you can comment on or make changes to this bug.