Bug 1942029

Tested in my env:

[root@builder osbuild-composer]# rpm -qa|grep compose
cockpit-composer-29-1.el8.noarch
osbuild-composer-core-28.2-1.el8.x86_64
osbuild-composer-28.2-1.el8.x86_64
composer-cli-28.14.58-1.el8.x86_64
osbuild-composer-worker-28.2-1.el8.x86_64

My steps:
1. Create an empty blueprint
2. create an image -> select RHEL for Edge Container 
3. In Repository URL don't add a trailing slash [http://x.x.x.x/repo]
4. UI shows an image is added to queue.
5. I checked the image queue, there is no image there, and I checked with command line, it also shows there is no image in process.
6. Try again to create image,in Repository URL add trailing slash [http://x.x.x.x/repo/]
7. UI shows a image is added to queue.
8. I checked the image queue, it shows a image is in process of building, also find this image in command line.

Conclusion:
If user input repository url without trailing slash, no image will be added to queue, although the UI says so.

Further investigation:
To figure out why, I tried CLI to create image without trailing slash, and CLI reject the command immediately. After I added trailing slash, the CLI accepted the command, and an image is in process of building.
[root@builder test]# composer-cli compose start-ostree emtpy rhel-edge-container --url http://192.168.100.1/repo
2021-03-24 16:18:44,505: encoding/hex: invalid byte: U+003C '<'
[root@builder test]# composer-cli compose start-ostree emtpy rhel-edge-container --url http://192.168.100.1/repo/
Compose 7f62706c-397a-42b9-9923-ee147734bc48 added to the queue

Solution:
UI should return clear message to tell customer that the URL is not correct and no image will be built. And also, the CLI should return clear message too, but not error like this "ncoding/hex: invalid byte: U+003C '<'"

In order the fix this the issue, the UI will check if the last character is a slash. For RHEL 8.4 we can show an error icon and prevent image creation if there is no trailing slash. But, we cannot display an error message since we are past the translations deadlines. In future RHEL versions, the validation of the URL will contain help text explaining why a url is invalid.

This error is coming from the osbuild-composer API, it should be fixed there, not in the UI.

The URL handling could be improved in the API, indeed.  At least the error message for invalid or unavailable URLs should be improved.

Pull request https://github.com/osbuild/osbuild-composer/pull/1329 is to fix this bug. I tested this pull request, and observed different actions:

1. For location not ending in / (http://example.org/repo):
This works as expected, with or without '/', cli can handle it. Although the command failed but this is because I didn't provide a repo there, and actually the cli handled the parameter correctly.
[root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo/
2021-03-29 05:53:21,263: ostree repository "http://localhost/repo/refs/heads/rhel/8/x86_64/edge" returned status: 404 Not
Found
[root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo
2021-03-29 05:53:23,084: ostree repository "http://localhost/repo/refs/heads/rhel/8/x86_64/edge" returned status: 404 Not
Found

2. For ref begins with / (location: http://example.org/repo/, ref: /ref):
This does not work, if the ref parameter is '/ref' or 'ref/', cli returns error message:
root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo --ref /ref
2021-03-29 05:59:48,317: Invalid ostree ref
[root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo --ref ref/
2021-03-29 06:03:39,732: Invalid ostree ref

3. For ref is a full URL (location: http://example.org/repo/, ref: http://example.com):
This does not work, cli returns error message
[root@localhost html]# composer-cli compose start-ostree ostree rhel-edge-container --url http://localhost/repo --ref
http://ref.com
2021-03-29 06:00:16,265: Invalid ostree ref

My env:
[root@localhost html]# rpm -qa|grep compose
osbuild-composer-worker-29-1.20210325git09904d4.el8.x86_64
osbuild-composer-29-1.20210325git09904d4.el8.x86_64
composer-cli-28.14.58-1.el8.x86_64
osbuild-composer-core-29-1.20210325git09904d4.el8.x86_64

My repo:
[osbuild-composer]
name=osbuild-composer 09904d4
baseurl=http://osbuild-composer-repos.s3-website.us-east-2.amazonaws.com/osbuild-composer/rhel-8.4/x86_64/09904d43c6d2da2bd54aa429953ea26fb9bd947b
enabled=1
gpgcheck=0
priority=5
[osbuild]
name=osbuild 5de2d3f96bd4841738846d5e4f1e0e4ed01e7a2a
baseurl=http://osbuild-composer-repos.s3-website.us-east-2.amazonaws.com/osbuild/rhel-8.4/x86_64/5de2d3f96bd4841738846d5e4f1e0e4ed01e7a2a
enabled=1
gpgcheck=0
priority=10

Pre-verified this bug with build, works as expected.

[osbuild-composer]
name=osbuild-composer 09904d4
baseurl=http://osbuild-composer-repos.s3-website.us-east-2.amazonaws.com/osbuild-composer/rhel-8.4/x86_64/09904d43c6d2da2bd54aa429953ea26fb9bd947b
enabled=1
gpgcheck=0
priority=5
[osbuild]
name=osbuild 5de2d3f96bd4841738846d5e4f1e0e4ed01e7a2a
baseurl=http://osbuild-composer-repos.s3-website.us-east-2.amazonaws.com/osbuild/rhel-8.4/x86_64/5de2d3f96bd4841738846d5e4f1e0e4ed01e7a2a
enabled=1
gpgcheck=0
priority=10

> 2. For ref begins with / (location: http://example.org/repo/, ref: /ref):
> This does not work, if the ref parameter is '/ref' or 'ref/', cli returns
> error message:
> root@localhost html]# composer-cli compose start-ostree ostree
> rhel-edge-container --url http://localhost/repo --ref /ref
> 2021-03-29 05:59:48,317: Invalid ostree ref
> [root@localhost html]# composer-cli compose start-ostree ostree
> rhel-edge-container --url http://localhost/repo --ref ref/
> 2021-03-29 06:03:39,732: Invalid ostree ref
> 
> 3. For ref is a full URL (location: http://example.org/repo/, ref:
> http://example.com):
> This does not work, cli returns error message
> [root@localhost html]# composer-cli compose start-ostree ostree
> rhel-edge-container --url http://localhost/repo --ref
> http://ref.com
> 2021-03-29 06:00:16,265: Invalid ostree ref


This is correct now since ostree refs can't start with a / and can't contain :. The URL parser doesn't check for this, but the API validates the ref before parsing.

The validation is based on the ostree source: https://github.com/ostreedev/ostree/blob/47bf29fed3b99d72c153d3c1581bf9a19c7a9b6d/src/libostree/ostree-core.c#L151-L152

@akoutsou Will this fix be included in RHEL8.4?

This wont be fixed for 8.4.

I've marked this bug as [Verified:Tested] with scratch/mock build. I still need a RHEL 8.5 osbuild/osbuild-composer to verify this bug finally.

(In reply to Xiaofeng Wang from comment #16)
> I've marked this bug as [Verified:Tested] with scratch/mock build. I still
> need a RHEL 8.5 osbuild/osbuild-composer to verify this bug finally.

@Xiaofeng, @Yi,
can you verify with the latest official builds in a nightly/devel tree ?

Verified this bug with nightly build, the osbuild-composer command line can accept url with or without a trailing slash.

Verify steps:
1. build edge installer iso image with trailing slash
[root@localhost workspace]# composer-cli compose start-ostree installer edge-installer --url http://192.168.100.1/repo/ --ref rhel/8/x86_64/edge
Compose d1ec10fd-922c-443b-b4f4-5c0ed4e52dc1 added to the queue
2. build edge installer iso image without trailing slash.
[root@localhost workspace]# composer-cli compose start-ostree installer edge-installer --url http://192.168.100.1/repo --ref rhel/8/x86_64/edge
Compose 4cc4f519-99b1-4317-8094-0530f0c26e0d added to the queue

My env:
[root@localhost osbuild-composer]# cat /etc/os-release 
NAME="Red Hat Enterprise Linux"
VERSION="8.5 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.5"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.5 Beta (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.5
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.5 Beta"


[root@localhost osbuild-composer]# rpm -qi osbuild-composer
Name        : osbuild-composer
Version     : 31
Release     : 1.el8
Architecture: x86_64
Install Date: Thu 26 Aug 2021 05:33:03 AM EDT
Group       : Unspecified
Size        : 17297
License     : ASL 2.0
Signature   : RSA/SHA256, Thu 19 Aug 2021 03:21:33 PM EDT, Key ID 199e2f91fd431d51
Source RPM  : osbuild-composer-31-1.el8.src.rpm
Build Date  : Tue 17 Aug 2021 06:52:44 AM EDT
Build Host  : x86-vm-54.build.eng.bos.redhat.com
Relocations : (not relocatable)
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor      : Red Hat, Inc.
URL         : https://github.com/osbuild/osbuild-composer
Summary     : An image building service based on osbuild

It's verified already. Just to clear Needinfo.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (osbuild bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4273