Bug 1817130
| Summary: | dnf isn't able to download with encoded url | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Christophe Besson <cbesson> |
| Component: | libdnf | Assignee: | Lukáš Hrázký <lhrazky> |
| Status: | CLOSED ERRATA | QA Contact: | Eva Mrakova <emrakova> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 8.1 | CC: | lhrazky, mdomonko, nsella, pkratoch, redhat |
| Target Milestone: | rc | Keywords: | Triaged |
| Target Release: | 8.0 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | libdnf-0.48.0-1.el8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-11-04 01:52:51 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1825061 | ||
|
Description
Christophe Besson
2020-03-25 16:07:06 UTC
I tested that RHEL in the default setup is not broken (it works even without urlencode): > $ podman run -it ubi8:8.1 > > $ rpm -q dnf libdnf > dnf-4.2.7-7.el8_1.noarch > libdnf-0.35.1-9.el8_1.x86_64 > > $ dnf download libstdc++ > (1/2): libstdc++-8.3.1-4.5.el8.x86_64.rpm ... > (2/2): libstdc++-8.3.1-4.5.el8.i686.rpm ... I also verified that the upstream version of dnf/libdnf doesn't urlencode the string: > sendto(41, "GET /pub/linux/fedora/linux/development/rawhide/Everything/x86_64/os/Packages/l/libstdc++-10.0.1-0.11.fc33.x86_64.rpm...) It looks like we should really urlencode the paths. PRs: https://github.com/rpm-software-management/librepo/pull/188 https://github.com/rpm-software-management/ci-dnf-stack/pull/819 AC: Special URL characters are escaped in URLs when dnf downloads packages. Christophe, I've been prompted to have a look at the URL RFC and the plus sign is actually not a reserved character in the path part of a URL (see RFC 1738, section 3.3 and possibly 2.2 for globally unsafe characters). Therefore, according to my interpretation, in case of a '+' DNF behavior is correct, it doesn't need to be escaped and the customer server implementation is at fault here. While there potentially may be other characters that need to be escaped, those are also invalid in RPM package names. Therefore I'm not convinced my earlier PRs should be merged... We've decided to put the encoding in place to make DNF more compatible. The customer should probably be advised to check their server as well... Well, to be honest I'm not sure how to interpret that RFC. As per https://www.w3schools.com/tags/ref_urlencode.ASP : "URL encoding normally replaces a space with a plus (+) sign or with %20." As we can see in the URL while googling for example. So I think the real + sign has to be encoded. That's also what we can see in the yum strace, "+" signs are replaced in the package names. So I continue to think there is a change of behaviour with dnf. It's just too complicated; '+' for encoding a space is allowed only in the query part of a URL, which is usually in the application/x-www-form-urlencoded format, which specifies using '+' for encoding spaces. In path you can only encode as %20 as per RFC 1738. (also the fact that majority of the servers actually work with unencoded '+'es in paths speaks for something) But it is a change of behaviour from yum, indeed. Not encoding '+' in path is strict behaviour, encoding it is (not entirely correctly) making dnf a bit more compatible with non-conformant servers. Hmm, yum/urlgrabber used the following urllib function:
# python -c 'import urllib; print(urllib.pathname2url("test++"))'
test%2B%2B
I don't think this function has been designed with non-conformant servers in mind.
If I search the string "python + urllib" in google, I have the following in my browser url :)
&q=python+%2B+urllib
^^^
oh okay for the "query part", I understand. I don't know what's the best thing to do here... Yeah, query is the part after a '?'. Don't worry, as I've said, we've decided to encode it anyway, for "better compatibility". yum did it before, so it shouldn't cause trouble. I just wanted to describe the situation for completeness. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (yum bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2020:4510 |