Bug 1452750 - glibc: switch to libidn2
Summary: glibc: switch to libidn2
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: glibc
Version: 28
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Florian Weimer
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1533121 1556954
Blocks: fedora28-switch-to-idna2008 CVE-2016-6261 CVE-2016-6263 CVE-2017-14062
TreeView+ depends on / blocked
 
Reported: 2017-05-19 14:38 UTC by Nikos Mavrogiannopoulos
Modified: 2018-05-27 19:31 UTC (History)
10 users (show)

Fixed In Version: glibc-2.26.9000-42.fc28 glibc-2.27-15.fc28
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-05-24 13:41:51 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1543021 None CLOSED libidn2: Guidance needed for avoiding STD3 rules 2019-06-21 07:21:57 UTC

Internal Links: 1543021

Description Nikos Mavrogiannopoulos 2017-05-19 14:38:03 UTC
Description of problem:
Internationalized domain names exist for quite some time (IDNA2003), although the protocols describing them have evolved in an incompatible way (IDNA2008). These incompatibilities will prevent applications written for IDNA2003 to access certain problematic domain names defined with IDNA2008, e.g., faß.de is translated to domain xn--fa-hia.de with IDNA2008, while in IDNA2003 it is translated to fass.de domain. That not only causes incompatibility problems, but may be used as an attack vector to redirect users to different web sites.

The change is about deprecating libidn, which supports IDNA2003, and switch all applications using libidn, to libidn2 2.0.0, which supports IDNA2008. The switch should be transparent as the libidn2 library is API compatible. See instructions at:
https://libidn.gitlab.io/libidn2/manual/libidn2.html#Converting-from-libidn


This is part of the IDNA2008 change:
https://fedoraproject.org/wiki/Changes/IDNA2008

If upstream is not aware of that change please involve them on the process.

Comment 1 Nikos Mavrogiannopoulos 2017-05-19 14:38:44 UTC
This was opened because glibc bundles a libidn version.

Comment 2 Jan Kurik 2017-08-15 07:24:55 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle.
Changing version to '27'.

Comment 3 Florian Weimer 2018-01-09 14:03:26 UTC
I'm working on this, but I'm blocked by an apparent lack of functionality in libidn2: It does not seem to be possible to convert back from an IDNA domain name to the original.  I understand that the conversion is lossy, but the glibc interfaces require it, and a large part of IDNA2008 was motivated by providing reverse mappings for letters like “ß”.

Comment 4 Nikos Mavrogiannopoulos 2018-01-09 14:24:52 UTC
(In reply to Florian Weimer from comment #3)
> I'm working on this, but I'm blocked by an apparent lack of functionality in
> libidn2: It does not seem to be possible to convert back from an IDNA domain
> name to the original.  I understand that the conversion is lossy, but the
> glibc interfaces require it, and a large part of IDNA2008 was motivated by
> providing reverse mappings for letters like “ß”.

It is, possible to convert back. You'll need however libidn2 2.0.x. See idn2_to_ascii_8z and idn2_to_unicode_8z8z.

https://libidn.gitlab.io/libidn2/manual/libidn2.html#Library-Functions

Comment 5 Florian Weimer 2018-01-09 14:26:20 UTC
(In reply to Nikos Mavrogiannopoulos from comment #4)
> (In reply to Florian Weimer from comment #3)
> > I'm working on this, but I'm blocked by an apparent lack of functionality in
> > libidn2: It does not seem to be possible to convert back from an IDNA domain
> > name to the original.  I understand that the conversion is lossy, but the
> > glibc interfaces require it, and a large part of IDNA2008 was motivated by
> > providing reverse mappings for letters like “ß”.
> 
> It is, possible to convert back. You'll need however libidn2 2.0.x. See
> idn2_to_ascii_8z and idn2_to_unicode_8z8z.

Thanks.  I need to work on locale strings, so I actually need idn2+to_unicode_lzlz, which is somewhat confusingly named as it works on locale strings, not UTF-8 strings.

Comment 6 Zbigniew Jędrzejewski-Szmek 2018-02-07 13:38:55 UTC
This change has hit rawhide, so please advise:
how to detect at compilation time whether glibc uses libidn2?

We get the following warning in systemd:
../src/basic/socket-util.c: In function 'socknameinfo_pretty':
../src/basic/socket-util.c:742:13: warning: NI_IDN_USE_STD3_ASCII_RULES is deprecated
         r = getnameinfo(&sa->sa, salen, host, sizeof(host), NULL, 0, IDN_FLAGS);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Actually, I think that the deprecation of that constant is premature, because it makes writing code that compiles cleanly on both old and new glibc harder.

Comment 7 Florian Weimer 2018-02-07 13:56:16 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #6)
> This change has hit rawhide, so please advise:
> how to detect at compilation time whether glibc uses libidn2?
> 
> We get the following warning in systemd:
> ../src/basic/socket-util.c: In function 'socknameinfo_pretty':
> ../src/basic/socket-util.c:742:13: warning: NI_IDN_USE_STD3_ASCII_RULES is
> deprecated
>          r = getnameinfo(&sa->sa, salen, host, sizeof(host), NULL, 0,
> IDN_FLAGS);
>              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> Actually, I think that the deprecation of that constant is premature,
> because it makes writing code that compiles cleanly on both old and new
> glibc harder.

Ah.  The real question is why you feel that you must use NI_IDN_USE_STD3_ASCII_RULES, considering that it rejects _ in IDNA names, which are quite frequently used in non-IDNA names.  Could you shed some light on that?

(The current behavior implemented in glibc in rawhide is not what I intended as far as the STD3 rules are concerned.  It will need fixing either way.)

Comment 8 Florian Weimer 2018-02-07 15:41:23 UTC
(In reply to Florian Weimer from comment #7)
> Ah.  The real question is why you feel that you must use
> NI_IDN_USE_STD3_ASCII_RULES, considering that it rejects _ in IDNA names,
> which are quite frequently used in non-IDNA names.  Could you shed some
> light on that?
> 
> (The current behavior implemented in glibc in rawhide is not what I intended
> as far as the STD3 rules are concerned.  It will need fixing either way.)

I filed bug 1543021 regarding the libidn2 implementation of STD3 rules.  I still think that we want to allow _ in names unconditionally.  (Shell meta-characters will still be disallowed.)  But there doesn't seem to be a good way to get that with current libidn2, unfortunately.

Comment 9 Zbigniew Jędrzejewski-Szmek 2018-02-09 12:34:34 UTC
> The real question is why you feel that you must use NI_IDN_USE_STD3_ASCII_RULES, considering that it rejects _ in IDNA names, which are quite frequently used in non-IDNA names.  Could you shed some light on that?

After looking at this upstream, we realized we don't need it. Yay!
https://github.com/systemd/systemd/pull/8145

Comment 10 Fedora End Of Life 2018-02-20 15:33:42 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 28 development cycle.
Changing version to '28'.

Comment 11 Fedora Update System 2018-05-24 17:42:15 UTC
glibc-2.27-15.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-916dfe0d86

Comment 12 Fedora Update System 2018-05-25 18:45:07 UTC
glibc-2.27-15.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-916dfe0d86

Comment 13 Fedora Update System 2018-05-27 19:31:45 UTC
glibc-2.27-15.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.