Bug 1452750

This was opened because glibc bundles a libidn version.

This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle.
Changing version to '27'.

I'm working on this, but I'm blocked by an apparent lack of functionality in libidn2: It does not seem to be possible to convert back from an IDNA domain name to the original.  I understand that the conversion is lossy, but the glibc interfaces require it, and a large part of IDNA2008 was motivated by providing reverse mappings for letters like “ß”.

(In reply to Florian Weimer from comment #3)
> I'm working on this, but I'm blocked by an apparent lack of functionality in
> libidn2: It does not seem to be possible to convert back from an IDNA domain
> name to the original.  I understand that the conversion is lossy, but the
> glibc interfaces require it, and a large part of IDNA2008 was motivated by
> providing reverse mappings for letters like “ß”.

It is, possible to convert back. You'll need however libidn2 2.0.x. See idn2_to_ascii_8z and idn2_to_unicode_8z8z.

https://libidn.gitlab.io/libidn2/manual/libidn2.html#Library-Functions

(In reply to Nikos Mavrogiannopoulos from comment #4)
> (In reply to Florian Weimer from comment #3)
> > I'm working on this, but I'm blocked by an apparent lack of functionality in
> > libidn2: It does not seem to be possible to convert back from an IDNA domain
> > name to the original.  I understand that the conversion is lossy, but the
> > glibc interfaces require it, and a large part of IDNA2008 was motivated by
> > providing reverse mappings for letters like “ß”.
> 
> It is, possible to convert back. You'll need however libidn2 2.0.x. See
> idn2_to_ascii_8z and idn2_to_unicode_8z8z.

Thanks.  I need to work on locale strings, so I actually need idn2+to_unicode_lzlz, which is somewhat confusingly named as it works on locale strings, not UTF-8 strings.

This change has hit rawhide, so please advise:
how to detect at compilation time whether glibc uses libidn2?

We get the following warning in systemd:
../src/basic/socket-util.c: In function 'socknameinfo_pretty':
../src/basic/socket-util.c:742:13: warning: NI_IDN_USE_STD3_ASCII_RULES is deprecated
         r = getnameinfo(&sa->sa, salen, host, sizeof(host), NULL, 0, IDN_FLAGS);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Actually, I think that the deprecation of that constant is premature, because it makes writing code that compiles cleanly on both old and new glibc harder.

(In reply to Zbigniew Jędrzejewski-Szmek from comment #6)
> This change has hit rawhide, so please advise:
> how to detect at compilation time whether glibc uses libidn2?
> 
> We get the following warning in systemd:
> ../src/basic/socket-util.c: In function 'socknameinfo_pretty':
> ../src/basic/socket-util.c:742:13: warning: NI_IDN_USE_STD3_ASCII_RULES is
> deprecated
>          r = getnameinfo(&sa->sa, salen, host, sizeof(host), NULL, 0,
> IDN_FLAGS);
>              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> Actually, I think that the deprecation of that constant is premature,
> because it makes writing code that compiles cleanly on both old and new
> glibc harder.

Ah.  The real question is why you feel that you must use NI_IDN_USE_STD3_ASCII_RULES, considering that it rejects _ in IDNA names, which are quite frequently used in non-IDNA names.  Could you shed some light on that?

(The current behavior implemented in glibc in rawhide is not what I intended as far as the STD3 rules are concerned.  It will need fixing either way.)

(In reply to Florian Weimer from comment #7)
> Ah.  The real question is why you feel that you must use
> NI_IDN_USE_STD3_ASCII_RULES, considering that it rejects _ in IDNA names,
> which are quite frequently used in non-IDNA names.  Could you shed some
> light on that?
> 
> (The current behavior implemented in glibc in rawhide is not what I intended
> as far as the STD3 rules are concerned.  It will need fixing either way.)

I filed bug 1543021 regarding the libidn2 implementation of STD3 rules.  I still think that we want to allow _ in names unconditionally.  (Shell meta-characters will still be disallowed.)  But there doesn't seem to be a good way to get that with current libidn2, unfortunately.

> The real question is why you feel that you must use NI_IDN_USE_STD3_ASCII_RULES, considering that it rejects _ in IDNA names, which are quite frequently used in non-IDNA names.  Could you shed some light on that?

After looking at this upstream, we realized we don't need it. Yay!
https://github.com/systemd/systemd/pull/8145

This bug appears to have been reported against 'rawhide' during the Fedora 28 development cycle.
Changing version to '28'.

glibc-2.27-15.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-916dfe0d86

glibc-2.27-15.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-916dfe0d86

glibc-2.27-15.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.