Bug 949525

> 4. cat /proc/1/environ

Just to be clear not just process 1 but many system daemon
are running in LANG=C including sshd (gdm, etc).

*** Bug 928643 has been marked as a duplicate of this bug. ***

IMHO this ought to be Alpha blocker but I dunno if it fits the criteria...

Sent patch to systemd-devel:

http://lists.freedesktop.org/archives/systemd-devel/2013-April/010235.html

upstream commit
http://cgit.freedesktop.org/systemd/systemd/commit/?id=b3680f49e20c51e31c8dea84a11207df7b8f0100

Discussed at 2013-04-08 freeze exception review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-04-08/f19alpha-blocker-review-5.2013-04-08-16.01.log.txt . Rejected as a freeze exception as it's a pretty sensitive component to poke this late, the bug isn't fatal, and could be fixed with a post-release update.

initial-setup-0.3.4-3.fc19, systemd-201-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/FEDORA-2013-5141/systemd-201-1.fc19,initial-setup-0.3.4-3.fc19

I am afraid systemd-201 does not seem to help much.

With systemd-200, for me running "dracut -f ; reboot" was enough to get
system processes running in the locale in /etc/locale.conf,
but with systemd-201 I now need to run "localectl set-locale LANG=...;
dracut -f ; reboot".  Other than that it don't see any adverse affects
but it doesn't fix the problem that systemd seems to rely only
on initramfs to determine the locale it and its processes use,
which seems wrong.  /etc/locale.conf should override initramfs IMHO.

This helps only partly for me.

I tested Fedora-19-Alpha-x86_64-DVD.iso:

- default installation in German (de_DE.UTF-8 locale) using qemu-kvm
  qemu command line:
  
  ionice -c 3 qemu-kvm -enable-kvm -m 2048M -smp 4 -drive file=./Fedora-19-Alpha-RC1-x86_64-DVD.iso.qcow2,index=0,media=disk,cache=unsafe -localtime -serial file:/tmp/qemu-Fedora-19-Alpha-RC1-x86_64-DVD.iso.qcow2-output.log -name Fedora-19-Alpha-RC1-x86_64-DVD.iso.qcow2 -cdrom /local/mfabian/iso/Fedora-19-Alpha-RC1/Fedora-19-Alpha-x86_64-DVD.iso -boot c -spice port=6000,disable-ticketing -vga qxl -display vnc=:4 -net nic -net user,hostname=Fedora-19-Alpha-RC1-x86_64-DVD.iso.qcow2,hostfwd=tcp::5560-:22 -monitor stdio -usb
  
- at first boot, I see some initial-setup dialog for the
  timezone, root password and user creation

  - I continue from initial setup and gnome-initial-setup starts

- I ssh into the qemu and check the environment of the gnome-shell process,
  it still has LANG=C

- I switch to the vconsole with
    chvt 2
  the console is still not in Unicode mode (“SchrA¶dingerâÇÖs Cat” is displayed)

- Now  I copy

        libgudev1-201-1.fc19.x86_64.rpm
        systemd-201-1.fc19.x86_64.rpm
        systemd-libs-201-1.fc19.x86_64.rpm
        systemd-python-201-1.fc19.x86_64.rpm
        systemd-sysv-201-1.fc19.x86_64.rpm

  from https://admin.fedoraproject.org/updates/FEDORA-2013-5141/systemd-201-1.fc19,initial-setup-0.3.4-3.fc19
  to qemu and install it.

- reboot

- proceed until gnome-initial-setup

- gnome-shell still runs  in LANG=C

- vconsole still broken

- make snapshot  in qemu “before-dracut” here

- run “dracut -f”

- reboot

- proceed until gnome-initial-setup

- “gnome-shell --mode=initial-setup” is now running in the correct locale (LANG=de_DE.UTF-8)

- vconsole is now in Unicode mode, it displays Schrôdingerūs Cat
  apparently only the font is wrong.

- try “setfont latarcyrheb-sun16”, it fixes the font problem

- add “FONT="latarcyrheb-sun16"” to /etc/vconsole.conf

- reboot again

- proceed until gnome-initial setup, switch to vconsole and check

- the font is still wrong (Schrôdingerūs Cat)


So even  with the new systemd-201, FONT in /etc/vconsole is apparently
still ignored.

And, although it fixes the locale of gnome-shell  of , it seems
wrong that “dracut -f” is needed, why should one regenerate initramfs just
to change the system locale?

Actually *changing* the system locale is possible without “dracut -f”
I edited /etc/locale.conf now from LANG="de_DE.UTF-8" to
LANG="ja_JP.UTF-8" and rebooted *without* running “dracut -f” again.
“gnome-shell --mode=initial-setup” is now running in ja_JP.UTF-8
locale.

Now I go back to the snapshot “before-dracut”

- edit /etc/locale.conf now from LANG="de_DE.UTF-8" to LANG="ja_JP.UTF-8"
  and reboot without calling “dracut -f”

- It does not fix the locale, “gnome-shell --mode=initial-setup” is
  still running in LANG=C.

But after the locale of gnome-shell is fixed *once* by running “dracut -f”,
one can edit /etc/locale.conf and reboot and “gnome-shell --mode=initial-setup”
is running  in the changed locale then, one does not have to run
“dracut -f” again.


Summary:
========

With systemd-201 and “dracut -f”, the locale problem seems fixed
but the FONT in /etc/vconsole.conf is still ignored.

Package initial-setup-0.3.4-3.fc19, systemd-200-4.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing initial-setup-0.3.4-3.fc19 systemd-200-4.fc19'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5141/systemd-200-4.fc19,initial-setup-0.3.4-3.fc19
then log in and leave karma (feedback).

(In reply to comment #9)
> This helps only partly for me.
> 
> I tested Fedora-19-Alpha-x86_64-DVD.iso:
> 

can you please show us what you put in /etc/vconsole.conf and /etc/locale.conf ?

Correct entries look like this:

$ cat /etc/vconsole.conf 
KEYMAP=de-latin1
FONT=latarcyrheb-sun16

$ cat /etc/locale.conf
LANG=de_DE.UTF-8

Note the absence of ""

Are you claiming that double-quotes are not parsed/allowed?  I think they are.

(FWIW anaconda has output KEYMAP="us", LANG="...", etc forever but anyway.)

I removed the double quotes now:

[mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/vconsole.conf 
KEYMAP=us
FONT=latarcyrheb-sun16

[mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/locale.conf 
LANG="de_DE.UTF-8"
[mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$

And then I repeated the procedure in comment#9.

No change.

- With updating to systemd-201, calling “dracut -f”, and reboot,
  gdm runs in the correct locale.

- The console is correctly in Unicode mode.

- After having done that, editing /etc/locale.conf and rebooting
  correctly changes the system locale.

- But the font on the console stays wrong (“Schrôdingerūs Cat” is displayed).
  
- “sudo setfont latarcyrheb-sun16” fixes the font
  and “Schrödinger’s Cat” is displayed correctly.

Thanks for further testing.

I note for the record that in F18 pid 1 also runs with LANG=C
and this is not a problem in itself - the problem with F19 is that
systemd then creates child processes also with LANG=C unlike F18.

Anyway from the systemd point of view what is the expected F19 behaviour?
Is the F18 behaviour correct or is it all anaconda's fault for
not setting up /etc/locale.conf and /etc/vconsole.conf before
dracut is run? :)

(In reply to comment #13)
> I removed the double quotes now:
> [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/locale.conf 
> LANG="de_DE.UTF-8"

I still see them

(In reply to comment #13)
> - But the font on the console stays wrong (“Schrôdingerūs Cat” is displayed).
>   
> - “sudo setfont latarcyrheb-sun16” fixes the font
>   and “Schrödinger’s Cat” is displayed correctly.

This is most likely a kernel bug with the char mapping table not getting copied over, when the kms driver is loaded.

(In reply to comment #14)
> Thanks for further testing.
> 
> I note for the record that in F18 pid 1 also runs with LANG=C
> and this is not a problem in itself - the problem with F19 is that
> systemd then creates child processes also with LANG=C unlike F18.
> 
> Anyway from the systemd point of view what is the expected F19 behaviour?
> Is the F18 behaviour correct or is it all anaconda's fault for
> not setting up /etc/locale.conf and /etc/vconsole.conf before
> dracut is run? :)

Jens: I really debugged systemd and I am most confident, that 

http://cgit.freedesktop.org/systemd/systemd/commit/?id=b3680f49e20c51e31c8dea84a11207df7b8f0100

which is part of systemd-201 fixes the issues with LANG=C.

Of course you have to have systemd-201 in the initramfs!!!!!

# journalctl  -ab _PID=1 | grep -F 'running in system mode'
Apr 12 08:41:38 lenovo systemd[1]: systemd 201 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ)
Apr 12 09:39:11 lenovo systemd[1]: systemd 201 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ)

(In reply to comment #17)
> # journalctl  -ab _PID=1 | grep -F 'running in system mode'
> Apr 12 08:41:38 lenovo systemd[1]: systemd 201 running in system mode. (+PAM
> +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ)
> Apr 12 09:39:11 lenovo systemd[1]: systemd 201 running in system mode. (+PAM
> +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ)


Sorry, that check should be:
$ journalctl  -ab | grep -F 'running in system mode'
Apr 12 08:41:37 lenovo systemd[1]: systemd 201 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ)
Apr 12 08:41:38 lenovo systemd[1]: systemd 201 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ)

(In reply to comment #15)
> (In reply to comment #13)
> > I removed the double quotes now:
> > [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/locale.conf 
> > LANG="de_DE.UTF-8"
> 
> I still see them

I removed them in vconsole.conf:

[mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/vconsole.conf 
KEYMAP=us
FONT=latarcyrheb-sun16

but not in locale.conf:

[mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/locale.conf 
LANG="de_DE.UTF-8"
[mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$

because the remaining problem for me was a problem with the font,
not with the locale.

Also, the double quotes in locale.conf seem to work OK because
changing the locale there and rebooting works.

(In reply to comment #19)
> (In reply to comment #15)
> > (In reply to comment #13)
> > > I removed the double quotes now:
> > > [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/locale.conf 
> > > LANG="de_DE.UTF-8"
> > 
> > I still see them
> 
> I removed them in vconsole.conf:
> 
> [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/vconsole.conf 
> KEYMAP=us
> FONT=latarcyrheb-sun16
> 
> but not in locale.conf:
> 
> [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/locale.conf 
> LANG="de_DE.UTF-8"
> [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$
> 
> because the remaining problem for me was a problem with the font,
> not with the locale.
> 
> Also, the double quotes in locale.conf seem to work OK because
> changing the locale there and rebooting works.

Well, systemd-vconsole-setup determines the utf8 mode from the locale.
So, if the locale has no "@UTF-8", then unicode mode is not turned on.

It seems we are seeing a combination of bugs, because if LANG=C was serialized from a systemd < 201 in the initramfs, systemd-vconsole-setup does not turn on utf8 mode.

So, please make sure, that you have a systemd-201 in the initramfs with:
# dracut -f

(In reply to comment #20)
> (In reply to comment #19)
> > (In reply to comment #15)
> > > (In reply to comment #13)
> > > > I removed the double quotes now:
> > > > [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/locale.conf 
> > > > LANG="de_DE.UTF-8"
> > > 
> > > I still see them
> > 
> > I removed them in vconsole.conf:
> > 
> > [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/vconsole.conf 
> > KEYMAP=us
> > FONT=latarcyrheb-sun16
> > 
> > but not in locale.conf:
> > 
> > [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$ cat /etc/locale.conf 
> > LANG="de_DE.UTF-8"
> > [mfabian@Fedora-19-Alpha-RC2-x86_64-DVD ~]$
> > 
> > because the remaining problem for me was a problem with the font,
> > not with the locale.
> > 
> > Also, the double quotes in locale.conf seem to work OK because
> > changing the locale there and rebooting works.
> 
> Well, systemd-vconsole-setup determines the utf8 mode from the locale.
> So, if the locale has no "@UTF-8", then unicode mode is not turned on.

".UTF-8" that is.

And also, if /etc/vconsole.conf and /etc/locale.conf are missing in the initramfs and no "vconsole.* and locale.LANG=*" is set on the kernel cmdline,
systemd-vconsole-setup sets up the wrong mode.

Then plymouth starts and probably keeps the console busy, so that the systemd-vconsole-setup from the real root, which has /etc/vconsole.conf and /etc/locale.conf, cannot set the console (-EBUSY because of plymouth).

So, to disable plymouth add "plymouth.enable=0" to the kernel cmdline.

(In reply to comment #22)
> And also, if /etc/vconsole.conf and /etc/locale.conf are missing in the
> initramfs and no "vconsole.* and locale.LANG=*" is set on the kernel cmdline,
> systemd-vconsole-setup sets up the wrong mode.

systemd-vconsole-setup in the initramfs that is.. running before plymouth.

> Then plymouth starts and probably keeps the console busy, so that the
> systemd-vconsole-setup from the real root, which has /etc/vconsole.conf and
> /etc/locale.conf, cannot set the console (-EBUSY because of plymouth).
> 
> So, to disable plymouth add "plymouth.enable=0" to the kernel cmdline.

systemd-201-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/FEDORA-2013-5118/systemd-201-1.fc19

(In reply to comment #17)
> Jens: I really debugged systemd and I am most confident, that 
> http://cgit.freedesktop.org/systemd/systemd/commit/
> ?id=b3680f49e20c51e31c8dea84a11207df7b8f0100
> which is part of systemd-201 fixes the issues with LANG=C.

Great - that is also why I left the bug in Modified :)
since we could not test installation with 201 yet.
Today I requested https://admin.fedoraproject.org/updates/FEDORA-2013-5118/systemd-201-1.fc19 be pushed (again) to updates-testing
so we should be able finally to do that soon.
The comment above were just based on our preliminary post-install
testing which suggested there could still be some edge cases,
but I think you are right that once 201 is in the initramfs
then the bug is indeed fixed, thanks.

I added this bug to the bodhi update replacing the old bug already fixed in 200-4.fc19.

Package systemd-201-1.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-201-1.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5118/systemd-201-1.fc19
then log in and leave karma (feedback).

jens: I appreciate you're trying to fix this bug, but there was a reason we kept squelching the 201 update: 200-4 fixed a blocker bug and so we needed to pull it into stable, but during freezes, we want to take the *minimum level of change necessary*. We don't want to pull a whole new systemd release just to fix the blocker bug which was already fixed with far less impact by 200-4.

So when your package has an open update that is a blocker fix and is part of the whole TC/RC build / freeze exception process, please try and avoid submitting a later build with more changes as an update until the 'blocker-fixing' update is pushed. thanks!

I just did a Japanese minimal network install with latest f19 including
systemd-201-1.fc19 in updates-testing (using F19 Alpha TC6 netinst.iso)
and can confirm that this bug is indeed fixed as Harald assured us.
Thanks! :)

(Note when verifying this that systemd (pid 1) is still running in LANG=C
but that is fine - system processes are now all running correctly
in the system locale given by localectl output and /etc/locale.conf.)

We in the i18n team are still quite new to dracut/initramfs
so the distinction with the version of systemd in the initramfs
image was not clear to us in our initial testing of 201 - thanks for
the explanation.

(In reply to comment #27)
> jens: I appreciate you're trying to fix this bug, but there was a reason we
> kept squelching the 201 update

Thanks Adam - I understand.

I still wish we could backport Harald's fix to systemd-200
and include it in Alpha - shipping Alpha with this bug included
is quite bad from the i18n point of view but I was hesitating
to push for that until we could confirm the fix in install testing.

Harald: is it possible to backport your patch to systemd-200?

If so I would like to re-propose this bug for AlphaFreezeException if possible.

This bug can't be fixed post install just by updating systemd
it requires regenerating the initramfs.

Happily, initramfs gets re-generated frequently with kernel updates, so it shouldn't be a huge deal. I'd be okay taking this as FE myself, if we could do it as a patch for systemd-200, but it's probably a bit late now - I'm about to do the RC3 request, which with any luck should be the last build.

systemd-201-1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

A netinstall of Fedora 19 Alpha contains systemd-202-3.fc19.x86_64 now
and the LANG=C problem is fixed indeed.

The console is in Unicode mode but the font is still wrong,
“’” in  “Schrödinger’s Cat” still displays as a box.

Adding the font “latarcyrheb-sun16” to /etc/vconsole.font like this

[mfabian@Fedora-19-Alpha-x86_64-netinst-r ~]$ cat /etc/vconsole.conf 
KEYMAP="us"
FONT="latarcyrheb-sun16"
[mfabian@Fedora-19-Alpha-x86_64-netinst-r ~]$

and rebooting does not fix the problem, the ’ is still displayed as a
box.

Manually calling  “setfont latarcyrheb-sun16” fixes the problem though.

(In reply to comment #33)
> Adding the font “latarcyrheb-sun16” to /etc/vconsole.font like this
> 
> [mfabian@Fedora-19-Alpha-x86_64-netinst-r ~]$ cat /etc/vconsole.conf 
> KEYMAP="us"
> FONT="latarcyrheb-sun16"
> [mfabian@Fedora-19-Alpha-x86_64-netinst-r ~]$
> 
> and rebooting does not fix the problem, the ’ is still displayed as a
> box.
> 
> Manually calling  “setfont latarcyrheb-sun16” fixes the problem though.

did you recreate the initramfs after installing vconsole.conf?

After editing /etc/vconsole.conf as in comment#34 and calling

    dracut -f

and rebooting, the problem is not fixed, the ’ is still displayed as a
box.

Manually calling  “setfont latarcyrheb-sun16” fixes the problem though.