Bug 623931

Summary: can't enable gtk-im-context-simple for unicode Ctrl-Shift-u input
Product: [Fedora] Fedora Reporter: Jens Petersen <petersen>
Component: imsettingsAssignee: Jens Petersen <petersen>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 13CC: eng-i18n-bugs, i18n-bugs, john, kevin, llim, mclasen, myllynen, overholt, petersen, rstrode, tagoh
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: imsettings-0.108.1-2.fc13 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 616061 Environment:
Last Closed: 2010-12-09 01:32:26 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On: 505100, 616061    
Bug Blocks:    

Description Jens Petersen 2010-08-13 03:34:29 EDT
Currently XIM is enabled by default in Fedora when
input-methods are disabled to allow consistency between
GTK, Qt and X apps.  However this breaks C-S-u input of Unicode
in GTK apps, and actually the only easy way to disable XIM
currently is to uninstall gtk2-immodule-xim (hardly very friendly).

To alleviate this situation I am proposing changing the none.conf state
of imsettings (the default for locales that don't use input-methods)
not to use XIM by default, bringing back gtk-im-context-simple as
the default immodule for "Western" locales.  Based on feedback from
bug 505100 at very least users of two locales need to use
X locale compose rules for input: pt_BR users using us_intl keyboards
and fi_FI users, so they will default to the X locale compose setting
by default.  (Going by im-cedilla locales maybe some more locales
may need to be added.)

The changes are in imsettings-0.108.1-2.fc{14,15}.
Testing and feedback particularly welcome for f14.


XIM should probably also be disabled for Qt by default
to be consistent (though qt actually defaults to XIM upstream
unlike GTK).


+++ This bug was initially created as a clone of Bug #616061 +++

Description of problem:
Can't enter unicode characters with Ctrl-Shift-u

How reproducible:
Always

Steps to Reproduce:
1. Try to enter a unicode character with Ctrl-Shift-u <unicode code>
  
Actual results:
No unicode character entered

Expected results:
Unicode character entered

Additional info:
halfline says that the problem stems from gtk-im-module being set to 'xim':

$ gconftool-2 -R /desktop/gnome/interface | grep gtk-im
 gtk-im-status-style = callback
 gtk-im-preedit-style = callback
 gtk-im-module = xim

--- Additional comment from tfujiwar@redhat.com on 2010-07-20 14:01:42 EST ---

GTK xim doesn't implement Ctrl-Shift-u.

There are three ways.
1. Set GTK_IM_MODULE=ibus and no any ibus engines.
   Run im-chooser and enable IBus. (you need ibus, ibus-libs and ibus-gtk only)

2. Uninstall all GTK IM modules (ibus-gtk and gtk2-immodule-xim) then you will use GTK_IM_MODULE=gtk-im-context-simple
   You could check /etc/X11/xinit/xinput.d/none.conf

3. Add your $HOME/.xinputrc
% cat $HOME/.xinputrc
XIM=none
XIM_PROGRAM=
XIM_ARGS=
GTK_IM_MODULE=gtk-im-context-simple
QT_IM_MODULE=xim
IMSETTINGS_IGNORE_ME=yes
DISABLE_IMSETTINGS=yes

Probably I'll close this bug.

--- Additional comment from overholt@redhat.com on 2010-07-20 22:29:34 EST ---

Thanks for the explanation.  I am concerned, though.  Ctrl-Shift-u is incredibly useful and well-documented online.  Is the default in RHEL 6 really going to be xim which doesn't implement Ctrl-Shift-u?

--- Additional comment from petersen@redhat.com on 2010-07-21 12:36:01 EST ---

The current workaround I can see are removing gtk2-immodule-xim
or enabling ibus.

I don't see a good solution though: gtk2-immodule-xim is
needed for Brazilian and European users to be able to use
full X locale compose input.

From UX POV using unicode input is wrong - it means our
desktop input system is incomplete.  And that should be fixed
not unicode input bandaids IMHO.

--- Additional comment from petersen@redhat.com on 2010-07-21 13:56:52 EST ---

(In reply to comment #4)
> Ctrl-Shift-u is incredibly useful and well-documented online.

It is a toolkit specific hack really - I think it only works for GTK, right?

--- Additional comment from petersen@redhat.com on 2010-07-21 18:18:14 EST ---

I believe current behaviour is at least consistent across X, GTK, and Qt.

--- Additional comment from tagoh@redhat.com on 2010-07-21 18:54:26 EST ---

Sure. so what do you suggest to get this fixed in imsettings?

--- Additional comment from rstrode@redhat.com on 2010-08-03 05:16:01 EST ---

(In reply to comment #6)
> The current workaround I can see are removing gtk2-immodule-xim
> or enabling ibus.
Removing gtk2-immodule-xim might make sense.  I think xim is pretty much legacy at this point right?  We'll have to make sure whatever is setting the gconf key stops setting it to xim as well though.

> I don't see a good solution though: gtk2-immodule-xim is
> needed for Brazilian and European users to be able to use
> full X locale compose input.
I don't think that's true.

> From UX POV using unicode input is wrong - it means our
> desktop input system is incomplete.  And that should be fixed
> not unicode input bandaids IMHO.    
Not always.  People use the feature for putting a pirate on irc or whatever.  I suppose we could add 

compose+yarrr = ☠

but ctrl-shift-u is useful and it should probably work out of the box...

--- Additional comment from otaylor@redhat.com on 2010-08-03 05:18:51 EST ---

(In reply to comment #6)
> The current workaround I can see are removing gtk2-immodule-xim
> or enabling ibus.
> 
> I don't see a good solution though: gtk2-immodule-xim is
> needed for Brazilian and European users to be able to use
> full X locale compose input.

This is not fully true. The situation is that the default compose sequences, in, say, a en_US environment, do the obvious thing for certain compose sequences, in particular:

 dead_acute + c => c with an acute accent

Brazilian users tend to want something else:

 dead_acute + c => c_cedilla

[ Brazilian users are the heaviest users of compose sequences in the world, because of the predominance of the us-intl keyboard among technical users there. ]

If the locale (in particular, LC_CTYPE) is set to any language that uses a Cedilla (pt, fr, tr, etc), then GTK+ defaults to a input method module that acts appropriately (called im-cedilla, as I recall)

Now, unfortunately, we don't have any way of setting the locale categories independently through the UI, so if you want English messages with compose sequences (or currency, date formatting, etc) from a locale with a differnet language, there's no easy way to do i:

Using XIM via gtk-im-module=xim:

 * Causes a large performance and memory hit at startup - the XIM compose sequence mechanism was designed for a few dozen compose sequences, not the hundreds or thousands that are in it currently.

 * Breaks various features built into the default GTK+ input module as described above

If we want Cedillas, then we could specify:

 gtk-im-module=im-cedilla 

And get that behavior without the other downsides of XIM. I don't see any point in taking away control-shift-u for our default set of applications on the default desktop just make it more consistent with other toolkits.

--- Additional comment from petersen@redhat.com on 2010-08-05 13:17:29 EST ---

I refer to the long discussion in bug 505100 where this came up in Fedora.

(In reply to comment #11)
> I think xim is pretty much legacy at this point right?

I am not suggesting to use XIM for input-methods, just leveraged
the XIM code path for X locale compose.  This gives consistent
user input experience across gtk, qt and X applications.

> but ctrl-shift-u is useful and it should probably work out of the box...    

I didn't say it is not useful - I said it is wrong UI. :-)

(In reply to comment #12)
> If the locale (in particular, LC_CTYPE) is set to any language that uses a
> Cedilla (pt, fr, tr, etc), then GTK+ defaults to a input method module that
> acts appropriately (called im-cedilla, as I recall)

Right, but that only works in gtk apps.

> Using XIM via gtk-im-module=xim:
> 
>  * Causes a large performance and memory hit at startup - the XIM compose
> sequence mechanism was designed for a few dozen compose sequences, not the
> hundreds or thousands that are in it currently.

Hmm - agree that is bad.

>  * Breaks various features built into the default GTK+ input module as
> described above

Ok, but I am arguing that such toolkit specific features are
not really a good thing.

> If we want Cedillas, then we could specify:
> 
>  gtk-im-module=im-cedilla 

That only works for pt_BR users.

> And get that behavior without the other downsides of XIM. I don't see any point
> in taking away control-shift-u for our default set of applications on the
> default desktop just make it more consistent with other toolkits.    

I dunno at least it is consistent.

Perhaps a compromise would be to only enable XIM for locales where
X locale compose is needed, leaving the rest of us in peace? :)
Comment 1 Fedora Update System 2010-08-13 03:40:24 EDT
imsettings-0.108.1-2.fc14 has been submitted as an update for Fedora 14.
http://admin.fedoraproject.org/updates/imsettings-0.108.1-2.fc14
Comment 2 Jens Petersen 2010-08-17 05:11:30 EDT
$ git clone git://anongit.freedesktop.org/xorg/lib/libX11
$ cd libX11/nls
$ $ for i in *; do if [ -d $i -a -s $i/Compose.pre ]; then if ! grep -q "This file currently has no entries." $i/Compose.pre ; then  echo $i; fi; fi; done | grep UTF
am_ET.UTF-8
el_GR.UTF-8
en_US.UTF-8
fi_FI.UTF-8
pt_BR.UTF-8
ru_RU.UTF-8
$ 

Those seem to be the only UTF-8 locales with actual compose sequences.

pt_BR and fi_FI are already covered.  I am pretty user
regular en_US users don't need it by default.  am_ET was for
OLPC and can be handled otherwise I think.  That leaves just
ru_RU and el_GR, which can be added later if necessary.
So in conclusion special-casing X compose seems to make
good sense for now.

Though Marko Myllynen told me about http://www.csc.fi/english/pages/meek
which might lead to more compose usage in the EU in the future perhaps.
Comment 3 Fedora Update System 2010-08-19 00:30:50 EDT
imsettings-0.108.1-2.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/imsettings-0.108.1-2.fc13
Comment 4 Fedora Update System 2010-08-19 22:15:57 EDT
imsettings-0.108.1-2.fc13 has been pushed to the Fedora 13 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update imsettings'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/imsettings-0.108.1-2.fc13
Comment 5 Jens Petersen 2010-08-22 22:03:52 EDT
I added am_ET, el_GR, and ru_RU also to imsettings/master.
Comment 6 Fedora Update System 2010-08-23 21:58:30 EDT
imsettings-0.108.1-2.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 7 Fedora Update System 2010-08-24 17:14:41 EDT
imsettings-0.108.1-2.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 8 Kevin Kofler 2010-08-25 22:09:05 EDT
IMHO this "bugfix" is a regression, you've now made GTK+ use its own nonstandard and incomplete (no support for compose sequences!) input handling by default instead of using the shared input platform everything in Fedora uses. And no, we have no plans whatsoever to change Qt not to use XIM, it would have no advantage anyway, that Ctrl+Shift+u combo is a feature specific to GTK+'s input implementation, I don't think Qt implements that.
Comment 9 Kevin Kofler 2010-08-25 22:13:15 EDT
By the way:
* How do I set GTK+ back to using XIM after this "fix"?
* Is there a way we can default GTK+ to XIM input for KDE users, without breaking other things such as IBus support? I really don't see why a KDE user would want GTK+ apps to behave differently from the rest of his KDE desktop.
Comment 10 Kevin Kofler 2010-08-25 22:18:29 EDT
PS: In the past, when the default was "simple", I always thought XChat was broken for not supporting Compose. Then I noticed it's all GTK+ apps. And then I found out that it was a GTK+ "feature", a.k.a. bug 505100. I was happy that this was finally fixed. I really don't see why we now reverted that bugfix just to support a GTK+-only hack which is implemented completely at the wrong point of the input stack. (Something like Ctrl+Shift+u needs to be implemented in an input method framework, so all apps benefit from it, not at toolkit level.)
Comment 11 Kevin Kofler 2010-08-25 22:24:28 EDT
PPS: Actually #505100 is not the correct bug reference. But GTK+ not supporting Compose is a bug, which you just reintroduced.

(Sorry for the quadruple post, always more stuff to add.)
Comment 12 Matthias Clasen 2010-08-26 15:23:11 EDT
> own nonstandard and incomplete (no support for compose sequences!)

not sure what you are talking about. GTK's builtin input method supports compose sequences alright.
Comment 13 Ray Strode [halfline] 2010-08-26 15:34:59 EDT
GTK supports the compose key fine.  Sometimes which sequences it supports lags a bit, but overall it works.  If i type 

<compose> + - + > i get → 
<compose> + < + 3 i get ♥
<compose> + ' + a i get á
etc

You can force gtk apps to use a different im module by setting the "Gtk/IMModule" X setting in your settings daemon app.  Although, there's no guarantee the user will have gtk2-immodule-xim installed.
Comment 14 Marko Myllynen 2010-08-26 16:13:16 EDT
> GTK's builtin input method supports compose sequences alright.

Please note that GTK's builtins conflicts with some compose maps like fi_FI (which is actually needed to comply with a national standard).

One example from https://bugzilla.redhat.com/show_bug.cgi?id=505100#c71:

dead_acute + space produces ' if using en_US.UTF-8/Compose (or GTK builtins)
dead_acute + space produces ´ if using fi_FI.UTF-8/Compose

The above comment provides some additional details. Thanks.
Comment 15 Jens Petersen 2010-08-26 20:54:37 EDT
Well it is clear that GTK compose rules are different from
the more standard compose sequences supported by libX11.

The main improvement of this update is that it is now at least
possible to turn off XIM for GTK which was not possible before.

Perhaps we should just default to XIM compose for everyone except en_*
then?
Comment 16 Jens Petersen 2010-08-26 21:04:50 EDT
(In reply to comment #9)
> * How do I set GTK+ back to using XIM after this "fix"?

In im-chooser turn on IM and select X compose.

> * Is there a way we can default GTK+ to XIM input for KDE users, without
> breaking other things such as IBus support? I really don't see why a KDE user
> would want GTK+ apps to behave differently from the rest of his KDE desktop.

I think we should try to agree to do the same across all desktops
otherwise it impacts QA testing complexity and UX consistency.

How about my suggestion in the previous comment?
Comment 17 Jens Petersen 2010-08-26 21:08:08 EDT
(In reply to comment #10)
> PS: In the past, when the default was "simple", I always thought XChat was
> broken for not supporting Compose. Then I noticed it's all GTK+ apps. And then
> I found out that it was a GTK+ "feature", a.k.a. bug 505100. I was happy that
> this was finally fixed. I really don't see why we now reverted that bugfix just
> to support a GTK+-only hack which is implemented completely at the wrong point
> of the input stack. (Something like Ctrl+Shift+u needs to be implemented in an
> input method framework, so all apps benefit from it, not at toolkit level.)

Thanks for speaking up anyway - glad some people liked the default gtk XIM in F13.

Kevin: BTW what locale do you normally use?
Comment 18 Kevin Kofler 2010-08-26 23:31:25 EDT
> In im-chooser turn on IM and select X compose.

I have no im-chooser running on my KDE system in the first place! It's a GNOME tool which does not autostart in KDE.

I ended up hand-editing /etc/X11/xinit/xinput.d/none.conf (which is probably a hack, is that file even marked %config?) for now.

> Perhaps we should just default to XIM compose for everyone except en_*
> then?

IMHO it should just be everyone, including en_*.

> Kevin: BTW what locale do you normally use?

de_AT.UTF-8 with de keyboard layout and compose on Shift+AltGr enabled in the xkb settings. (I miss the time that that was the default, too. It got changed because some newbies can't deal with the fact that Shift+AltGr is not the same thing as AltGr+Shift, i.e. ordering matters. See the discussion in bug 193922. I'm still unhappy about that change, too.)
Comment 19 Kevin Kofler 2010-08-26 23:32:35 EDT
(But at least there's a way to set the xkb settings from KDE's System Settings. There's no such thing for imsettings settings.)
Comment 20 Jens Petersen 2010-08-26 23:43:44 EDT
(In reply to comment #18)
> > In im-chooser turn on IM and select X compose.
> 
> I have no im-chooser running on my KDE system in the first place! It's a GNOME
> tool which does not autostart in KDE.

No it isn't - it is a desktop tool installed by default in Fedora.

Anyway this bug is about GTK apps.

> I ended up hand-editing /etc/X11/xinit/xinput.d/none.conf (which is probably a
> hack, is that file even marked %config?) for now.

It is not really a user-configuration file.

You should override with "~/.xinputrc".
Comment 21 Matthias Clasen 2010-08-27 18:40:53 EDT
(In reply to comment #15)

> Perhaps we should just default to XIM compose for everyone except en_*
> then?

No.
Comment 22 Matthias Clasen 2010-08-27 18:42:13 EDT
(In reply to comment #18)
> > In im-chooser turn on IM and select X compose.
> 
> I have no im-chooser running on my KDE system in the first place! It's a GNOME
> tool which does not autostart in KDE.
> 

The im-chooser is not a GNOME tool.
Comment 23 Jens Petersen 2010-08-28 00:58:57 EDT
I don't see any consensus emerging so I plan to take
this discussion to the mailing lists.
Comment 24 Jens Petersen 2010-08-28 06:00:06 EDT
Just for the record, ibus also has some compose support
(might not be working currently in 1.3.7 though).
Comment 25 Jens Petersen 2010-08-29 23:24:35 EDT
I am surprised more country keyboard layouts don't provide
a Compose key by default.  AFAICT only standard gb, ie, and fi layouts
do that.  (Many countries use ISO_Level_3_Shift though instead.)
Comment 26 Kevin Kofler 2010-08-30 06:19:17 EDT
It's not really an "instead". If you enable Shift+AltGr compose on layouts which use ISO_Level_3_Shift, you'll have Shift+AltGr (in this order) = Compose and AltGr+Shift (in that order) = ISO_Level_3_Shift.
Comment 27 Jens Petersen 2010-08-30 22:57:19 EDT
I am going to look at the various compose rules provided by
X, gtk, and ibus to get a better idea how they compare.
Comment 28 Jens Petersen 2010-08-30 22:59:37 EDT
(Compose should be working again in ibus-1.3.7-3.fc14 btw.)
Comment 29 Jens Petersen 2010-09-05 21:21:01 EDT
Kevin, do you have some examples of where gtkimcontextsimple compose
behaves differently from how you expect it?  That would be useful data.

According to gtk/gtkimcontextsimpleseqs.h gtk it ought to support
most compose sequences in libX11 and was last updated 2008.

http://git.gnome.org/browse/gtk+/tree/gtk/gtkimcontextsimpleseqs.h
Comment 30 John Drinkwater 2010-09-09 05:44:58 EDT
(In reply to comment #10)
> of the input stack. (Something like Ctrl+Shift+u needs to be implemented in an
> input method framework, so all apps benefit from it, not at toolkit level.)
(In reply to comment #25)
> I am surprised more country keyboard layouts don't provide
> a Compose key by default.  AFAICT only standard gb, ie, and fi layouts
>.do that.  (Many countries use ISO_Level_3_Shift though instead.)

So where’s the place to discuss such things? I’m a hefty user of C+S+u even with a compose-aware keymap, and would dislike it not returning.
Comment 31 Jens Petersen 2010-09-10 03:06:25 EDT
(In reply to comment #30)
> (In reply to comment #10)
> > (Something like Ctrl+Shift+u needs to be implemented in an
> > input method framework, so all apps benefit from it, not at toolkit level.)

ibus-gtk already does.  Perhaps ibus-qt could do it too.
I still think that compose and input methods are
more natural input systems than direct Unicode though.

> So where’s the place to discuss such things? I’m a hefty user of C+S+u even
> with a compose-aware keymap, and would dislike it not returning.

It is already back as of comment 6.  What are you missing? :)
Comment 32 Jens Petersen 2010-09-15 04:11:11 EDT
Anyone have any more input on Gtk vs X compose rules?

I am planning to close the bug after a week.  It would probably
be better to open a new bug anyway for any further required changes.