Bug 875965 - RFE: [WebTrans] enable visible white space in source
RFE: [WebTrans] enable visible white space in source
Status: CLOSED CURRENTRELEASE
Product: Zanata
Classification: Community
Component: Usability (Show other bugs)
development
Unspecified Unspecified
high Severity high
: ---
: 3.6
Assigned To: Luke Brooker
Damian Jansen
: Triaged
Depends On:
Blocks: Zanata-3.6.1
  Show dependency treegraph
 
Reported: 2012-11-12 18:26 EST by Patrick Huang
Modified: 2016-04-18 05:37 EDT (History)
7 users (show)

See Also:
Fixed In Version: 3.6.1-SNAPSHOT (git-jenkins-zanata-server-github-pull-requests-3111)
Doc Type: Bug Fix
Doc Text:
Story Points: 1
Clone Of:
: 880444 (view as bug list)
Environment:
Last Closed: 2015-04-19 21:06:33 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Patrick Huang 2012-11-12 18:26:13 EST
Description of problem:
With the newly released Zanata version 2.0 Firefox's spell check does not seem to work anymore. I experienced this on FF 3.6, Leticia confirmed this for her FF 10.0.6. The spell-check used to work on both versions with Zanata 1.7.x

Version-Release number of selected component (if applicable):
2.0

Additional info:
Original bug is 872384. We introduced an option to switch between Code mirror editor and plain textarea editor as short term solution.
This bug is for permanently enable spell check in code mirror editor.
Comment 1 Patrick Huang 2012-11-12 18:28:50 EST
Comment copied from bug 872384:
Patrick Huang 2012-11-08 23:12:13 EST
did some research and below options seems promising:

library: http://code.google.com/p/google-api-spelling-java/ 
description: This is a simple Java API that makes it very easy to call Google's spell checker service from Java applications.
pros: simple and easy to use
cons: calling google api means external network traffic

library: https://github.com/dren-dk/HunspellJNA
description: Hunspell is the spell checker of LibreOffice, OpenOffice.org, Mozilla Firefox 3 & Thunderbird, Google Chrome.
pros: lots of dictionaries
cons: written in c++ and required a JNA binding

library: http://jazzy.sourceforge.net/
description: pure java based spellchecker and based on most of the algorithms that aspell has
pros: pure java implementation
cons: no update since 2005 and sourceforge only have English dictionary


[reply] [-] Comment 8 Sean Flanigan 2012-11-09 02:22:43 EST
Hmm, Hunspell sounds good, but deploying Zanata could become a real nightmare if we start using JNA.

I still think a toggle is the only option that makes sense for a short-term solution, but just for completeness, I did spot these quick and dirty Hunspell ports too:

Javascript: http://code.google.com/p/bjspell/

Java: http://code.google.com/p/lucene-hunspell/ (tied to Lucene for some reason)
Comment 2 Runa Bhattacharjee 2012-11-26 13:17:47 EST
Hi, can we try to squeeze this in for the upcoming release? Thanks.
Comment 3 Sean Flanigan 2012-11-26 20:17:49 EST
This (ie syntax highlighting editor with spell-checking) isn't the sort of thing we can squeeze in, if we want to implement it properly, and test it properly.  It's far too big.  

If we want to cover a decent number of languages in our own spell-checker, we pretty much have to have Hunspell, but there only seems to be one Java port of Hunspell, but it is designed for a different use case, and it is only a prototype.  From what I can tell, it's quite a mess, and doesn't even have a build file.

Even if we get Hunspell working, we'll have a fair bit of UI work to integrate a spell-checker into CodeMirror.  Especially if we want a pop-up suggestions features?


Alternatively, we could look for an alternative to CodeMirror.  According to http://en.wikipedia.org/wiki/Comparison_of_JavaScript-based_source_code_editors these support browser-based spell-checking:

http://codepress.sourceforge.net/
http://www.cdolivet.com/editarea/

But they could turn out to have show-stopping problems (eg focus/event problems) when integrated into Zanata.  Or cross-platform bugs.  And I can't get spell-checking to work in Chrome for either of them so far...
Comment 4 Patrick Huang 2012-11-26 20:45:54 EST
I've got spell check working in codemirror in firefox only. It's NOT cross platform (uses firefox only attributes) and doesn't (seems to) support pop up suggestion/add to dictionary etc. BUT it highlights the word in red and it uses firefox build-in spell check facility.

We can definitely squeeze this in :)
Comment 5 Sean Flanigan 2012-11-26 21:14:42 EST
Okay, that's good.  I suggest creating another bz for the Firefox-only implementation.  We should provide a user option to turn it off.  (Partly so that the user can turn off spell-check, and partly in case of unexpected problems.)
Comment 6 Patrick Huang 2012-11-29 20:23:00 EST
I submitted issues both to Code Mirror and Chrome. Looks like Chrome is doing something differently on elements with attribute "contenteditable=true". Bug 880444 is working in Firefox.
Comment 7 Sean Flanigan 2012-11-29 20:37:58 EST
Could you please link to the Chrome and CodeMirror bugs here?
Comment 8 Patrick Huang 2012-11-29 20:42:57 EST
CodeMirror issue raised at: https://bugzilla.redhat.com/show_bug.cgi?id=875965
Chrome issue is raised using Chrome's build in (menu->About Google Chrome->Report an Issue). I don't have any URL.
Comment 9 Patrick Huang 2012-11-29 20:44:41 EST
(In reply to comment #8)
> CodeMirror issue raised at:
> https://bugzilla.redhat.com/show_bug.cgi?id=875965
> Chrome issue is raised using Chrome's build in (menu->About Google
> Chrome->Report an Issue). I don't have any URL.
Wrong url: https://github.com/marijnh/CodeMirror/issues/1017
Comment 10 Sean Flanigan 2012-12-03 00:13:13 EST
Found a bit more information about the Hunspell situation:

According to http://wiki.apache.org/solr/Hunspell the Lucene-Hunspell code was contributed to Solr.  Lucene-Hunspell seems to be a bit dead, but it sounds as if the Solr team is maintaining a fork.

This seems to be it:

http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/analysis/common/src/java/org/apache/lucene/analysis/hunspell/

http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22lucene-analyzers-common%22


It looks like it's targeted more at spelling suggestions than spell checking per se, but it may be possible to modify it for spell checking.
Comment 11 Sean Flanigan 2012-12-03 01:27:38 EST
To modify it for spell checking will probably require re-implementing Hunspell::spell() in Java: http://hunspell.cvs.sourceforge.net/viewvc/hunspell/hunspell/src/hunspell/hunspell.cxx?view=markup
Comment 12 Sean Flanigan 2012-12-06 23:47:34 EST
Apparently the next version of ACE (a rival to CodeMirror, used by Skynet) will have some sort of spell-checking (not sure if it will have all of Hunspell's languages):

https://groups.google.com/forum/?fromgroups=#!topic/ace-discuss/-dwNC_rX7R0


I also spotted a link to a Hunspell implementation in JavaScript:

https://github.com/cfinke/Typo.js/blob/master/typo/typo.js

I wonder if it would run inside Rhino?  But I'm sure it could be ported to Java, if it's any good.

Given the size of Hunspell dictionaries, I'm not sure whether it would be better to load the dictionary in the client (one dictionary for every editor instance) or in the server (one dictionary per language).
Comment 13 Michelle Kim 2015-03-10 01:35:14 EDT
Technical Issue to be discussed. We may take a look at some alternatives.
Comment 14 Patrick Huang 2015-03-10 20:30:26 EDT
The problem with codemirror spellcheck is:
I used to hack it and enabled spellchecking. But when you correct the word using browser's built-in function, codemirror doesn't receive a change event which result in translation unsaved.
From the issue I put in codemirror, they will never support spellcheck. Integrate third party javascript library is possible. But then you have to keep track of updating the dictionary yourself instead of relying on browser's. Not a viable option either.
The best we can get for the old editor is to keep codemirror to just do syntax highlighting. And use plain text for spellcheck. There is already a shortcut ctrl+alt+h to toggle between the two.
Comment 15 Michelle Kim 2015-03-16 20:10:29 EDT
As we discussed with translators, we agreed upon the solutions as follows:

1. Enable codemirror (syntax highlighting) in source , so that translators will be able to view spacing highlight etc

2. Disable codemirror in target by default, so that spellcheck is available. If translator wants, they can use either a shortcut ctrl + alt + h to toggle, or use the editor setting option for consistent change.

Long term:

New Text Editor with both syntax highlighting and spellcheck enabled.
Comment 16 Sean Flanigan 2015-03-17 00:25:14 EDT
The short term plan sounds fine.

Long term, the new text editor probably won't make the problems which Patrick mentioned any easier to solve.  There aren't any components out there which solve the problem for us.  Ace and CodeMirror do live highlighting, but they don't do spell-check.  

There are possible solutions, but they all involve a fair bit of engineering (and maintenance), and may amount to re-implementing the browser's multilingual spell-check ourselves.  Or convincing the browser makers to expose spellcheck as an API we can call...
Comment 17 Luke Brooker 2015-03-17 00:41:44 EDT
This is not entirely true. There are options to "overlay" highlighting and other enhancements over text areas. I have some browser extensions that do this already. There are also options with contenteditable.

Basically, going back to a plain text editor and only implementing the basic features we need, should make it *easier* to maintain and add new features as we don't need to deal with the complexity of codemirror and all it's features we don't use.
Comment 18 Michelle Kim 2015-03-17 02:02:36 EDT
Thanks guys for your input. I will narrow down the scope of this bug to enable syntax highlighting / CodeMirror in source,  so that we can meet the short term goals first.

For the long term goals, we shall track with new editor RFE.
Comment 19 Michelle Kim 2015-03-17 21:34:41 EDT
Syntax highlighting is already active in Source, so this bug is to make sure white space highlighting is always on in source.
 - standard copy (Ctrl + C) should not copy the visible white space characters
Comment 20 Alex Eng 2015-03-24 01:19:47 EDT
Pull request for zanata-server:
https://github.com/zanata/zanata-server/pull/742
Comment 21 Damian Jansen 2015-03-25 01:51:47 EDT
Verified (release) at 90df3ec5fe4a3e441a8aafe77be5367246ecad1c

Note You need to log in before you can comment on or make changes to this bug.