Bug 580360

Summary: Publican does not allow programlisting/screen in entry
Product: [Community] Publican Reporter: Laura Bailey <lbailey>
Component: publicanAssignee: Jeff Fearn 🐞 <jfearn>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: 1.6CC: jfearn, jito, misty, mmcallis, mospina, publican-list, raphael, rlandman
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: publican-1.6.3-0.fc12 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-05-17 18:56:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Screenshot of a programlisting inside a table
none
Screenshot of the same data presented as nested variablelists, compliant with the new Publican rules
none
HTML output from a real-world example
none
Chapter with table used for test in comment #15
none
PO file used for test in Comment #15 none

Description Laura Bailey 2010-04-08 04:27:10 UTC
Description of problem:
The fix for https://bugzilla.redhat.com/show_bug.cgi?id=571633 mandates that programlisting and screen cannot be used in an entry tag in a table, as follows, although this is valid docbook:

<row>
    <entry valign="top">
        <para>
            .....
        </para>
        <programlisting>
            .....
        </programlisting>
    </entry>   
</row>

This affects a number of documents, most importantly the Seam Reference Guide:
http://downtown.englab.bne.redhat.com/drafts/JBoss_Enterprise_Application_Platform/5.0.1/Seam_Reference_Guide/html-single/#itext.document

This document is currently in use in four products (EAP 5.0, 5.0.1, 4.2, 4.3), plus upstream.

In particular, our translation team relies on this document remaining buildable between the time of product release, and the time of translation. Our docs team has very little scope to go back and correct syntax errors that are a direct result of upgrading publican. This book alone is five instances of 800p, and other books are affected.

Can programlisting and screen be made valid inside entry tags again please?


Version-Release number of selected component (if applicable):
publican-1.6.2-0.fc12.src.rpm

How reproducible:
Every time programlisting or screen appears within an entry tag, as shown above.

Steps to Reproduce:
1. svn co http://anonsvn.jboss.org/repos/jbossas/projects/docs/enterprise/5.0.1/Seam_Reference_Guide/
2. publican update_pot

Actual Results:
DEBUG: Publican::Translate: TODO: header: confirm this doesn't break on update
	Processing file en-US/Security.xml
ERROR: Verbatim content can not be embedded in translatable content, found a programlisting in a entry! at /usr/bin/publican line 486

Expected Results:
Updated pot files.

Comment 1 Jeff Fearn 🐞 2010-04-08 05:05:19 UTC
(In reply to comment #0)
> Can programlisting and screen be made valid inside entry tags again please?

Get translation to sign off on this and I'll happily remove the restriction; it will of course reintroduce the bug fixed in 571633.

Comment 2 Jeff Fearn 🐞 2010-04-08 05:23:09 UTC
FWIW there is another option, but it also invalidates "valid" DocBook.

We could remove entry from the translatable tag list, people would have to use para tags for text inside entry tags they wanted translated.

e.g. this would no longer work for translations

<entry>The fists of doom</entry>

it would have to be

<entry><para>The fists of doom</para></entry>

That way a programlisting inside an entry, but not inside a para inside an entry, would work.

No idea how many existing books this would affect.

Mixed mode tags are Evil.

Comment 3 Laura Bailey 2010-04-08 06:01:49 UTC
That alternative would probably take just as long, if not longer, to change in the books.

I suppose what I was hoping for would be an alternate fix to BZ#571633 to prevent the programlistings/screens from losing content, rather than using these workarounds. I have no idea if this is possible, though.

Comment 4 Jeff Fearn 🐞 2010-04-08 22:45:57 UTC
programlisting, screen, and literallayout are not translatable content [1] and should never be visible in the translation files. When those tags are embedded in tags, like para and entry, that do contain translatable content, then the non translatable content leaks in to the translatable content.

The problem here is that even though your content is sanely laid out without mixed-mode content, the tag it's in is, unfortunately, one of the mixed mode tags and thus gets caught in the "though shall not embed non-translatable content in translatable content" logic.

e.g. this stupidity is legal in DocBook, but how do you translate it?

<entry>
  Start a sentence here,
  <para>Insert a paragraph!</para>
<screen>
Add
a
screen of
commands
</screen>
  then finish your sentence here.
</entry>


It may be possible to detect sane versus insane layout, but I couldn't give a time line on when that would be complete and it would probably introduce complex corner cases :(

Cheers, Jeff.

[1] As discussed with the translation team.

Comment 5 Ruediger Landmann 2010-04-09 06:33:19 UTC
(In reply to comment #2)
> FWIW there is another option, but it also invalidates "valid" DocBook.
> 
> We could remove entry from the translatable tag list, people would have to use
> para tags for text inside entry tags they wanted translated.
> 
> That way a programlisting inside an entry, but not inside a para inside an
> entry, would work.

This actually seems to me like the best long-term solution. 

Sometimes, a <screen> or <programlisting> inside an <entry> is exactly right, semantically. 

What you're suggesting forces writers to make the contents of <entry>s more semantically meaningful, which has an immediate benefit in indicating whether this content is supposed to be translatable or not.

However, I have no idea either of what the impact on existing docs is likely to be. I suspect it would be rather severe. :(

Comment 6 Jeff Fearn 🐞 2010-04-09 06:45:25 UTC
FWIW I'm questioning the position that verbatim tags aren't translatable.

You'd put the output of a command in a screen, lots of commands have localised output, surely if localised is available it should be translated?

e.g.

wget --help

VS.

LANG=ja_JP wget --help

Maybe the position that verbatim tags aren't translatable is flat out wrong?

Comment 7 Jeff Fearn 🐞 2010-04-09 09:56:36 UTC
FYI I have checked in a mod to the 1.6 branch, which attempts to handle translating verbatim tags. I'm not sure what it may have broken because it's pretty evil!

Will require much testing and gnashing of teeth.

Cheers, Jeff.

Comment 8 Jeff Fearn 🐞 2010-04-09 22:41:12 UTC
I just checked in a mod to the 1.6 branch, which given the following idiocy:


<entry>
 Start a sentence here, 
 <para>
  Insert a paragraph inside!
 </para>
<screen>
Add
    a
screen of
	commands
</screen>
 then finish your sentence here.
</entry>


Will generates the following translation entries:

msgid "Start a sentence here,"
msgstr ""

msgid "Insert a paragraph inside!"
msgstr ""

msgid ""
"\n"
"Add\n"
"    a\n"
"screen of\n"
"	commands\n"
msgstr ""

msgid ""
"\n"
"  then finish your sentence here.\n"
msgstr ""

The last entry is incorrectly formatted due to the way it splits tags when determining translation strings. It merges correctly in to the translated XML though.

Translatable idiocy ... I'm guessing I'm going to hell.

Comment 9 Misty Stanley-Jones 2010-04-12 07:01:15 UTC
I'm attaching two screenshots showing some data presented in a tablular view (with a programlisting inside an entry) and the same data presented as nested variablelists. It is eminently more clear in the tabular format, at least with the current stylesheets.

Comment 10 Misty Stanley-Jones 2010-04-12 07:10:50 UTC
Created attachment 405896 [details]
Screenshot of a programlisting inside a table

Comment 11 Misty Stanley-Jones 2010-04-12 07:11:32 UTC
Created attachment 405898 [details]
Screenshot of the same data presented as nested variablelists, compliant with the new Publican rules

Comment 12 Jeff Fearn 🐞 2010-04-12 07:55:19 UTC
(In reply to comment #9)
> I'm attaching two screenshots showing some data presented in a tablular view
> (with a programlisting inside an entry) and the same data presented as nested
> variablelists. It is eminently more clear in the tabular format, at least with
> the current stylesheets.    

They both look pretty poor to me, at least the second one is on brand.

I've repeatedly suggested to ECS that they sit down and define a usage of reference|synopsis|classsynopsis etc for use in documenting APIs, which this is very similar too. This would give you a style dedicated to this kind of information instead of trying to shoe horn it in to generic layouts.

No one has ever taken me up on that, so we still get ugly stuff like both these examples and every writer does it in a different way and completely ignores translation in the process.

In any event you should join the publican list discussion on translating verbatim items.

Cheers, Jeff.

Comment 13 Jeff Fearn 🐞 2010-04-13 21:11:13 UTC
*** Bug 581767 has been marked as a duplicate of this bug. ***

Comment 14 Jeff Fearn 🐞 2010-05-04 03:48:33 UTC
As per discussion on list, made verbatim tags translatable, thus allowing nesting.

Comment 15 Ruediger Landmann 2010-05-06 04:17:19 UTC
I tested the crazy example in comment #8 on version 1.6.3.t122; the POT file is created correctly, and strings are matched back without error; however, the final string ("then finish your sentence here") doesn't get used. 

So:

#. Tag: entry
#, no-c-format
msgid "Start a sentence here,"
msgstr "Start einen Satz hier,"

#. Tag: para
#, no-c-format
msgid "Insert a paragraph inside!"
msgstr "Legen Sie einen Absatz hier!"

#. Tag: screen
#, no-c-format
msgid "\n"
"Add\n"
"    a\n"
"screen of\n"
" commands\n"
""
msgstr "\n"
"Schreibe\n"
"    einen\n"
"Bildschirm von\n"
" Befehlen\n"
""

#. Tag: screen
#, no-c-format
msgid "\n"
" then finish your sentence here.\n"
""
msgstr "\n"
"  dann beenden Sie Ihre Satz hier.\n"
""

builds into:

Start einen Satz hier,
Legen Sie einen Absatz hier!

Schreibe
    einen
Bildschirm von
 Befehlen


then finish your sentence here.

Comment 16 Ruediger Landmann 2010-05-06 19:46:17 UTC
Created attachment 412145 [details]
HTML output from a real-world example

I also had a translator test a slightly less crazy example from an actual manual -- Publican HTML output attached. 

This book had <programlisting>s embedded in table <entry>s; the translated content got matched and used successfully, and the <programlisting>s kept their correct formatting.

The "slightly less crazy" part was that in this example, there was no <para> content trailing the <programlisting>s.

Comment 17 Ruediger Landmann 2010-05-11 01:30:59 UTC
Created attachment 413015 [details]
Chapter with table used for test in comment #15

Comment 18 Ruediger Landmann 2010-05-11 01:32:44 UTC
Created attachment 413016 [details]
PO file used for test in Comment #15

Comment 19 Jeff Fearn 🐞 2010-05-11 04:10:25 UTC
Fixed in commit 1.6.2 t169

Comment 20 Ruediger Landmann 2010-05-11 04:33:31 UTC
Verified in 1.6.2.t169 -- trailing paras are replaced with contents from PO file

Output from example in comment #15 is now as expected:

Start einen Satz hier,
Legen Sie einen Absatz hier!

Schreibe
    einen
Bildschirm von
 Befehlen


dann beenden Sie Ihre Satz hier.

Comment 21 jito 2010-05-11 22:25:20 UTC
The sample po file translated and built fine in Japanese.

Comment 22 Fedora Update System 2010-05-13 22:20:21 UTC
publican-1.6.3-0.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/publican-1.6.3-0.fc13

Comment 23 Fedora Update System 2010-05-13 22:21:41 UTC
publican-1.6.3-0.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/publican-1.6.3-0.fc12

Comment 24 Fedora Update System 2010-05-15 20:35:58 UTC
publican-1.6.3-0.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update publican'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/publican-1.6.3-0.fc12

Comment 25 Fedora Update System 2010-05-15 20:45:17 UTC
publican-1.6.3-0.fc13 has been pushed to the Fedora 13 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update publican'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/publican-1.6.3-0.fc13

Comment 26 Fedora Update System 2010-05-17 18:55:13 UTC
publican-1.6.3-0.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 27 Fedora Update System 2010-05-17 18:59:06 UTC
publican-1.6.3-0.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.