Bug 1036937

Summary: RFE: Build translated PO files
Product: [Community] PressGang CCMS Reporter: Lee Newson <lnewson>
Component: DocBook-builderAssignee: Lee Newson <lnewson>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 1.2CC: mcaspers, rlandman, ykatabam
Target Milestone: ---   
Target Release: 1.6   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-05-28 21:56:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1095978, 1101075    
Bug Blocks:    

Description Lee Newson 2013-12-02 22:46:23 UTC
Currently we have no mechanism to build PO files from translated topics/content specs, however since Publican deals with translations via POT/PO files it would be handy to have a way to do this.

Comment 6 Matthew Casperson 2014-05-22 03:30:55 UTC
With csprocessor 1.6-SNAPSHOT, Build 201405201916 (i.e. 1.6-1) and a spec with no subtitle Book_Info.xml has this:

<subtitle><<contentSpec.subtitle>></subtitle>

Comment 7 Matthew Casperson 2014-05-22 03:50:25 UTC
NPE building a fairly simple spec:

csprocessor preview -y 11533
CSProcessor client version: 1.6-SNAPSHOT, Build 201405201916
Loading configuration from /home/mcasperson/.config/csprocessor.ini
Connecting to PressGang server: http://skynet-dev.usersys.redhat.com:8080/pressgang-ccms/rest/

Starting to validate...
Starting first validation pass...
Attempting to download all the latest topics...
Starting second validation pass...
WARN:  No Abstract specified, so a default will be used instead.
INFO:  The Content Specification is valid.

Starting to build...
Doing en-US Populate Database Pass
Doing Revisions Fixed URL Pass
Doing en-US First topic pass
        Processing 1 Topics
        First topic Pass 100% Done
Doing en-US Topic Link Pass
Doing en-US Spec Topic Pass
        Processing 1 Spec Topics
        Processing Pass 100% Done
Building the ZIP file
java.lang.NullPointerException
        at org.jboss.pressgang.ccms.contentspec.builder.utils.DocBookBuildUtilities.getKeyValueNodeText(DocBookBuildUtilities.java:858)
        at org.jboss.pressgang.ccms.contentspec.builder.DocBookBuilder.buildBookBase(DocBookBuilder.java:1678)
        at org.jboss.pressgang.ccms.contentspec.builder.DocBookBuilder.doBuildZipPass(DocBookBuilder.java:1641)
        at org.jboss.pressgang.ccms.contentspec.builder.DocBookBuilder.buildBook(DocBookBuilder.java:532)
        at org.jboss.pressgang.ccms.contentspec.builder.DocBookBuilder.buildBook(DocBookBuilder.java:402)
        at org.jboss.pressgang.ccms.contentspec.builder.ContentSpecBuilder.buildBook(ContentSpecBuilder.java:97)
        at org.jboss.pressgang.ccms.contentspec.client.commands.BuildCommand.buildContentSpec(BuildCommand.java:729)
        at org.jboss.pressgang.ccms.contentspec.client.commands.BuildCommand.process(BuildCommand.java:517)
        at org.jboss.pressgang.ccms.contentspec.client.commands.AssembleCommand.process(AssembleCommand.java:113)
        at org.jboss.pressgang.ccms.contentspec.client.commands.PreviewCommand.process(PreviewCommand.java:51)
        at org.jboss.pressgang.ccms.contentspec.client.Client.processArgs(Client.java:258)
        at org.jboss.pressgang.ccms.contentspec.client.Client.main(Client.java:111)

Internal processing error!

Spec is

ID = 11533
Title = CCMS Workflow
Subtitle = CCMS Workflow
Product = PressGang CCMS
Edition = 1
Format = DocBook 4.5
Copyright Holder = Red Hat
Locale = en-US

Chapter: This is a test chapter x2
  Section: A sub section x3
    JBoss Logging Features [4578]

Comment 8 Lee Newson 2014-05-23 02:52:02 UTC
Added in 1.6-SNAPSHOT build 201405231224

Firstly the above errors are now fixed.

Secondly onto details:

The build command now accepts the "publican-po" format option which will produce a normal en-US build with the required po and pot files specified for the "--lang" option.

Example command:

csprocessor preview 6895 --lang ja --target-lang ja-JP --format publican-po

There are a few things to note:

- The keywords produced by topic tags will never have translations.

- The publican bugs I mentioned in Comment #1 have been fixed or are being fixed so I haven't added workarounds to get them to work in publican 4.1 or lower.

- Building this way will not produce the translation chapter or reports, as that would be producing content that is different to en-US which is not possible. This also applies to the "Translate additional XML" links for revision histories.

- There is a slight difference in normal en-US builds to publican-po builds. That difference is that entities are resolved, instead of leaving publican to resolve them. This is needed as the strings in Zanata will have had their entities resolved as well.

Additional Technical Details:

Due to previous translation bugs I've added code that will check if a translation exists for the V1 version and break it down, since it wasn't breaking content down into small enough chunks.

For the  V2 version, it's problem is that it was removing whitespace in verbatim elements, as such I've made it re-run the over verbatim elements and used the V3 for the original and translated V2 string for the translated component in the pot/po files.

Comment 10 Lee Newson 2014-05-23 03:00:30 UTC
Another thing I forgot to note, is that any error/report chapters will be untranslated, since the builder doesn't have any of those strings translated.

Comment 16 Matthew Casperson 2014-05-26 04:30:05 UTC
Confirmed that freshly pushed content is now results in the same final output using the PO or inline method.

Comment 17 Matthew Casperson 2014-05-26 05:02:24 UTC
See BZ#1101050 for the publican bug that prevents verbatim elements with cdata content from being built from a PO file.

Comment 19 Matthew Casperson 2014-05-26 21:07:44 UTC
Pushed a topic with entities to zanata, translated them, synced them back and built a book ok.