Bug 1120034 - pushing translations is too slow
pushing translations is too slow
Status: CLOSED CURRENTRELEASE
Product: Zanata
Classification: Community
Component: Performance (Show other bugs)
3.3
Unspecified Unspecified
unspecified Severity high
: ---
: 3.5
Assigned To: zanata-dev-internal
Ding-Yi Chen
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2014-07-16 01:59 EDT by Akira TAGOH
Modified: 2015-03-02 18:17 EST (History)
5 users (show)

See Also:
Fixed In Version: 3.5.0-SNAPSHOT (git-jenkins-zanata-server-github-pull-requests-2090)
Doc Type: Bug Fix
Doc Text:
Story Points: 3
Clone Of:
Environment:
Last Closed: 2015-01-26 18:28:01 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Akira TAGOH 2014-07-16 01:59:12 EDT
Description of problem:
just transferring a few bytes of translations takes 10~20 seconds. it's too slow and takes the time too long when pushing multiple documents.
Comment 1 Ding-Yi Chen 2014-07-27 22:18:45 EDT
Mind telling us what projects you were pushing and what command did you use for pushing?


It is possible you were experiencing Bug 1102964, which is fixed in 3.4.2.

I hereby close this bug as duplicate of Bug 1102964.
Please reopen this bug if same thing happened again for Zanata server version after 3.4.2.

*** This bug has been marked as a duplicate of bug 1102964 ***
Comment 2 Akira TAGOH 2014-07-28 02:51:23 EDT
(In reply to Ding-Yi Chen from comment #1)
> Mind telling us what projects you were pushing and what command did you use
> for pushing?

man-pages-l10n

> It is possible you were experiencing Bug 1102964, which is fixed in 3.4.2.

There are no way to try 3.4.2 at this moment since the instance on zanata.org is still old one apparently. though trying with --copy-trans false is even slow.

Aside from that pulling in translations is slow too.
Comment 3 Ding-Yi Chen 2014-07-28 20:15:38 EDT
(In reply to Akira TAGOH from comment #2)
> (In reply to Ding-Yi Chen from comment #1)
> > Mind telling us what projects you were pushing and what command did you use
> > for pushing?
> 
> man-pages-l10n
> 
> > It is possible you were experiencing Bug 1102964, which is fixed in 3.4.2.
> 
> There are no way to try 3.4.2 at this moment since the instance on
> zanata.org is still old one apparently. though trying with --copy-trans
> false is even slow.
> 
> Aside from that pulling in translations is slow too.

This is what I see in https://translate.zanata.org/zanata/

Zanata 3.4.2 (git-server-3.4.2-dirty)

What version did you see?

I will investigate the pull part.
Comment 4 Akira TAGOH 2014-07-29 04:39:18 EDT
Ah, I thought that meant the API version.

[INFO] client API version: 3.3.1, server API version: 3.3.1
Comment 5 Akira TAGOH 2014-07-29 04:40:44 EDT
and tried the above "--copy-trans false" thing on that version.
Comment 6 Ding-Yi Chen 2014-07-30 01:26:12 EDT
Guess it is different problem then, reopen.
Comment 7 Carlos Munoz 2014-07-30 01:37:43 EDT
Some notes from our estimation discussions:

Processing on the server is the main time taken. Network transfer is not the bottleneck.
With import = merge, we have to check history. Might be able to improve this query (i.e. load history in memory rather than query database each time).
Issues started after we moved to OpenShift - could be related to async and threading performance on OpenShift.
We'll take a time-boxed approach, investigate and fix what we can in the allocated time.
Comment 8 Carlos Munoz 2014-08-12 00:27:45 EDT
Topics of interest:
- Pushing of small files
- merge=import history search
- Web sockets for eap 6.3
- HornetQ long polling
Comment 10 Patrick Huang 2014-09-10 19:36:50 EDT
https://github.com/zanata/zanata-server/pull/571
Comment 13 Ding-Yi Chen 2014-09-22 22:57:35 EDT
On  my machine: man-pages-l10n tooks 86.11s user 8.43s system 3% cpu 49:45.66 total with
command:
time mvn zanata:push -Dzanata.pushType=both -Dzanata.srcDir=pot -Dzanata.transDir=po

man-pages-l10n contains 69849 text flows (entry), 0.04 was translated,
so it is about 72642.96 strings to be processed.

So the speed is 24.33 strings/sec. Much better then previous 3 strings/sec.
Zanata 3.5.0-SNAPSHOT (git-jenkins-zanata-server-github-pull-requests-2090)

Note You need to log in before you can comment on or make changes to this bug.