Bug 1120034 - pushing translations is too slow
Summary: pushing translations is too slow
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Zanata
Classification: Retired
Component: Performance
Version: 3.3
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 3.5
Assignee: zanata-dev-internal
QA Contact: Ding-Yi Chen
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-07-16 05:59 UTC by Akira TAGOH
Modified: 2015-03-02 23:17 UTC (History)
5 users (show)

Fixed In Version: 3.5.0-SNAPSHOT (git-jenkins-zanata-server-github-pull-requests-2090)
Doc Type: Bug Fix
Doc Text:
Story Points: 3
Clone Of:
Environment:
Last Closed: 2015-01-26 23:28:01 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1124615 None None None Never
Red Hat Bugzilla 1197902 None None None Never

Internal Links: 1124615 1197902

Description Akira TAGOH 2014-07-16 05:59:12 UTC
Description of problem:
just transferring a few bytes of translations takes 10~20 seconds. it's too slow and takes the time too long when pushing multiple documents.

Comment 1 Ding-Yi Chen 2014-07-28 02:18:45 UTC
Mind telling us what projects you were pushing and what command did you use for pushing?


It is possible you were experiencing Bug 1102964, which is fixed in 3.4.2.

I hereby close this bug as duplicate of Bug 1102964.
Please reopen this bug if same thing happened again for Zanata server version after 3.4.2.

*** This bug has been marked as a duplicate of bug 1102964 ***

Comment 2 Akira TAGOH 2014-07-28 06:51:23 UTC
(In reply to Ding-Yi Chen from comment #1)
> Mind telling us what projects you were pushing and what command did you use
> for pushing?

man-pages-l10n

> It is possible you were experiencing Bug 1102964, which is fixed in 3.4.2.

There are no way to try 3.4.2 at this moment since the instance on zanata.org is still old one apparently. though trying with --copy-trans false is even slow.

Aside from that pulling in translations is slow too.

Comment 3 Ding-Yi Chen 2014-07-29 00:15:38 UTC
(In reply to Akira TAGOH from comment #2)
> (In reply to Ding-Yi Chen from comment #1)
> > Mind telling us what projects you were pushing and what command did you use
> > for pushing?
> 
> man-pages-l10n
> 
> > It is possible you were experiencing Bug 1102964, which is fixed in 3.4.2.
> 
> There are no way to try 3.4.2 at this moment since the instance on
> zanata.org is still old one apparently. though trying with --copy-trans
> false is even slow.
> 
> Aside from that pulling in translations is slow too.

This is what I see in https://translate.zanata.org/zanata/

Zanata 3.4.2 (git-server-3.4.2-dirty)

What version did you see?

I will investigate the pull part.

Comment 4 Akira TAGOH 2014-07-29 08:39:18 UTC
Ah, I thought that meant the API version.

[INFO] client API version: 3.3.1, server API version: 3.3.1

Comment 5 Akira TAGOH 2014-07-29 08:40:44 UTC
and tried the above "--copy-trans false" thing on that version.

Comment 6 Ding-Yi Chen 2014-07-30 05:26:12 UTC
Guess it is different problem then, reopen.

Comment 7 Carlos Munoz 2014-07-30 05:37:43 UTC
Some notes from our estimation discussions:

Processing on the server is the main time taken. Network transfer is not the bottleneck.
With import = merge, we have to check history. Might be able to improve this query (i.e. load history in memory rather than query database each time).
Issues started after we moved to OpenShift - could be related to async and threading performance on OpenShift.
We'll take a time-boxed approach, investigate and fix what we can in the allocated time.

Comment 8 Carlos Munoz 2014-08-12 04:27:45 UTC
Topics of interest:
- Pushing of small files
- merge=import history search
- Web sockets for eap 6.3
- HornetQ long polling

Comment 10 Patrick Huang 2014-09-10 23:36:50 UTC
https://github.com/zanata/zanata-server/pull/571

Comment 13 Ding-Yi Chen 2014-09-23 02:57:35 UTC
On  my machine: man-pages-l10n tooks 86.11s user 8.43s system 3% cpu 49:45.66 total with
command:
time mvn zanata:push -Dzanata.pushType=both -Dzanata.srcDir=pot -Dzanata.transDir=po

man-pages-l10n contains 69849 text flows (entry), 0.04 was translated,
so it is about 72642.96 strings to be processed.

So the speed is 24.33 strings/sec. Much better then previous 3 strings/sec.
Zanata 3.5.0-SNAPSHOT (git-jenkins-zanata-server-github-pull-requests-2090)


Note You need to log in before you can comment on or make changes to this bug.