Bug 833653 - Transaction time out when processing large glossary file
Summary: Transaction time out when processing large glossary file
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Zanata
Classification: Retired
Component: Performance
Version: 1.6-SNAPSHOT
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 1.7
Assignee: Alex Eng
QA Contact: Ding-Yi Chen
URL:
Whiteboard:
Depends On: 826821
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-06-20 03:36 UTC by James Ni
Modified: 2012-09-12 23:48 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Feature: Allow users to specify batch size for processing during glossary upload. Default size = 50 Reason: To eliminate transaction time out when processing large glossary file.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-09-11 05:11:25 UTC
Embargoed:


Attachments (Terms of Use)
part of log to show the traceback of error (18.50 KB, application/octet-stream)
2012-06-20 03:36 UTC, James Ni
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 826383 0 unspecified CLOSED [mvn] Glossary-push compendium PO file: OutOfMemory for big file 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 826821 0 unspecified CLOSED [zanata-python-client] glossary-push failed on big compendium po file 2021-02-22 00:41:40 UTC

Internal Links: 826383 826821

Description James Ni 2012-06-20 03:36:20 UTC
Created attachment 593119 [details]
part of log to show the traceback of error

Description of problem:

Transaction on server is time out due to processing this large file.

How reproducible:

Steps to Reproduce:
1. Using zanata python client to push compendium glossary file to server 
   zanata-empire.lab.eng.bne.redhat.com
2. Download compendium-zh_CN-LATEST.po and clean up
3. run zanata glossary push  --lang zh-Hans compendium-zh_CN-LATEST.po
  
Actual results:
The Client receive 500 error from server: The server encountered an internal error () that prevented it from fulfilling this request.

Expected results:
200 Success response from server

Additional info:

Actually, the Transaction on server is time out due to processing this large file, the attatchment is part of the log of server, it shows that server indeed receive the glossary items from client and store them to database, but since there are a huge amount of glossary items, the server failed at the end since timeout is set to 300s for Transaction.

I think maybe no need to change the Transaction timeout setting on server, but on the other hand, try to break the large file to small part on client and then push to server.

Comment 1 Alex Eng 2012-06-21 00:32:23 UTC
Implemented batch transaction on server side.



See https://github.com/zanata/zanata/commit/0136107b2c9bbd76bb34ae16c9cfca8e3c364505

Comment 2 Ding-Yi Chen 2012-06-27 04:22:48 UTC
This fix works for 1.7 Servers but not 1.6.1 servers.


Reassigned.

Comment 3 Ding-Yi Chen 2012-06-27 05:40:07 UTC
After talked to Alex, The server side fix will only go to 1.7.
As for 1.6 servers, workaround can be applied to client.


Note You need to log in before you can comment on or make changes to this bug.