Bug 879058

Summary: update_site takes a long time to run
Product: [Community] Publican Reporter: Marco Grigull <mgrigull>
Component: publicanAssignee: Jeff Fearn <jfearn>
Status: CLOSED NOTABUG QA Contact: Ruediger Landmann <rlandman>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 2.8CC: rlandman
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-11-26 23:20:38 EST Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Marco Grigull 2012-11-21 18:31:01 EST
Description of problem:
on a system with 1000's of publican documentation packages running publican update_site regenerates all index files regardless of whether they were updated/deleted.  Given that it can take ~ 10 minutes to run, and an external rsync process may be syncing the entire layout to another location, some of those index's disappear causing additional warnings:


Nov 21 15:13:08 documentation-01 rsyncd[1085]: rsync: read errors mapping "docs/kn-IN/Red_Hat_Subscription_Asset_Manager/1.0/index.
html" (in docsredhat): No data available (61)
Nov 21 15:13:09 documentation-01 rsyncd[1085]: rsync: read errors mapping "docs/kn-IN/Red_Hat_Update_Infrastructure/index.html" (in
 docsredhat): No data available (61)
Nov 21 15:13:09 documentation-01 rsyncd[1085]: rsync: read errors mapping "docs/kn-IN/Red_Hat_Update_Infrastructure/versions_menu.h
tml" (in docsredhat): No data available (61)
Nov 21 15:13:16 documentation-01 rsyncd[1085]: rsync: read errors mapping "docs/ko-KR/JBoss_Web_Framework_Kit/1.0/Release_Notes/for
mat_menu.html" (in docsredhat): No data available (61)
Nov 21 15:13:16 documentation-01 rsyncd[1085]: rsync: read errors mapping "docs/ko-KR/JBoss_Web_Framework_Kit/1.1/RichFaces_Develop
er_Guide/index.html" (in docsredhat): No data available (61)
Nov 21 15:13:16 documentation-01 rsyncd[1085]: rsync: read errors mapping "docs/ko-KR/JBoss_Web_Framework_Kit/1.2/Snowdrop_User_Gui
de/index.html" (in docsredhat): No data available (61)
Nov 21 15:13:16 documentation-01 rsyncd[1085]: rsync: read errors mapping "docs/ko-KR/JBoss_Web_Framework_Kit/2/Hibernate_Search/in
dex.html" (in docsredhat): No data available (61)



Expected results:
Indexes are selectively updated, or only regenerated whenever package installs/removes/changes have occured.
Comment 1 Jeff Fearn 2012-11-21 18:33:19 EST
*** Bug 879056 has been marked as a duplicate of this bug. ***
Comment 2 Jeff Fearn 2012-11-26 23:20:38 EST
Testing revealed:

# publican site_stats

The database contains books that cover 23 products, 39 languages,  totaling 1573 packages

# time /usr/bin/publican update_site

real	1m43.849s
user	1m26.366s
sys	0m9.034s

This is pretty good and drastic improvements would be hard to achieve.