Bug 1269833 - Sat 6.1.1 "Resync-ing" unchanged repo takes more than 50% of initial sync time
Sat 6.1.1 "Resync-ing" unchanged repo takes more than 50% of initial sync time
Status: CLOSED DUPLICATE of bug 1264560
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Content Management (Show other bugs)
6.1.1
x86_64 Linux
unspecified Severity high (vote)
: Unspecified
: --
Assigned To: Katello Bug Bin
Katello QA List
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-08 06:25 EDT by Pradeep Kumar Surisetty
Modified: 2015-10-09 15:52 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-10-09 15:52:23 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Pradeep Kumar Surisetty 2015-10-08 06:25:37 EDT
Description of problem:

Resyncing couple of repositories  ( RHEL 5 x86_64, RHEL 6 i386, RHEL 7 x86_64) without any changes in local repo.  Still it takes more than 50% of initial sync time. 
Have setup local repo. Not Through CDN.


Initial Sync timings:
----------------------
Repo :  Size:  Timing

1) RHEL 6 i386 :  22.7 GB  :  0:22:08
2) RHEL 5 x86_64: 38.2 GB  :  0:25:22
3) RHEL7 x86_64:  8.64 GB  :  0:09:12


Resync timings:
----------------

1) RHEL 6 i386 :  22.7 GB  :  0:11:40
2) RHEL 5 x86_64: 38.2 GB  :  0:11:40
3) RHEL7 x86_64:  8.64 GB  :  0:05:07




Version-Release number of selected component (if applicable):

6.1.1 Satelitte

How reproducible:


Steps to Reproduce:
1. Sync Above repos for the first time. Capture timings.

   time hammer -u admin -p changeme repository synchronize --id 1 --organization-id 1

2. Resync same without any changes in repo. capture timings. 
    
 time hammer -u admin -p changeme repository synchronize --id 1 --organization-id 1

3.

Actual results:

The required to time perform a resync is more than 50% that of the original sync

Expected results:

"Resyncing" a repo should only require downloading the files under repodata/ to check for changes and if necessary downloading anything that has changed.  This should be minimal on a resync with no changes.

Additional info:
Comment 1 Pradeep Kumar Surisetty 2015-10-08 06:26:49 EDT
Capture IO Traffic as well. I see 100 MB being copied even though no change in repo. 

[root@pman05 home]#  netstat -e -I="ens1f1";time hammer -u admin -p changeme repository synchronize --id 1 --organization-id 1; netstat -e -I="ens1f1"
Kernel Interface table
ens1f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.50.5  netmask 255.255.255.0  broadcast 172.17.50.255
        inet6 fe80::20f:53ff:fe08:23dd  prefixlen 64  scopeid 0x20<link>
        ether 00:0f:53:08:23:dd  txqueuelen 1000  (Ethernet)
        RX packets 360686294  bytes 545770882763 (508.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 59144624  bytes 4214775507 (3.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 42 

[...........................................................] [100%]
No new packages.


real    11m34.004s
user    0m4.505s
sys    0m1.676s
Kernel Interface table
ens1f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.50.5  netmask 255.255.255.0  broadcast 172.17.50.255
        inet6 fe80::20f:53ff:fe08:23dd  prefixlen 64  scopeid 0x20<link>
        ether 00:0f:53:08:23:dd  txqueuelen 1000  (Ethernet)
        RX packets 360721771  bytes 545824192035 (508.3 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 59150786  bytes 4215209922 (3.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 42
Comment 2 Pradeep Kumar Surisetty 2015-10-08 06:27:41 EDT
Apache access log:

Data is being copied. 

rver/i386/os/.treeinfo HTTP/1.1" 404 255 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:04:59:20 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/treeinfo HTTP/1.1" 404 254 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:21 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/repomd.xml HTTP/1.1" 200 3824 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/f4e9b0b4cbecc01842f5b628e5a0acff752421b1-comps-rhel-i386-server-6.xml.gz HTTP/1.1" 200 213186 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/ea5e62b03b31e5fbed3c01e2ba35eb413e200cbd-comps-rhel-i386-server-6.xml HTTP/1.1" 200 1160646 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/productid.gz HTTP/1.1" 200 1690 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/80a76bc2c4e527067649520361f5eca4458002431637039651337de1e94091e3-updateinfo.xml.gz HTTP/1.1" 200 2671565 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/ec0bd3d63cb517a7e674ca67358b80a830c68d18-other.xml.gz HTTP/1.1" 200 4156942 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/908951ac5376d50084593c2c850e4893c1295afa-primary.xml.gz HTTP/1.1" 200 19671710 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/e019a0cfc2c5fb5cb621e5133db3b9899d189ff5-filelists.xml.gz HTTP/1.1" 200 22944836 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x8
Comment 5 Bryan Kearney 2015-10-09 15:52:23 EDT

*** This bug has been marked as a duplicate of bug 1264560 ***

Note You need to log in before you can comment on or make changes to this bug.