Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1269833 - Sat 6.1.1 "Resync-ing" unchanged repo takes more than 50% of initial sync time
Summary: Sat 6.1.1 "Resync-ing" unchanged repo takes more than 50% of initial sync time
Keywords:
Status: CLOSED DUPLICATE of bug 1264560
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Content Management
Version: 6.1.1
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: Unspecified
Assignee: Katello Bug Bin
QA Contact: Katello QA List
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-10-08 10:25 UTC by Pradeep Kumar Surisetty
Modified: 2015-10-09 19:52 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-10-09 19:52:23 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Pradeep Kumar Surisetty 2015-10-08 10:25:37 UTC 
Description of problem:

Resyncing couple of repositories  ( RHEL 5 x86_64, RHEL 6 i386, RHEL 7 x86_64) without any changes in local repo.  Still it takes more than 50% of initial sync time. 
Have setup local repo. Not Through CDN.


Initial Sync timings:
----------------------
Repo :  Size:  Timing

1) RHEL 6 i386 :  22.7 GB  :  0:22:08
2) RHEL 5 x86_64: 38.2 GB  :  0:25:22
3) RHEL7 x86_64:  8.64 GB  :  0:09:12


Resync timings:
----------------

1) RHEL 6 i386 :  22.7 GB  :  0:11:40
2) RHEL 5 x86_64: 38.2 GB  :  0:11:40
3) RHEL7 x86_64:  8.64 GB  :  0:05:07




Version-Release number of selected component (if applicable):

6.1.1 Satelitte

How reproducible:


Steps to Reproduce:
1. Sync Above repos for the first time. Capture timings.

   time hammer -u admin -p changeme repository synchronize --id 1 --organization-id 1

2. Resync same without any changes in repo. capture timings. 
    
 time hammer -u admin -p changeme repository synchronize --id 1 --organization-id 1

3.

Actual results:

The required to time perform a resync is more than 50% that of the original sync

Expected results:

"Resyncing" a repo should only require downloading the files under repodata/ to check for changes and if necessary downloading anything that has changed.  This should be minimal on a resync with no changes.

Additional info:
Comment 1 Pradeep Kumar Surisetty 2015-10-08 10:26:49 UTC 
Capture IO Traffic as well. I see 100 MB being copied even though no change in repo. 

[root@pman05 home]#  netstat -e -I="ens1f1";time hammer -u admin -p changeme repository synchronize --id 1 --organization-id 1; netstat -e -I="ens1f1"
Kernel Interface table
ens1f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.50.5  netmask 255.255.255.0  broadcast 172.17.50.255
        inet6 fe80::20f:53ff:fe08:23dd  prefixlen 64  scopeid 0x20<link>
        ether 00:0f:53:08:23:dd  txqueuelen 1000  (Ethernet)
        RX packets 360686294  bytes 545770882763 (508.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 59144624  bytes 4214775507 (3.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 42 

[...........................................................] [100%]
No new packages.


real    11m34.004s
user    0m4.505s
sys    0m1.676s
Kernel Interface table
ens1f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.50.5  netmask 255.255.255.0  broadcast 172.17.50.255
        inet6 fe80::20f:53ff:fe08:23dd  prefixlen 64  scopeid 0x20<link>
        ether 00:0f:53:08:23:dd  txqueuelen 1000  (Ethernet)
        RX packets 360721771  bytes 545824192035 (508.3 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 59150786  bytes 4215209922 (3.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 42
Comment 2 Pradeep Kumar Surisetty 2015-10-08 10:27:41 UTC 
Apache access log:

Data is being copied. 

rver/i386/os/.treeinfo HTTP/1.1" 404 255 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:04:59:20 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/treeinfo HTTP/1.1" 404 254 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:21 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/repomd.xml HTTP/1.1" 200 3824 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/f4e9b0b4cbecc01842f5b628e5a0acff752421b1-comps-rhel-i386-server-6.xml.gz HTTP/1.1" 200 213186 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/ea5e62b03b31e5fbed3c01e2ba35eb413e200cbd-comps-rhel-i386-server-6.xml HTTP/1.1" 200 1160646 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/productid.gz HTTP/1.1" 200 1690 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/80a76bc2c4e527067649520361f5eca4458002431637039651337de1e94091e3-updateinfo.xml.gz HTTP/1.1" 200 2671565 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/ec0bd3d63cb517a7e674ca67358b80a830c68d18-other.xml.gz HTTP/1.1" 200 4156942 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/908951ac5376d50084593c2c850e4893c1295afa-primary.xml.gz HTTP/1.1" 200 19671710 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x86_64"
172.17.50.5 - - [08/Oct/2015:05:11:22 -0400] "GET /pub//content/dist/rhel/server/6/6Server/i386/os/repodata/e019a0cfc2c5fb5cb621e5133db3b9899d189ff5-filelists.xml.gz HTTP/1.1" 200 22944836 "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-229.11.1.el7.x8
Comment 5 Bryan Kearney 2015-10-09 19:52:23 UTC 

*** This bug has been marked as a duplicate of bug 1264560 ***
Note You need to log in before you can comment on or make changes to this bug.