Bug 756433

Summary: Grinder ignores proxy setting
Product: [Retired] Pulp Reporter: Lukas Zapletal <lzap>
Component: user-experienceAssignee: John Matthews <jmatthew>
Status: CLOSED NOTABUG QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 1.0.0CC: skarmark
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-14 14:43:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Lukas Zapletal 2011-11-23 15:06:20 UTC 
Description of problem:

When I setup pulp proxy, pulp does not use it when syncing packages. I just do not see any traffic on my squid.

grep proxy /etc/pulp/pulp.conf 
proxy_url: http://192.168.1.4
proxy_port: 3128

service pulp-server restart

I am trying to sync Fedora 15 yum repository (OS).

Version-Release number of selected component (if applicable):
pulp-0.0.244-5.fc15.noarch

How reproducible:


Steps to Reproduce:
1. Have a clean pulp install
2. Install squid, enable it, test it
3. Configure proxy for pulp
4. Restart pulp
5. Initiate a sync, watch the squid log
  
Additional info:

A bit from grinder.log:

2011-11-23 16:03:23,230 3396:140406926841600: grinder.BaseFetch:INFO: activeobject:160 Fetching 14052 bytes: symlinks-1.4-3.fc15.x86_64.rpm from http://ftp.linux.cz/pub/linux/fedora/linux/releases/15/Fedora/x86_64/os/Packages/symlinks-1.4-3.fc15.x86_64.rpm
2011-11-23 16:03:23,527 3396:140406926841600: grinder.BaseFetch:INFO: activeobject:160 Create a link in repo directory for the package at /var/lib/pulp//repos/ACME_Corporation/Locker/f15/f15_64//Packages/symlinks-1.4-3.fc15.x86_64.rpm to ../../../../../../packages/symlinks/1.4/3.fc15/x86_64/f10/symlinks-1.4-3.fc15.x86_64.rpm
2011-11-23 16:03:23,528 3396:140406926841600: grinder.ParallelFetch:INFO: ParallelFetch:138 4 threads are active. 2727 items left to be fetched
2011-11-23 16:03:23,596 3396:140406926841600: grinder.BaseFetch:INFO: activeobject:160 Creating directory: /var/lib/pulp//packages/senamirmir-washra-fonts-common/4.1/6.fc15/noarch/f42
2011-11-23 16:03:23,601 3396:140406926841600: grinder.BaseFetch:INFO: activeobject:160 Fetching 312392 bytes: senamirmir-washra-fonts-common-4.1-6.fc15.noarch.rpm from http://ftp.linux.cz/pub/linux/fedora/linux/releases/15/Fedora/x86_64/os/Packages/senamirmir-washra-fonts-common-4.1-6.fc15.noarch.rpm
2011-11-23 16:03:25,784 3396:140406926841600: grinder.BaseFetch:INFO: activeobject:160 Create a link in repo directory for the package at /var/lib/pulp//repos/ACME_Corporation/Locker/f15/f15_64//Packages/senamirmir-washra-fonts-common-4.1-6.fc15.noarch.rpm to ../../../../../../packages/senamirmir-washra-fonts-common/4.1/6.fc15/noarch/f42/senamirmir-washra-fonts-common-4.1-6.fc15.noarch.rpm
2011-11-23 16:03:25,790 3396:140406926841600: grinder.ParallelFetch:INFO: ParallelFetch:138 4 threads are active. 2726 items left to be fetched
2011-11-23 16:03:25,873 3396:140406926841600: grinder.BaseFetch:INFO: activeobject:160 Creating directory: /var/lib/pulp//packages/amarok/2.4.1/1.fc15/x86_64/f06
2011-11-23 16:03:25,881 3396:140406926841600: grinder.BaseFetch:INFO: activeobject:160 Fetching 9559892 bytes: amarok-2.4.1-1.fc15.x86_64.rpm from http://ftp.linux.cz/pub/linux/fedora/linux/releases/15/Fedora/x86_64/os/Packages/amarok-2.4.1-1.fc15.x86_64.rpm
Comment 1 Lukas Zapletal 2011-11-23 15:13:28 UTC 
Additional note - I have selinux and iptables disabled.
Comment 2 John Matthews 2011-12-09 20:40:12 UTC 
Using pulp RPMs built from master on 12/9/2011
0.0.254-1.git.5.39971e9.fc15.noarch

Changed /etc/pulp/pulp.conf
[yum]
proxy_url: http://IP_ADDRESS
proxy_port: 3128


Below is squid.log output from syncing a Pulp Fedora repo.

Access is going through Proxy as expected.


1323479608.565    313 10.210.67.63 TCP_MISS/200 3413 GET http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-15/x86_64/repodata/repomd.xml - DIRECT/85.236.55.7 text/xml
1323479608.859    289 10.210.67.63 TCP_MISS/200 11583 GET http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-15/x86_64/repodata/36074a6a66a90aa2f12826349b4aa3bde23657acb2b0f99938bf1ccf26f508b5-primary.sqlite.bz2 - DIRECT/85.236.55.7 application/x-bzip2
1323479609.168    305 10.210.67.63 TCP_MISS/200 12348 GET http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-15/x86_64/repodata/2659f11af5e6edc72c62a322fea83cbd99bf5cf8f8d131910b98e75d988a1e7e-filelists.xml.gz - DIRECT/85.236.55.7 application/x-gzip
1323479609.475    303 10.210.67.63 TCP_MISS/200 6875 GET http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-15/x86_64/repodata/24e3f2af0c255ba0119fab68738e2a8cae830abe3e8f2d97e382b32b621c3cf5-primary.xml.gz - DIRECT/85.236.55.7 application/x-gzip
1323479609.879    400 10.210.67.63 TCP_MISS/200 14125 GET http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-15/x86_64/repodata/015970537365ddc5debba717fef623f666a0ce5a94eeb8a23506124f54cc1026-other.xml.gz - DIRECT/85.236.55.7 application/x-gzip
1323479610.281    399 10.210.67.63 TCP_MISS/200 15979 GET http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-15/x86_64/repodata/3ecaf3545b3cdb03eaf23b9d2367d59c191634559dd027f8d4a4c5fb765a71d9-filelists.sqlite.bz2 - DIRECT/85.236.55.7 application/x-bzip2
1323479610.478    192 10.210.67.63 TCP_REFRESH_UNMODIFIED/200 11689 GET http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-15/x86_64/repodata/36074a6a66a90aa2f12826349b4aa3bde23657acb2b0f99938bf1ccf26f508b5-primary.sqlite.bz2 - DIRECT/85.236.55.7 application/x-bzip2
1323479610.886    390 10.210.67.63 TCP_MISS/200 17713 GET http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-15/x86_64/repodata/b131e9ad7be035840939c6a6f0efb474fa071bc03000a98be4076fdb3180e916-other.sqlite.bz2 - DIRECT/85.236.55.7 application/x-bzip2
Comment 3 John Matthews 2011-12-09 20:42:06 UTC 
Lukas would you retry and see if you still see a problem?

I tested in EC2 with SELinux enabled, the sync worked with squid logs displaying the expected traffic.
Comment 4 Lukas Zapletal 2011-12-14 14:08:11 UTC 
1323871426.462   3035 192.168.1.16 TCP_MISS/200 515794 CONNECT cdn.redhat.com:443 - DIRECT/92.122.244.251 -
1323871427.402  10272 192.168.1.16 TCP_MISS/200 2370271 CONNECT cdn.redhat.com:443 - DIRECT/92.122.244.251 -
1323871427.582  10446 192.168.1.16 TCP_MISS/200 2519359 CONNECT cdn.redhat.com:443 - DIRECT/92.122.244.251 -
1323871429.164   2864 192.168.1.16 TCP_MISS/200 592388 CONNECT cdn.redhat.com:443 - DIRECT/92.122.244.251 -
1323871430.629   2978 192.168.1.16 TCP_MISS/200 356908 CONNECT cdn.redhat.com:443 - DIRECT/92.122.244.251 -
1323871430.765   3151 192.168.1.16 TCP_MISS/200 611417 CONNECT cdn.redhat.com:443 - DIRECT/92.122.244.251 -

ACK

pulp-0.0.244-5.fc15.noarch

It had to be some misconfiguration on my side (but I'd swear I did setup it correctly) :-(
Comment 5 John Matthews 2011-12-14 14:43:39 UTC 
Lukas,

Thank you for retrying.  

QE:  I am marking this as NOTABUG.