Hide Forgot
Description of problem: The setup has 4 nodes, (rhui-manager, rhua, cds1,cds2) The proxy server uses port 3128 rhui-manager node => No firewall rules to block 443 port used. rhua node => Firewall rule set to block OUTBOUND 443 port. cds1 and cds2 => none The pulp.log file ouput 2011-07-15 20:57:47,615 23289:139870947231504: pulp.server.webservices.controllers.repositories:INFO: repositories:600 sync timeout passed : None 2011-07-15 20:57:47,640 23289:139870686742288: pulp.server.api.repo_sync:INFO: repo_sync:212 Sync invoked for repo <rhui-1.2-5Server-x86_64> 2011-07-15 20:57:47,644 23289:139870686742288: pulp.server.api.repo_sync:INFO: repo_sync:222 Sync of rhui-1.2-5Server-x86_64 starting, skip_dict = {} 2011-07-15 20:57:47,646 23289:139870686742288: pulp.server.api.synchronizers:INFO: synchronizers:390 cacert = </etc/pki/content/rhui-1.2-5Server-x86_64/feed-rhui-1.2-5Server-x86_64.ca>, cert = </etc/pki/content/rhui-1.2-5Server-x86_64/feed-rhui-1.2-5Server-x86_64.cert> 2011-07-15 20:57:51,429 23289:139870686742288: pulp.server.tasking.task:ERROR: task:380 Task failed: Task fe81254f-aef6-11e0-a3a1-5452002a3cef: _sync(rhui-1.2-5Server-x86_64, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f36285cb6d0>, skip_dict={}, max_speed=None, threads=None, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f36285cb590>>) Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 329, in run result = self.callable(*self.args, **self.kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 225, in _sync progress_callback, synchronizer, max_speed, threads) File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 303, in fetch_content progress_callback, max_speed, threads) File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 429, in sync report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback) File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 459, in fetchYumRepo self.yumFetch.getRepoData() File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 138, in getRepoData for ftype in self.getRepoXmlFileTypes(): File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 123, in getRepoXmlFileTypes return self.repo.repoXML.fileTypes() File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1417, in <lambda> repoXML = property(fget=lambda self: self._getRepoXML(), File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1413, in _getRepoXML raise Errors.RepoError, msg RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again Version-Release number of selected component (if applicable): pulp 238 , rhui -2.0.38 How reproducible: Using proxy for RHUA. NOTE: No proxies used on RHUI-Manager as its not supported yet. Steps to Reproduce: 1. Provide proxy details in answers_file 2. generate the rhua-config-rpms and install them, which brings in proxy details for pulp.conf 3. Actual results: RHUA fails to sync/fetch repos via proxy Expected results: RHUA should be able to fetch repos via proxy. Additional info:
wget access via proxy works from the rhua. [root@dhcp201-206 ~]# export http_proxy=http://dhcp201-180.englab.pnq.redhat.com:3128 [root@dhcp201-206 ~]# wget --proxy=on http://download.fedora.redhat.com/pub/epel/beta/6/i386/epel-release-6-5.noarch.rpm --2011-07-15 20:32:09-- http://download.fedora.redhat.com/pub/epel/beta/6/i386/epel-release-6-5.noarch.rpm Resolving dhcp201-180.englab.pnq.redhat.com... 10.65.201.180 Connecting to dhcp201-180.englab.pnq.redhat.com|10.65.201.180|:3128... connected. Proxy request sent, awaiting response... 200 OK Length: 14064 (14K) [application/x-rpm] Saving to: “epel-release-6-5.noarch.rpm” 100%[=======================================================================================================================================>] 14,064 --.-K/s in 0s 2011-07-15 20:32:09 (788 MB/s) - “epel-release-6-5.noarch.rpm” saved [14064/14064] The proxy entries in the pulp.conf file are as below, [root@dhcp201-206 ~]# cat /etc/pulp/pulp.conf | grep -i "^proxy" proxy_url = https://dhcp201-180.englab.pnq.redhat.com/ proxy_port = 3128 Please let me know if anything else needs to be tried.
From the above comment got to know that port 443 shouldn't be blocked. So now, grinder command used , grinder yum -U https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/ --cacert=/root/cdn.redhat.com-chain.crt --cert=/root/rhui-amazon.pem --label=/root/rhui123 --proxy_url=http://dhcp201-proxy.englab.pnq.redhat.com --proxy_port=3128 1) grinder works via proxy when 443 port is blocked and proxy specified. (On proxy node /var/log/squid/access.log does get updated - To verify whether traffic is being sent via proxy server) 2) grinder again works via proxy when 443 port is not blocked and proxy specified. (On proxy node /var/log/squid/access.log does get updated - To verify whether traffic is being sent via proxy server) pulp commands used, a) pulp-admin -u admin -p admin repo create --id=rhui123_proxy --feed=https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/ --feed_cert=/root/rhui-amazon.pem --feed_ca=/root/cdn.redhat.com-chain.crt b) pulp-admin -u admin -p admin repo sync --id=rhui123_proxy -F c) NOTE: The below entry is added by rhua-config-rpm in /etc/pulp/pulp.conf file on installing. [root@dhcp201-pulp ~]# cat /etc/pulp/pulp.conf | grep -i "^proxy" proxy_url = https://dhcp201-proxy.englab.pnq.redhat.com proxy_port = 3128 d)Also, tried with proxy_url = http://dhcp201-proxy.englab.pnq.redhat.com proxy_port = 3128 1) pulp doesn't fetch via proxy when port 443 is not blocked and proxy specified like c) and d) above in /etc/pulp/pulp.conf (On proxy node /var/log/squid/access.log does not get updates at all - To verify whether traffic is being sent via proxy server)
Fixed in Pulp .211
verified with 211 build. Now traffic does pass via the proxy server. 1311078088.260 1129 10.65.201.206 TCP_MISS/200 992 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078090.200 1938 10.65.201.206 TCP_MISS/200 14905 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078092.176 1953 10.65.201.206 TCP_MISS/200 14905 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078094.311 2126 10.65.201.206 TCP_MISS/200 14942 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078095.974 1660 10.65.201.206 TCP_MISS/200 14905 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078097.930 1942 10.65.201.206 TCP_MISS/200 14905 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078102.764 2867 10.65.201.206 TCP_MISS/200 42803 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078103.458 3563 10.65.201.206 TCP_MISS/200 116283 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078103.579 3811 10.65.201.206 TCP_MISS/200 102449 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078103.935 4182 10.65.201.206 TCP_MISS/200 89431 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078104.271 4377 10.65.201.206 TCP_MISS/200 109809 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078104.599 4811 10.65.201.206 TCP_MISS/200 51272 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078105.269 5372 10.65.201.206 TCP_MISS/200 69773 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078105.699 5802 10.65.201.206 TCP_MISS/200 101073 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078106.709 6941 10.65.201.206 TCP_MISS/200 829364 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311078109.328 9480 10.65.201.206 TCP_MISS/200 195422 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 -
grinder can download via proxy , but pulp is failing via proxy. [root@dhcp201-159 ~]# grinder yum -U https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/ --cacert=/etc/rhui/cdn.redhat.com-chain.crt --cert=/root/gen_certs/rhui-amazon.pem --label=/root/rhui1234 --proxy_url=http://dhcp201-180.englab.pnq.redhat.com --proxy_port=3128 grinder.RepoFetch: INFO fetchYumRepo() basepath = ./ grinder.RepoFetch: INFO Fetching repo metadata... grinder.RepoFetch: INFO Determining downloadable Content bits... grinder.RepoFetch: INFO 13 packages have been marked to be fetched grinder.RepoFetch: INFO Preparing to fetch any available trees.. grinder.BaseFetch: INFO Fetching Unknown bytes: .treeinfo from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os//.treeinfo grinder.BaseFetch: WARNING Retrying fetch of: .treeinfo with 1 retry attempts left. grinder.BaseFetch: INFO Fetching Unknown bytes: .treeinfo from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os//.treeinfo grinder.BaseFetch: WARNING Retrying fetch of: .treeinfo with 0 retry attempts left. grinder.BaseFetch: INFO Fetching Unknown bytes: .treeinfo from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os//.treeinfo grinder.BaseFetch: WARNING ERROR: Response = 404 fetching https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os//.treeinfo. grinder.BaseFetch: INFO Fetching Unknown bytes: treeinfo from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os//treeinfo grinder.BaseFetch: WARNING Retrying fetch of: treeinfo with 1 retry attempts left. grinder.BaseFetch: INFO Fetching Unknown bytes: treeinfo from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os//treeinfo grinder.BaseFetch: WARNING Retrying fetch of: treeinfo with 0 retry attempts left. grinder.BaseFetch: INFO Fetching Unknown bytes: treeinfo from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os//treeinfo grinder.BaseFetch: WARNING ERROR: Response = 404 fetching https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os//treeinfo. grinder.RepoFetch: INFO Unable to read the tree info config. grinder.BaseFetch: INFO Fetching 36143 bytes: rh-cds-0.27-1.el5_5.noarch.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/rh-cds-0.27-1.el5_5.noarch.rpm grinder.BaseFetch: INFO Fetching 94512 bytes: grinder-0.0.57-1.el5.noarch.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/grinder-0.0.57-1.el5.noarch.rpm grinder.BaseFetch: INFO Fetching 54614 bytes: libyaml-0.1.2-3.el5.x86_64.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/libyaml-0.1.2-3.el5.x86_64.rpm grinder.BaseFetch: INFO Fetching 74190 bytes: python-pycurl-7.15.5.1-4.el5.x86_64.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/python-pycurl-7.15.5.1-4.el5.x86_64.rpm grinder.BaseFetch: INFO Fetching 1290719 bytes: httpd-2.2.3-43.el5_5.3.x86_64.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/httpd-2.2.3-43.el5_5.3.x86_64.rpm grinder.ParallelFetch: INFO 5 threads are active. 13 items left to be fetched grinder.BaseFetch: INFO Fetching 85775 bytes: rh-rhua-0.91-1.el5_5.noarch.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/rh-rhua-0.91-1.el5_5.noarch.rpm grinder.ParallelFetch: INFO 5 threads are active. 12 items left to be fetched grinder.BaseFetch: INFO Fetching 308686 bytes: rpm-build-4.4.2.3-20.el5_5.1.x86_64.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/rpm-build-4.4.2.3-20.el5_5.1.x86_64.rpm grinder.ParallelFetch: INFO 5 threads are active. 11 items left to be fetched grinder.BaseFetch: INFO Fetching 27697 bytes: python-hashlib-20081119-5.el5.x86_64.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/python-hashlib-20081119-5.el5.x86_64.rpm grinder.ParallelFetch: INFO 5 threads are active. 10 items left to be fetched grinder.BaseFetch: INFO Fetching 100913 bytes: rh-rhui-tools-0.76-1.el5_5.noarch.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/rh-rhui-tools-0.76-1.el5_5.noarch.rpm grinder.ParallelFetch: INFO 5 threads are active. 9 items left to be fetched grinder.BaseFetch: INFO Fetching 811728 bytes: mod_python-3.3.1-12.el5.x86_64.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/mod_python-3.3.1-12.el5.x86_64.rpm grinder.ParallelFetch: INFO 5 threads are active. 8 items left to be fetched grinder.BaseFetch: INFO Fetching 179839 bytes: PyYAML-3.08-4.el5.x86_64.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/PyYAML-3.08-4.el5.x86_64.rpm grinder.ParallelFetch: INFO 5 threads are active. 7 items left to be fetched grinder.BaseFetch: INFO Fetching 506997 bytes: m2crypto-0.16-6.1.el5_5.1.x86_64.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/m2crypto-0.16-6.1.el5_5.1.x86_64.rpm grinder.ParallelFetch: INFO 5 threads are active. 6 items left to be fetched grinder.BaseFetch: INFO Fetching 87158 bytes: rh-rhua-0.95-1.el5_5.noarch.rpm from https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/rh-rhua-0.95-1.el5_5.noarch.rpm grinder.ParallelFetch: INFO 5 threads are active. 5 items left to be fetched grinder.ParallelFetch: INFO 4 threads are active. 4 items left to be fetched grinder.ParallelFetch: INFO 3 threads are active. 3 items left to be fetched grinder.ParallelFetch: INFO 2 threads are active. 2 items left to be fetched grinder.ParallelFetch: INFO 1 threads are active. 1 items left to be fetched grinder.ParallelFetch: INFO All threads have finished. grinder.ParallelFetch: INFO ParallelFetch: 13 items successfully processed, 13 downloaded, 0 items had errors grinder.ParallelFetch: INFO Transferred [3658971] bytes of [rpm] grinder.ParallelFetch: INFO Transferred [3658971] total bytes in 42.6798038483 seconds grinder.RepoFetch: INFO Finalizing metadata, moving /root/rhui1234/repodata.new to /root/rhui1234/repodata grinder.RepoFetch: INFO Processed <13> items in [42] seconds grinder.RepoFetch: INFO Cleaning any orphaned packages.. /var/log/squid/access.log 43 - DIRECT/173.222.100.251 - 1311876214.362 12414 10.65.201.159 TCP_MISS/200 523630 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311876215.889 22625 10.65.201.159 TCP_MISS/200 1309871 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - 1311876216.023 42013 10.65.201.159 TCP_MISS/200 68496 CONNECT cdn.redhat.com:443 - DIRECT/173.222.100.251 - VIA PULP [root@dhcp201-159 ~]# pulp-admin -u admin -p admin repo create --id=test_proxy_sync1 --feed=https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os/ --feed_cert=/root/gen_certs/rhui-amazon.pem --feed_ca=/etc/rhui/cdn.redhat.com-chain.crt Successfully created repository [ test_proxy_sync1 ] [root@dhcp201-159 ~]# pulp-admin -u admin -p admin repo sync --id=test_proxy_sync1 -F Sync for repository test_proxy_sync1 started Sync: Error Item Details: RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again 2011-07-28 23:23:59,054 3978:139664386148112: pulp.server.api.repo:INFO: repo:467 Delete API call invoked test_proxy_sync1 2011-07-28 23:23:59,058 3978:139664386148112: pulp.server.api.keystore:INFO: keystore:200 mkdir: /var/lib/pulp/published/gpg/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os 2011-07-28 23:23:59,059 3978:139664386148112: pulp.server.api.keystore:INFO: keystore:185 unlinking: /var/lib/pulp/published/gpg/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os 2011-07-28 23:23:59,148 3978:139664386148112: pulp.server.api.repo:INFO: repo:570 removing repo files .... /var/lib/pulp//repos/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os 2011-07-28 23:23:59,149 3978:139664386148112: pulp.repo_auth.repo_cert_utils:INFO: repo_cert_utils:87 Deleting certificate bundles at [/etc/pki/content/test_proxy_sync1] 2011-07-28 23:24:17,112 3978:139664530855696: pulp.repo_auth.repo_cert_utils:INFO: repo_cert_utils:348 Storing repo cert file [/etc/pki/content/test_proxy_sync1/feed-test_proxy_sync1.ca] 2011-07-28 23:24:17,113 3978:139664530855696: pulp.repo_auth.repo_cert_utils:INFO: repo_cert_utils:348 Storing repo cert file [/etc/pki/content/test_proxy_sync1/feed-test_proxy_sync1.cert] 2011-07-28 23:24:17,116 3978:139664530855696: pulp.server.util:INFO: util:362 started repo metadata update 2011-07-28 23:24:20,097 3978:139664530855696: pulp.server.util:INFO: util:368 [createrepo --database --checksum sha256 --update /var/lib/pulp//repos/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os ] on /var/lib/pulp//repos/content/dist/rhel/rhui/server/5Server/x86_64/rhui/1.2/os finished 2011-07-28 23:24:20,097 3978:139664530855696: pulp.server.util:INFO: util:385 Noting further to check; we got our fresh metadata 2011-07-28 23:24:28,593 3978:139664386148112: pulp.server.webservices.controllers.repositories:INFO: repositories:600 sync timeout passed : None 2011-07-28 23:24:28,650 3978:139664270366480: pulp.server.api.repo_sync:INFO: repo_sync:212 Sync invoked for repo <test_proxy_sync1> 2011-07-28 23:24:28,651 3978:139664270366480: pulp.server.api.repo_sync:INFO: repo_sync:222 Sync of test_proxy_sync1 starting, skip_dict = {} 2011-07-28 23:24:28,653 3978:139664270366480: pulp.server.api.synchronizers:INFO: synchronizers:390 cacert = </etc/pki/content/test_proxy_sync1/feed-test_proxy_sync1.ca>, cert = </etc/pki/content/test_proxy_sync1/feed-test_proxy_sync1.cert> 2011-07-28 23:24:29,328 3978:139664270366480: pulp.server.tasking.task:ERROR: task:380 Task failed: Task a3aa7035-b942-11e0-9a73-5452000007d0: _sync(test_proxy_sync1, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f06185e4b10>, skip_dict={}, max_speed=None, threads=None, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f06185e4b50>>) Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 329, in run result = self.callable(*self.args, **self.kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 225, in _sync progress_callback, synchronizer, max_speed, threads) File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 303, in fetch_content progress_callback, max_speed, threads) File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 431, in sync report = self.yum_repo_grinder.fetchYumRepo(store_path, callback=progress_callback) File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 459, in fetchYumRepo self.yumFetch.getRepoData() File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 138, in getRepoData for ftype in self.getRepoXmlFileTypes(): File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 123, in getRepoXmlFileTypes return self.repo.repoXML.fileTypes() File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1417, in <lambda> repoXML = property(fget=lambda self: self._getRepoXML(), File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1413, in _getRepoXML raise Errors.RepoError, msg RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again
The issue was that the rhua-config-rpms adds proxy entries to the /etc/pulp/pulp.conf file, during installation. NOTE: This addition brings in a trailing /, which needs to be removed. This entry has to be, proxy_url = https://dhcp201-159.englab.pnq.redhat.com and not proxy_url = https://dhcp201-159.englab.pnq.redhat.com/ Bug : 726420
moving to release pending
closing out, product released