Bug 1175332 - yum-presto can't find the drpms, delta rpm files
Summary: yum-presto can't find the drpms, delta rpm files
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Pulp
Classification: Retired
Component: rpm-support
Version: 2.4.3
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: 2.6.1
Assignee: Barnaby Court
QA Contact: pulp-qe-list
URL:
Whiteboard:
: 1175331 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-12-17 14:02 UTC by ronald
Modified: 2015-02-28 23:16 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-28 23:16:48 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Pulp Redmine 644 0 None None None Never

Internal Links: 1248232

Description ronald 2014-12-17 14:02:23 UTC
Description of problem:
The drpms files are in the wrong sub-directory after a sync and publish .

Version-Release number of selected component (if applicable):
pulp 2.4.3 Product https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.4
Centos 6.5/6.6

How reproducible:
create,sync and publish centos feed ie  
http://ftp.halifax.rwth-aachen.de/centos/6.6/updates/x86_64/

Steps to Reproduce:
1.pulp-admin rpm repo create  --repo-id centos66updates --serve-http=true --serve-https=true  --feed  http://ftp.halifax.rwth-aachen.de/centos/6.6/updates/x86_64/
2. pulp-admin rpm repo sync run --bg --repo-id centos66updates
3. pulp-admin rpm repo publish run --bg  --repo-id centos66updates
4. yum install yum-presto

pulp-consumer rpm bind --repo-id  centos66updates

yum --disablerepo=* --enablerepo=centos66updates,centos65updates   update 

Actual results:

Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Download delta size: 8.1 M
https://192.168.40.55/pulp/repos/centos/6.5/updates/x86_64/drpms/cpio-2.10-11.el6_3_2.10-12.el6_5.x86_64.drpm: [Errno 14] PYCURL ERROR 22 - "NSS: client certificate not found (nickname not specified)"
Trying other mirror.
https://192.168.40.55/pulp/repos/centos/6.6/updates/x86_64/drpms/curl-7.19.7-37.el6_4_7.19.7-40.el6_6.3.x86_64.drpm: [Errno 14] PYCURL ERROR 22 - "NSS: client certificate not found (nickname not specified)"
Trying other mirror.
https://192.168.40.55/pulp/repos/centos/6.6/updates/x86_64/drpms/cyrus-sasl-2.1.23-13.el6_3.1_2.1.23-15.el6_6.1.x86_64.drpm: [Errno 14] PYCURL ERROR 22 - "NSS: client certificate not found (nickname not specified)"
Trying other mirror.
https://192.168.40.55/pulp/repos/centos/6.6/updates/x86_64/drpms/cyrus-sasl-lib-2.1.23-13.el6_3.1_2.1.23-15.el6_6.1.x86_64.drpm: [Errno 14] PYCURL ERROR 22 - "NSS: client certificate not found (nickname not specified)"
Trying other mirror.
https://192.168.40.55/pulp/repos/centos/6.6/updates/x86_64/drpms/device-mapper-1.02.79-8.el6_1.02.90-2.el6_6.1.x86_64.drpm: [Errno 14] PYCURL ERROR 22 - "NSS: client certificate not found (nickname not specified)"
Trying other mirror.
https://192.168.40.55/pulp/repos/centos/6.6/updates/x86_64/drpms/device-mapper-libs-1.02.79-8.el6_1.02.90-2.el6_6.1.x86_64.drpm: [Errno 14] PYCURL ERROR 22 - "NSS: client certificate not found (nickname not specified)"
Trying other mirror.
https://192.168.40.55/pulp/repos/centos/6.6/updates/x86_64/drpms/iproute-2.6.32-31.el6_2.6.32-33.el6_6.x86_64.drpm: [Errno 14] PYCURL ERROR 22 - "NSS: client certificate not found (nickname not specified)"
Trying other mirror.
..
..
..

Expected results:
yum to find the delta rpms in the expected directory


Additional info:
The feed centos feed has 3 subdirs, ie http://ftp.tudelft.nl/centos.org/6.5/updates/x86_64/:
Packages
drpms
repodata

The pulp repo-mirror has 2 subdirs:
drpms/drpms
repodata

yum presto can't find the delta rpm files in the subdir pulp/repos/centos.org/6.5/updates/x86_64/drpms/drpms
because yum is searching in pulp/repos/centos.org/6.5/updates/x86_64/drpms

Comment 1 ronald 2014-12-17 14:14:35 UTC
*** Bug 1175331 has been marked as a duplicate of this bug. ***

Comment 2 Sayli Karmarkar 2014-12-22 16:55:28 UTC
This does not look like a Pulp issue. Please re-open if you still see the issue after fixing SSL errors. It seems that yum is not able to establish connection with your host.

Comment 3 ronald 2015-01-12 12:52:55 UTC
The https connection is not the problem.
The pulpserver returns to yum  the file 
https://192.168.40.55/pulp/repos/centos/6.5/updates/x86_64/drpms/cpio-2.10-11.el6_3_2.10-12.el6_5.x86_64.drpm

and cannot find it because pulp dumps the file in:/pulp/repos/centos/6.5/updates/x86_64/drpms/drpms/
so the pulpserver should return to yum https://192.168.40.55/pulp/repos/centos/6.5/updates/x86_64/drpms/drpms/cpio-2.10-11.el6_3_2.10-12.el6_5.x86_64.drpm

Comment 4 ronald 2015-01-12 12:56:44 UTC
(In reply to Sayli Karmarkar from comment #2)
> This does not look like a Pulp issue. Please re-open if you still see the
> issue after fixing SSL errors. It seems that yum is not able to establish
> connection with your host.

The https connection is not the problem. It can connect over https.

Comment 5 Sayli Karmarkar 2015-01-13 00:51:38 UTC
This indeed seems to be the problem. Pulp is creating another redundant subdirectory when storing and publishing drpms. 

$ sudo ls -l /var/lib/pulp/published/yum/master/yum_distributor/test-drpm/1421108941.15/drpms/drpms
total 72
lrwxrwxrwx. 1 apache apache 85 Jan 12 16:29 yum-3.2.29-20.fc16_from_el6_3.4.3-8.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.2.29-20.fc16_from_el6_3.4.3-8.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 76 Jan 12 16:29 yum-3.4.3-10.fc16_3.4.3-11.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.4.3-10.fc16_3.4.3-11.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 76 Jan 12 16:29 yum-3.4.3-11.fc16_3.4.3-16.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.4.3-11.fc16_3.4.3-16.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 76 Jan 12 16:29 yum-3.4.3-16.fc16_3.4.3-18.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.4.3-16.fc16_3.4.3-18.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 76 Jan 12 16:29 yum-3.4.3-18.fc16_3.4.3-19.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.4.3-18.fc16_3.4.3-19.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 76 Jan 12 16:29 yum-3.4.3-19.fc16_3.4.3-20.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.4.3-19.fc16_3.4.3-20.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 76 Jan 12 16:29 yum-3.4.3-20.fc16_3.4.3-22.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.4.3-20.fc16_3.4.3-22.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 74 Jan 12 16:29 yum-3.4.3-8.fc16_3.4.3-9.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.4.3-8.fc16_3.4.3-9.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 75 Jan 12 16:29 yum-3.4.3-9.fc16_3.4.3-10.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-3.4.3-9.fc16_3.4.3-10.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 90 Jan 12 16:29 yum-cron-3.2.29-20.fc16_from_el6_3.4.3-8.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.2.29-20.fc16_from_el6_3.4.3-8.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 81 Jan 12 16:29 yum-cron-3.4.3-10.fc16_3.4.3-11.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.4.3-10.fc16_3.4.3-11.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 81 Jan 12 16:29 yum-cron-3.4.3-11.fc16_3.4.3-16.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.4.3-11.fc16_3.4.3-16.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 81 Jan 12 16:29 yum-cron-3.4.3-16.fc16_3.4.3-18.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.4.3-16.fc16_3.4.3-18.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 81 Jan 12 16:29 yum-cron-3.4.3-18.fc16_3.4.3-19.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.4.3-18.fc16_3.4.3-19.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 81 Jan 12 16:29 yum-cron-3.4.3-19.fc16_3.4.3-20.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.4.3-19.fc16_3.4.3-20.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 81 Jan 12 16:29 yum-cron-3.4.3-20.fc16_3.4.3-22.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.4.3-20.fc16_3.4.3-22.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 79 Jan 12 16:29 yum-cron-3.4.3-8.fc16_3.4.3-9.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.4.3-8.fc16_3.4.3-9.fc16.noarch.drpm
lrwxrwxrwx. 1 apache apache 80 Jan 12 16:29 yum-cron-3.4.3-9.fc16_3.4.3-10.fc16.noarch.drpm -> /var/lib/pulp/content/drpm/drpms/yum-cron-3.4.3-9.fc16_3.4.3-10.fc16.noarch.drpm


With some basic debugging, problem seems to be here - https://github.com/pulp/pulp_rpm/blob/master/plugins/pulp_rpm/plugins/distributors/yum/publish.py#L516. 

Setting severity to high since we definitely should fix this.

Comment 6 Barnaby Court 2015-02-09 17:42:19 UTC
https://github.com/pulp/pulp_rpm/pull/646

Comment 7 Brian Bouterse 2015-02-28 23:16:48 UTC
Moved to https://pulp.plan.io/issues/644


Note You need to log in before you can comment on or make changes to this bug.