Bug 805925 - pulp changes needed to support syncing RHEL5 CDN-backed repositories
pulp changes needed to support syncing RHEL5 CDN-backed repositories
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Content Management (Show other bugs)
Unspecified Unspecified
high Severity high (vote)
: Unspecified
: --
Assigned To: Katello Bug Bin
Og Maciel
: Triaged
Depends On: 803944
  Show dependency treegraph
Reported: 2012-03-22 09:15 EDT by James Laska
Modified: 2014-09-18 11:32 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 803944
Last Closed: 2012-08-22 14:31:44 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Comment 1 James Laska 2012-03-22 09:16:26 EDT
This bug is intended to track potential pulp code changes needed to accommodate the RHEL5 CDN layout.
Comment 5 James Laska 2012-03-26 09:20:00 EDT
On Sat, 2012-03-24 at 17:39 -0400, James Laska wrote:
> > Few suggestions,
> > 
> > * double check versions of pulp/grinder rpms running,
> See previous mail for katello and pulp versions.  "All tests are performed using pulp-1.0.0-7.el6 and katello-0.1.306-1.el6.". I didn't check the grinder version, but I assume it's on there considering -7 pulp has it as a required.  

 * pulp-1.0.0-7.el6.noarch
 * katello-0.1.306-1.el6.noarch
 * grinder-0.0.142-1.el6.noarch

> > * make sure you're pointing to the correct promoted urls.
> Good point. I believe they are, since the installs are able to
> download the vmlinuz/initrd.img/stage2.img just fine ... It's only the
> repodata discovery that fails. 
> Additionally, the rhel6 installs work fine.  I stuffed the anaconda
> logs from the failed install attempts. I'll confirm they are using the
> promoted urls. 

Using the pulp promoted RHEL-5 x86_64 installation repository, I have attempted both automated (via conductor/oz/imagefactory) and manual installations.  Both automated and manual installs fail when attempting to install packages and parse repodata.  Both installs boot with the correct installation URL.


The ks.cfg used in the automated case is fairly simple (http://pastebin.test.redhat.com/82901), and has no installation or repo definitions to cause problems.

The sha256sums of the promoted pulp kernel, initrd and anaconda stage2.img match that of RHEL-5-U8.

> d320ded10bc13b48337659116551201360ccae7fe5ead2822bfa3a7bdd29ec3b  /mnt/redhat/released/RHEL-5-Server/U8/x86_64/os/images/pxeboot/vmlinuz
> dae1ef3781f5e158699fb837512f907aac2fccb5b84b1568e0ae65c88b0c1693  /mnt/redhat/released/RHEL-5-Server/U8/x86_64/os/images/pxeboot/initrd.img
> 1758d36993052a3f04b155cc75da8443f13ea955a3442175c1cf423f62a8578c  /mnt/redhat/released/RHEL-5-Server/U8/x86_64/os/images/stage2.img

> d320ded10bc13b48337659116551201360ccae7fe5ead2822bfa3a7bdd29ec3b  redhat/Dev/content/dist/rhel/server/5/5Server/x86_64/os/images/pxeboot/vmlinuz
> dae1ef3781f5e158699fb837512f907aac2fccb5b84b1568e0ae65c88b0c1693  redhat/Dev/content/dist/rhel/server/5/5Server/x86_64/os/images/pxeboot/initrd.img
> 1758d36993052a3f04b155cc75da8443f13ea955a3442175c1cf423f62a8578c  redhat/Dev/content/dist/rhel/server/5/5Server/x86_64/os/images/stage2.img

Looking at the httpd access_log on my katello/pulp server, I see failed attempts to access the repomd.xml

> - - [26/Mar/2012:08:36:51 -0400] "GET /pulp/ks/redhat/Dev/content/dist/rhel/server/5/5Server/x86_64/os///disc1/.discinfo HTTP/1.1" 404 380 "-" "urlgrabber/3.1.0"
> - - [26/Mar/2012:08:36:54 -0400] "GET /pulp/ks/redhat/Dev/content/dist/rhel/server/5/5Server/x86_64/os//Server/repodata/repomd.xml HTTP/1.1" 404 390 "-" "urlgrabber/3.1.0 yum/3.2.22"
> - - [26/Mar/2012:08:40:24 -0400] "GET /pulp/ks/redhat/Dev/content/dist/rhel/server/5/5Server/x86_64/os/Server/repodata/repomd.xml HTTP/1.1" 404 389 "-" "urlgrabber/3.1.0 yum/3.2.22"

It seems the problem is that the installer is expecting to see "os/Server/repodata/repomd.xml" ... but no such file is found.  This file is *not* present in the 'Library' or the promoted environment 'Dev'.  It *is* present on the CDN (http://cdn.rcm-qa.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os/Server/repodata/repomd.xml).  

Manually downloading that file into the pulp directory solves the problem and allows manual and automation installations to proceed.  

This suggests pulp is not downloading the required installation file.  

I'm moving this bug back to ASSIGNED.  If you've prefer tracking any subsequent change, or if my analysis is incorrect, I'm happy to adjust the bz tracking as needed.
Comment 6 Pradeep Kilambi 2012-03-26 09:23:41 EDT
This is not a pulp iaaue. The treeinfo is missing repomd.xml reference.moving back to modified
Comment 7 Mike McCune 2012-04-10 17:58:46 EDT
This was resolved in:

Comment 8 Mike McCune 2012-04-10 18:00:06 EDT
should have been more clear with my previous comment, this should get retested with the latest Beta6 puddle and upstream changes in the CDN to see if it is an issue.
Comment 9 Og Maciel 2012-04-17 19:56:11 EDT
# ls -ltd /var/www/pub/ks/Raleigh/Dev1/content/dist/rhel/server/5/5Server/x86_64/os/*
drwxr-xr-x. 2 apache apache    4096 Apr 17 19:17 /var/www/pub/ks/Raleigh/Dev1/content/dist/rhel/server/5/5Server/x86_64/os/repodata
-rw-r--r--. 1 apache apache 9987985 Apr 17 19:17 /var/www/pub/ks/Raleigh/Dev1/content/dist/rhel/server/5/5Server/x86_64/os/updateinfo.xml
drwxr-xr-x. 3 apache apache    4096 Apr 17 18:12 /var/www/pub/ks/Raleigh/Dev1/content/dist/rhel/server/5/5Server/x86_64/os/Server
drwxr-xr-x. 3 apache apache    4096 Apr 17 18:12 /var/www/pub/ks/Raleigh/Dev1/content/dist/rhel/server/5/5Server/x86_64/os/images
lrwxrwxrwx. 1 apache apache     106 Apr 17 18:12 /var/www/pub/ks/Raleigh/Dev1/content/dist/rhel/server/5/5Server/x86_64/os/treeinfo -> ../../../../../../../../../../../distributions/ks-Red Hat Enterprise Linux Server-None-5.8-x86_64/treeinfo
drwxr-xr-x. 2 apache apache    4096 Apr 17 18:12 /var/www/pub/ks/Raleigh/Dev1/content/dist/rhel/server/5/5Server/x86_64/os/drpms
drwxr-xr-x. 2 apache apache 1253376 Apr 17 18:11 /var/www/pub/ks/Raleigh/Dev1/content/dist/rhel/server/5/5Server/x86_64/os/Packages
Comment 10 James Laska 2012-04-18 07:44:44 EDT
Just some extra data points ...

I have successfully synced RHEL 5.7, 5.8 and 5Server (i386 and x86_64) from cdn.rcm-qa.redhat.com using pulp-1.0.4-1.el6.noarch

I have not yet tested against cdn.redhat.com.  I'm told that some of the most recent CDN layout changes (5.7 i386 specifically) are not yet on cdn.redhat.com.
Comment 11 Og Maciel 2012-04-18 10:27:43 EDT
Comment #9 was syncing 5Server using cdn.redhat.com... should I close this?
Comment 12 James Laska 2012-04-18 10:42:01 EDT
(In reply to comment #11)
> Comment #9 was syncing 5Server using cdn.redhat.com... should I close this?

For completeness, if we are able to successfully build and push a CloudForms Cloud Engine image using your cdn.redhat.com repository, that will touch all the pain points we've encountered with RHEL5 CDN URLs
Comment 13 Og Maciel 2012-04-18 14:14:02 EDT
Downloaded template and imported into CF system. Then built image and launched.
Comment 14 Og Maciel 2012-04-18 14:16:01 EDT
* candlepin-0.5.26-1.el6.noarch
* candlepin-tomcat6-0.5.26-1.el6.noarch
* katello-0.1.309-1.el6.noarch
* katello-all-0.1.309-1.el6.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.0.4-1.el6.noarch
* katello-cli-0.1.107-1.el6.noarch
* katello-cli-common-0.1.107-1.el6.noarch
* katello-common-0.1.309-1.el6.noarch
* katello-configure-0.1.107-1.el6.noarch
* katello-glue-candlepin-0.1.309-1.el6.noarch
* katello-glue-foreman-0.1.309-1.el6.noarch
* katello-glue-pulp-0.1.309-1.el6.noarch
* katello-qpid-broker-key-pair-1.0-1.noarch
* katello-qpid-client-key-pair-1.0-1.noarch
* katello-selinux-0.1.10-1.el6.noarch
* pulp-1.0.4-1.el6.noarch
* pulp-common-1.0.4-1.el6.noarch
* pulp-selinux-server-1.0.4-1.el6.noarch

Note You need to log in before you can comment on or make changes to this bug.