Bug 1302072

Summary: kickstarting RHEL5 from Proxy server on RHEL6 fails
Product: Red Hat Satellite Proxy 5 Reporter: Fotios Tsiadimos <ftsiadim>
Component: ServerAssignee: Gennadii Altukhov <galtukho>
Status: CLOSED CURRENTRELEASE QA Contact: Red Hat Satellite QA List <satqe-list>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 570CC: dyordano, ftsiadim, galtukho, hartsjc, jdobes, mmello, rdrazny, risantam, tlestach
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-22 11:43:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1127217    

Description Fotios Tsiadimos 2016-01-26 17:21:58 UTC
Description of problem:
While kickstarting from proxy server we get this error in the install.log on the client:
Installing kernel-2.6.18-398.el5.x86_64
error: unpacking of archive failed on file /boot/vmlinuz-2.6.18-398.el5;56495c6d: cpio: read
 and the server fails to boot because it can not find the kernel to boot off of

Version-Release number of selected component (if applicable):


How reproducible:
Satellite Proxy 5.5, 5.6 and 5.7 installed RHEL6. Trying to kickstart a RHEL5 system it fails. Works when kickstarting a RHEL6 or RHEL7

RHN Satellite Proxy 5.6 on a RHEL5 and we were able to provision the system without any issues.

Comment 1 Tomas Lestach 2016-01-27 08:57:55 UTC
Just got an confirmation from QA, they're able to kickstart RHEL5 over RHN Proxy on RHEL6 without any issues. This does not look to be a RHN Proxy bug.

Make sure, the kernel-2.6.18-398.el5.x86_64.rpm has the right size/checksum.

Comment 4 Gennadii Altukhov 2016-02-17 09:34:03 UTC
Taking...

Comment 10 Gennadii Altukhov 2016-03-14 16:52:47 UTC
Hello, 
Yes, I can reproduce the bug and after some investigation I have:

1) There are, actually, 2 bugs:
- On the side of Proxy (squid)
- On the side of Anaconda (for RHEL 5).


2) Why it happens for RHEL 5 and doesn't for RHEL 6?
Installation of packages by Anaconda can be divided on 3 steps:

----1--- 
Downloading of metadata for a repository: repodata/repomd.xml, primary.xml.gz, comps-rhel5-vt.xml, filelists.xml.gz

----2--- 
Extract from metadata (primary.xml) information about range of RPM-header for every package, something like this:
<rpm:header-range start="440" end="19045"/>

and download RPM-header with range GET request, e.g.:

GET /ty/5cuO9OQ6/Server/kernel-2.6.18-398.el5.x86_64.rpm HTTP/1.1^M
Accept-Encoding: identity^M
Range: bytes=440-996355^M
Connection: close^M
Host: dhcp131-19.brq.redhat.com^M
User-agent: urlgrabber/3.1.0 yum/3.2.22^M


In /var/log/squid/access.log:
363:1457694721.703    417 127.0.0.1 TCP_MISS/206 996393 GET http://dhcp131-19.brq.redhat.com/ty-cksm/fc669e3a67c667843a2105e66a8c7ab7/Server/kernel-2.6.18-398.el5.x86_64.rpm - DIRECT/10.34.131.19 application/octet-stream

TCP_MISS/206 looks OK, we have no the package in the cache and this is Range-request (code 206).

----3-----

Download all packages to install them.

In config of squid we have parameter:
range_offset_limit -1 KB
It means that squid download whole file, even range request.

Actually during downloading packages Squid do not return the whole package to Anaconda after range request.
It is possible to reproduce the bug with curl
curl -v --get \
     -H 'Host: dhcp131-19.brq.redhat.com' \
     -H 'Connection: close' \
     -H 'Accept-Encoding: identity' \
     -H 'Range: bytes=440-996355' \
     -A 'urlgrabber/3.1.0 yum/3.2.22' \
     'http://dhcp131-19.brq.redhat.com/ty/5cuO9OQ6/Server/kernel-2.6.18-398.el5.x86_64.rpm' -o kernel-2.6.18-398.el5.x86_64.rpm-head

curl -v --get \
     -H 'Host: dhcp131-19.brq.redhat.com' \
     -H 'Connection: close' \
     -H 'Accept-Encoding: identity' \
     -A 'urlgrabber/3.1.0 yum/3.2.22' \
     'http://dhcp131-19.brq.redhat.com/ty/5cuO9OQ6/Server/kernel-2.6.18-398.el5.x86_64.rpm' -o kernel-2.6.18-398.el5.x86_64.rpm

You will see an error 
curl: (18) transfer closed with 20755079 bytes remaining to read

The same error in anaconda.log, but on RHEL 6 Anaconda tries to download package again, e.g.:

15:26:33,326 WARNING : Try 1/10 for http://dhcp131-19.brq.redhat.com/ty/6BGjRC8Z/Packages/kernel-firmware-2.6.32-573.el6.noarch.rpm failed: [Errno 14] PYCURL ERROR 18 - "transfer closed with 9163780 bytes remaining to read"
15:26:33,579 WARNING : Failed to get http://dhcp131-19.brq.redhat.com/ty/6BGjRC8Z/Packages/kernel-firmware-2.6.32-573.el6.noarch.rpm from mirror 1/1, or downloaded file is corrupt
15:26:33,581 WARNING : package download failure, retrying automatically

Comment 11 Gennadii Altukhov 2016-03-14 16:55:45 UTC
As a workaround you can do:
- Downgrade squid to version squid-3.1.10-29.el6.x86_64
yum downgrade squid-3.1.10-29.el6.x86_64

OR

- change squid config

remove: range_offset_limit -1 KB
add: request_header_access Range deny all

Comment 12 Gennadii Altukhov 2016-03-15 11:50:47 UTC
Anaconda Bug 1317864

Comment 13 Gennadii Altukhov 2016-03-21 11:13:43 UTC
Squid Bug 1319705

Comment 14 Tomas Lestach 2016-05-13 14:03:11 UTC
The blocking squid issue has been fixed in RHEL 6.8 (Bug 1319705) and the plan is to backport the fix to RHEL 6.7.z (Bug 1322709).

Based on that I'm switching this BZ to ON_QA.

Comment 15 Radovan Drazny 2016-05-19 13:17:06 UTC
Tested with squid-3.1.23-16.el6 containing the fix for BZ 1319705. Performed reprovisioning of RHEL 5 client via RHN Proxy on RHEL 6 server 30 times, never failed, always went through without any problem.

VERIFIED