Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
DescriptionMasahiro Matsuya
2018-06-04 07:51:27 UTC
Description of problem:
A customer tried to sync a yum repository into local directory with reposync command. The repository on the CDN server of Customer Portal is configured with subscription-manager, and a proxy is configured for that. So, any contents are received via the proxy. The proxy is configured properly, but reposync output the following error repeatedly. Even when the same reposync command is exeucted again, the same error is output. It could not get the correct package.
js-1.8.5-19.el7.x86_64.rpm FAILED
After some investigation, I found that the truncating an existing file in local directory with receiving "HTTP/1.0 200 Connection established" message from the proxy caused the problem.
When an incomplete file exists in the local repository accidentally, reposync tries to get the remaining content only with HTTP Range request. When the proxy is configured, HTTP CONNECT request is used to have the proxy establish with the CDN server on Customer Portal. When it's established successfully, a message "HTTP/1.0 200 Connection established" is reported back from the proxy. urlgrabber truncates the local file upon the message with the following code.
/usr/lib/python2.7/site-packages/urlgrabber/grabber.py
def _hdr_retrieve(self, buf):
...
elif self.append and self._hdr_dump == '' and ' 200 ' in buf:
# reget was attempted but server sends it all
# undo what we did in _build_range()
self.append = False
self.reget_time = None
self._amount_read = 0
self._reget_length = 0
self.fo.truncate(0)
Just the number 200 is checked, assuming that it's "200 OK".
The message "200 OK" means that it got a whole of the file, so truncating it is reasonable. But, in this case, it's trying to get the remaining (partial) contents, and "206 Partial Contents" HTTP reply should be given for a HTTP GET request. urlgrabber just should append the given data into the existing file. But, it truncated the file upon the HTTP reply "HTTP/1.0 200 Connection established" from the proxy. After the truncation, the downloaded partial content is written into the file. But, of course, the rpm file verification fails because it's a partial content. As a result, "FAILED" message in above was output.
I created a very simple patch for this problem, and the problem was resolved.
I will attach it on this bugzilla.
Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux 7
How reproducible:
Always
Steps to Reproduce:
1. Prepare any yum repository. Or just use a repository on customer portal with subscription-manager.
2. Built a proxy server with squid between the client and the yum repository server.
3. Configure the client to use proxy.
4. Run reposync successfully with a yum repository. For example:
# reposync -l -p /data/rhel7 --repoid=rhel-7-server-rpms --downloadcomps --download-metadata -n
5. To emulate a partial content in local directory, use dd command.
For example:
# cd (the local directory including synced packages)
# mv js-1.8.5-20.el7.x86_64.rpm /root
# dd if=/root/js-1.8.5-20.el7.x86_64.rpm of=js-1.8.5-20.el7.x86_64.rpm bs=50000 count=1
6. Run reposync command again
Actual results:
FAILED messages is output repeatedly and it never get the correct package file.
Expected results:
The remaining date received as a partial content is appended into the existing file in local directory, and the correct package file is stored.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2020:1123
Description of problem: A customer tried to sync a yum repository into local directory with reposync command. The repository on the CDN server of Customer Portal is configured with subscription-manager, and a proxy is configured for that. So, any contents are received via the proxy. The proxy is configured properly, but reposync output the following error repeatedly. Even when the same reposync command is exeucted again, the same error is output. It could not get the correct package. js-1.8.5-19.el7.x86_64.rpm FAILED After some investigation, I found that the truncating an existing file in local directory with receiving "HTTP/1.0 200 Connection established" message from the proxy caused the problem. When an incomplete file exists in the local repository accidentally, reposync tries to get the remaining content only with HTTP Range request. When the proxy is configured, HTTP CONNECT request is used to have the proxy establish with the CDN server on Customer Portal. When it's established successfully, a message "HTTP/1.0 200 Connection established" is reported back from the proxy. urlgrabber truncates the local file upon the message with the following code. /usr/lib/python2.7/site-packages/urlgrabber/grabber.py def _hdr_retrieve(self, buf): ... elif self.append and self._hdr_dump == '' and ' 200 ' in buf: # reget was attempted but server sends it all # undo what we did in _build_range() self.append = False self.reget_time = None self._amount_read = 0 self._reget_length = 0 self.fo.truncate(0) Just the number 200 is checked, assuming that it's "200 OK". The message "200 OK" means that it got a whole of the file, so truncating it is reasonable. But, in this case, it's trying to get the remaining (partial) contents, and "206 Partial Contents" HTTP reply should be given for a HTTP GET request. urlgrabber just should append the given data into the existing file. But, it truncated the file upon the HTTP reply "HTTP/1.0 200 Connection established" from the proxy. After the truncation, the downloaded partial content is written into the file. But, of course, the rpm file verification fails because it's a partial content. As a result, "FAILED" message in above was output. I created a very simple patch for this problem, and the problem was resolved. I will attach it on this bugzilla. Version-Release number of selected component (if applicable): Red Hat Enterprise Linux 7 How reproducible: Always Steps to Reproduce: 1. Prepare any yum repository. Or just use a repository on customer portal with subscription-manager. 2. Built a proxy server with squid between the client and the yum repository server. 3. Configure the client to use proxy. 4. Run reposync successfully with a yum repository. For example: # reposync -l -p /data/rhel7 --repoid=rhel-7-server-rpms --downloadcomps --download-metadata -n 5. To emulate a partial content in local directory, use dd command. For example: # cd (the local directory including synced packages) # mv js-1.8.5-20.el7.x86_64.rpm /root # dd if=/root/js-1.8.5-20.el7.x86_64.rpm of=js-1.8.5-20.el7.x86_64.rpm bs=50000 count=1 6. Run reposync command again Actual results: FAILED messages is output repeatedly and it never get the correct package file. Expected results: The remaining date received as a partial content is appended into the existing file in local directory, and the correct package file is stored.