Bug 1303982

Summary: PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
Product: Red Hat Enterprise Linux 6 Reporter: Jaroslav Škarvada <jskarvad>
Component: redhat-upgrade-toolAssignee: Michal Bocek <mbocek>
Status: CLOSED ERRATA QA Contact: Release Test Team <release-test-team>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.8CC: jhornice, jskarvad, mganisin, ovasik, phracek, ttomecek
Target Milestone: rcKeywords: Extras
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: redhat-upgrade-tool-0.7.45-1.el6 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 09:02:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Debug data
none
Log
none
/usr/bin/redhat-upgrade-tool with logging fix
none
Logs
none
Logs none

Description Jaroslav Škarvada 2016-02-02 15:40:43 UTC
Created attachment 1120476 [details]
Debug data

Description of problem:
I was trying to upgrade from RHEL-6.7 to RHEL-7.2, but the upgrade process failed.

Version-Release number of selected component (if applicable):
preupgrade-assistant-2.1.4-6.el6.noarch
preupgrade-assistant-contents-0.6.43-1.el6.noarch

How reproducible:
Always

Steps to Reproduce:
1. cd /etc/yum.repos.d
2. wget 'http://git.app.eng.bos.redhat.com/git/preupgrade-assistant.git/plain/repo-files/preupgrade-el6.repo'
3. yum install redhat-upgrade-tool preupgrade-assistant-contents
4. preupg --force -s RHEL6_7
5. redhat-upgrade-tool-cli --cleanup-post --addrepo optional=http://download.devel.redhat.com/released/RHEL-7/7.2/Server-optional/x86_64/os/ --instrepo=http://download.devel.redhat.com/released/RHEL-7/7.2/Server/x86_64/os --network 7.2

Actual results:
Downloading failed: Při stahování balíčků došlo k chybě.
  crash-7.0.2-6.el7.x86_64: failure: Packages/crash-7.0.2-6.el7.x86_64.rpm from cmdline-instrepo: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
  perl-Test-Harness-3.28-2.el7.noarch: failure: Packages/perl-Test-Harness-3.28-2.el7.noarch.rpm from cmdline-instrepo: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
  ttmkfdir-3.0.9-41.el7.x86_64: failure: Packages/ttmkfdir-3.0.9-41.el7.x86_64.rpm from cmdline-instrepo: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
  7:lvm2-2.02.105-14.el7.x86_64: failure: Packages/lvm2-2.02.105-14.el7.x86_64.rpm from cmdline-instrepo: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
  gtk3-3.8.8-5.el7.x86_64: failure: Packages/gtk3-3.8.8-5.el7.x86_64.rpm from cmdline-instrepo: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
  pulseaudio-libs-3.0-22.el7.x86_64: failure: Packages/pulseaudio-libs-3.0-22.el7.x86_64.rpm from cmdline-instrepo: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
...

Expected results:
No errors

Additional info:

Comment 2 David Shea 2016-02-02 15:57:45 UTC
Please attach /var/log/redhat_upgrade_tool.log.

The versions of the packages in the errors are from RHEL 7.1, so it looks like something went wrong in the repo setup.

Comment 3 Jaroslav Škarvada 2016-02-02 16:01:47 UTC
Created attachment 1120486 [details]
Log

Comment 4 David Shea 2016-02-04 15:06:22 UTC
Created attachment 1121125 [details]
/usr/bin/redhat-upgrade-tool with logging fix

There's a bug in the logging in the main script. I've attached a new version of the main redhat-upgrade-tool script. Can you replace the script in /usr/bin with the attachment, run redhat-upgrade-tool again, and attach the /var/log/redhat_upgrade_tool.log from that?

The diff is just this:

diff --git a/redhat-upgrade-tool.py b/redhat-upgrade-tool.py
index 0a614ee..28caef2 100755
--- a/redhat-upgrade-tool.py
+++ b/redhat-upgrade-tool.py
@@ -46,7 +46,7 @@ from preup import xccdf
 from preup import settings
 
 import logging
-log = logging.getLogger("redhat-upgrade-tool")
+log = logging.getLogger("redhat_upgrade_tool.main")
 def message(m):
     print m
     log.info(m)

Comment 5 Jaroslav Škarvada 2016-02-04 15:25:30 UTC
Ups, I have just returned the machine. I will take another, but it may take some time.

Comment 6 Jaroslav Škarvada 2016-02-05 15:47:01 UTC
Created attachment 1121424 [details]
Logs

Different machine, different error:
getting boot images...
.treeinfo                                                | 2.2 kB     00:00     
setting up update...
verify local files 100% [======================================================]
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Importing GPG key 0xFD431D51:
 Userid : Red Hat, Inc. (release key 2) <security>
 Package: redhat-release-server-6Server-6.7.0.3.el6.x86_64 (@anaconda-RedHatEnterpriseLinux-201507020259.x86_64/6.7)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Importing GPG key 0x2FA658E0:
 Userid : Red Hat, Inc. (auxiliary key) <security>
 Package: redhat-release-server-6Server-6.7.0.3.el6.x86_64 (@anaconda-RedHatEnterpriseLinux-201507020259.x86_64/6.7)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

Downloading failed: Didn't install any keys

Comment 7 David Shea 2016-02-05 15:52:55 UTC
The key error is normal. redhat-upgrade-tool attempts to check gpg signatures by default, and the packages on download.devel are not signed. You can add --nogpgcheck to the command line for that.

Obviously this isn't the most friendly failure mode. I think there's a bug open to ask the user whether to add the keys, like yum does, instead of just failing.

Comment 8 Jaroslav Škarvada 2016-02-05 16:06:58 UTC
Thanks, it finished OK. I will retry on another machine and will try to exactly reproduce the steps that lead to the failure.

Comment 9 Jaroslav Škarvada 2016-02-05 16:48:49 UTC
Created attachment 1121453 [details]
Logs

I finally reproduced the problem by using the following steps (some may be redundant):
# cd /etc/yum.repos.d
# wget 'http://git.app.eng.bos.redhat.com/git/preupgrade-assistant.git/plain/repo-files/preupgrade-el6.repo'
# cd
# yum install redhat-upgrade-tool preupgrade-assistant-contents
# redhat-upgrade-tool-cli --cleanup-post --addrepo optional=http://download.devel.redhat.com/released/RHEL-7/7.0/Server-optional/x86_64/os/ --instrepo=http://download.devel.redhat.com/released/RHEL-7/7.0/Server/x86_64/os --network 7.0
# preupg --force -s RHEL6_7
# redhat-upgrade-tool-cli --cleanup-post --addrepo optional=http://download.devel.redhat.com/released/RHEL-7/7.0/Server-optional/x86_64/os/ --instrepo=http://download.devel.redhat.com/released/RHEL-7/7.0/Server/x86_64/os --network 7.0
# redhat-upgrade-tool-cli --cleanup-post --addrepo optional=http://download.devel.redhat.com/released/RHEL-7/7.0/Server-optional/x86_64/os/ --instrepo=http://download.devel.redhat.com/released/RHEL-7/7.2/Server/x86_64/os --network 7.2
# redhat-upgrade-tool-cli --cleanup-post --addrepo optional=http://download.devel.redhat.com/released/RHEL-7/7.2/Server-optional/x86_64/os/ --instrepo=http://download.devel.redhat.com/released/RHEL-7/7.2/Server/x86_64/os --network 7.2

I think the last two steps are important. The cache/workdir is probably not cleared.

Comment 10 David Shea 2016-02-08 15:06:19 UTC
I see. The quick fix is don't do that ;-)  I'll add a check for whether there is an existing upgrade directory and print a message about it needing to be cleaned before another upgrade can be run.

Comment 11 Michal Bocek 2016-07-27 17:20:01 UTC
Fix prepared:
https://github.com/upgrades-migrations/redhat-upgrade-tool/commit/d3968611e3b81bdc53e83d472bea813880898b53
This is a minor bug so it will be added to a next version of the tool.

Comment 14 errata-xmlrpc 2016-11-04 09:02:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2619.html