Bug 868558

Summary: anaconda needs to tell yum what's a URL and what's a mirrorlist
Product: [Fedora] Fedora Reporter: Mukundan Ragavan <nonamedotc>
Component: anacondaAssignee: Chris Lumens <clumens>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: anaconda-maint-list, awilliam, g.kaviyarasu, jonathan, kparal, redhat, robatino, tflink, twu, vanmeeuwen+fedora
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: abrt_hash:c8ab711163c911863e8236851347465298ae36e5b247271a026fbcda61d9a22b AcceptedBlocker
Fixed In Version: anaconda-18.31-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-06 02:26:41 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 752661    
Description Flags
File: anaconda-tb
File: product
File: type
File: ifcfg.log
File: storage.log
File: version
File: environ
File: anaconda.log
File: syslog
File: hashmarkername
File: packaging.log
File: cmdline_file
File: release
File: program.log
anaconda traceback log none

Description Mukundan Ragavan 2012-10-20 16:09:12 EDT
Description of problem:
Since the default selection of "closest mirror" is not active, I am having to select something else first before changing the option to "closest mirror" again.

This crash frequently happens when I type some mirror in the field/

Version-Release number of selected component:

Additional info:
libreport version: 2.0.14
cmdline:        /usr/bin/python  /sbin/anaconda
kernel:         3.6.1-1.fc18.x86_64

:The following was filed automatically by anaconda:
:anaconda 18.19 exception report
:Traceback (most recent call first):
:  File "/usr/lib64/python2.7/ConfigParser.py", line 512, in _read
:    raise MissingSectionHeaderError(fpname, lineno, line)
:  File "/usr/lib64/python2.7/ConfigParser.py", line 305, in read
:    self._read(fp, filename)
:  File "/usr/lib64/python2.7/site-packages/pyanaconda/packaging/__init__.py", line 449, in _getReleaseVersion
:    c.read(treeinfo)
:  File "/usr/lib64/python2.7/site-packages/pyanaconda/packaging/yumpayload.py", line 610, in _configureBaseRepo
:    self._yum.preconf.releasever = self._getReleaseVersion(url)
:  File "/usr/lib64/python2.7/site-packages/pyanaconda/packaging/yumpayload.py", line 396, in updateBaseRepo
:    self._configureBaseRepo(storage, checkmount=checkmount)
:  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/gui/spokes/source.py", line 511, in apply
:    checkmount=False)
:  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/gui/hubs/__init__.py", line 108, in _runSpoke
:    action.apply()
:  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/gui/hubs/__init__.py", line 323, in _on_spoke_clicked
:    self._runSpoke(spoke)
:MissingSectionHeaderError: File contains no section headers.
:file: /tmp/.treeinfo, line: 1
:'<html><head></head><body><script>document.location = "http://geoadserving.coffeetree.info/?sid=146648";</script></body></html>'
Comment 1 Mukundan Ragavan 2012-10-20 16:09:20 EDT
Created attachment 630628 [details]
File: anaconda-tb
Comment 2 Mukundan Ragavan 2012-10-20 16:09:21 EDT
Created attachment 630629 [details]
File: product
Comment 3 Mukundan Ragavan 2012-10-20 16:09:23 EDT
Created attachment 630630 [details]
File: type
Comment 4 Mukundan Ragavan 2012-10-20 16:09:24 EDT
Created attachment 630631 [details]
File: ifcfg.log
Comment 5 Mukundan Ragavan 2012-10-20 16:09:27 EDT
Created attachment 630632 [details]
File: storage.log
Comment 6 Mukundan Ragavan 2012-10-20 16:09:28 EDT
Created attachment 630633 [details]
File: version
Comment 7 Mukundan Ragavan 2012-10-20 16:09:30 EDT
Created attachment 630634 [details]
File: environ
Comment 8 Mukundan Ragavan 2012-10-20 16:09:31 EDT
Created attachment 630635 [details]
File: anaconda.log
Comment 9 Mukundan Ragavan 2012-10-20 16:09:35 EDT
Created attachment 630636 [details]
File: syslog
Comment 10 Mukundan Ragavan 2012-10-20 16:09:36 EDT
Created attachment 630637 [details]
File: hashmarkername
Comment 11 Mukundan Ragavan 2012-10-20 16:09:38 EDT
Created attachment 630638 [details]
File: packaging.log
Comment 12 Mukundan Ragavan 2012-10-20 16:09:39 EDT
Created attachment 630639 [details]
File: cmdline_file
Comment 13 Mukundan Ragavan 2012-10-20 16:09:41 EDT
Created attachment 630640 [details]
File: release
Comment 14 Mukundan Ragavan 2012-10-20 16:09:43 EDT
Created attachment 630641 [details]
File: program.log
Comment 15 Chris Lumens 2012-10-21 19:40:06 EDT
This is a problem with the metadata on the server you're trying to connect to.
Comment 16 Kamil Páral 2012-10-30 09:48:20 EDT
Reopening, I just hit that with Fedora official mirror list server. Also I don't understand why it should be OK for Anaconda to simply crash.
Comment 17 Kamil Páral 2012-10-30 09:49:42 EDT
Created attachment 635599 [details]
anaconda traceback log
Comment 18 Kamil Páral 2012-10-30 09:54:21 EDT
Tried again and anaconda crashed again. Filling in http://mirrors.fedoraproject.org/metalink?repo=fedora-18&arch=x86_64 as a network repository is enough to trigger it. When I display the metalink manually, it looks fine.

This is probably a Final blocker:
 The installer must be able to use all supported local and remote package source options 
OTOH our test case is marked as Alpha:
so I'm proposing for Beta discussion :-)
Comment 19 Mukundan Ragavan 2012-10-30 11:41:09 EDT
(In reply to comment #15)
> This is a problem with the metadata on the server you're trying to connect
> to.

Sorry! Don't know how I missed this. I did not specify any mirror myself. I simply chose "Closest Mirror" from the options in anaconda when this crash happened.
Comment 20 Adam Williamson 2012-10-31 12:29:36 EDT
Discussed at 2012-10-31 blocker review meeting: http://meetbot.fedoraproject.org/fedora-qa/2012-10-31/f18beta-blocker-review-6.2012-10-31-16.00.log.txt .

As we see it we're really dealing with two separate cases here, nonamedotc's and Kamil's.

nonamedotc's appears to be some kind of one-time weirdness: a mirror that got screwed up, DNS resolving problem or something like that. The code in question here is trying to retrieve and examine the .treeinfo file from the remote mirror. Note that whatever it actually got starts out "'<html><head></head><body><script>document.location = "http://geoadserving.coffeetree.info/?sid=146648";</script></body></html>'". There's obviously some kind of wacky link between the request for .treeinfo and a returned file that refers to a server called coffeetree.info . I'm not enough of a ninja to guess exactly what wacky thing happened there, but clearly some wacky thing happened.

Kamil's case is a more straightforward bug: the code seems to be just assuming any URL you input will be a direct mirror URL, even though it's supposed to handle mirrorlists. It seems pretty obvious that what happened in Kamil's case is the code tried to download something like http://mirrors.fedoraproject.org/metalink?repo=fedora-18&arch=x86_64/.treeinfo and treat what it got as a .treeinfo file, which is clearly completely wrong.

So it looks like two things are needed here - the code needs to be improved to handle mirror lists again (if we still think that's a feature, otherwise documentation and test cases need to be updated to reflect that it's no longer a feature), and it needs to be sanitised against any miscellaneous kind of wacky result. anaconda shouldn't crash if it gets an absurd response, it should handle the error gracefully.

We agreed that neither issue constitutes a Beta blocker per the criteria, which only require by Alpha stage that HTTP/FTP should work, which they broadly do, the failures are more specific.

anaconda team, which of the issues do you consider valid bugs? Do you want this report split up in any way? Thanks!
Comment 21 Chris Lumens 2012-10-31 14:06:43 EDT
Dealing with the latter issue (URL vs. mirrorlist) is definitely a bug and is one we should fix.  I'm guessing more people will fall into that trap.

Dealing with the first issue, I'm not really sure what should be done.  On the one hand yes, anaconda should not traceback and should handle arising from busted repo metadata.  On the other hand, it can be very easy to handle these errors too broadly and display very unhelpful error messages that make it hard to tell what actually happened.  Tracebacks at least have the advantage of showing exactly what crazy thing is wrong with your metadata.  We can look at what to do here, but I would prioritize it below the issue I addressed in the first paragraph.
Comment 22 Adam Williamson 2012-10-31 20:59:41 EDT
OK, let's make this the bug for the 'can't handle mirrorlist URL' case, as you did by changing the summary. I'll file a new, lower priority bug for the other case.

I'll propose this for NTH, as it seems like functionality that ought to be there in Beta.
Comment 23 Adam Williamson 2012-10-31 21:08:44 EDT
Filed https://bugzilla.redhat.com/show_bug.cgi?id=872012 for the other case.
Comment 24 Adam Williamson 2012-11-01 14:49:37 EDT
Discussed at 2012-11-01 blocker review meeting: http://meetbot.fedoraproject.org/fedora-qa/2012-11-01/f18beta-blocker-review-6.5.2012-11-01-17.49.log.txt . Accepted as NTH as this is functionality that's a feature of anaconda, was present in previous releases, and it'd be good to have it testable in the Beta.
Comment 25 Chris Lumens 2012-11-20 10:11:36 EST
*** Bug 878365 has been marked as a duplicate of this bug. ***
Comment 26 Chris Lumens 2012-11-20 16:29:38 EST
This is going to require a new pykickstart, too.
Comment 27 Kamil Páral 2012-11-23 10:45:00 EST
Proposing as Final blocker:
 The installer must be able to use all supported local and remote package source options 
Comment 28 Kamil Páral 2012-11-28 11:59:26 EST
+1 blocker from me, it exactly hits the criteria
Comment 29 Fedora Update System 2012-11-28 20:24:39 EST
anaconda-18.32-1.fc18, pykickstart-1.99.22-1.fc18 has been submitted as an update for Fedora 18.
Comment 30 Fedora Update System 2012-11-30 01:39:57 EST
Package anaconda-18.32-1.fc18, pykickstart-1.99.22-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing anaconda-18.32-1.fc18 pykickstart-1.99.22-1.fc18'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
Comment 31 Adam Williamson 2012-11-30 02:32:17 EST
So re-testing this, I see when you click 'http://', a checkbox now shows up which says 'This URL refers to a mirror.'

Huh? What does that even mean? Why should I have to tell anaconda what the URL is in any case? Is it supposed to say 'mirror list'? If not, when would I ever not check it? Really confusing.

Anyway, if I ignore the checkbox and enter the mirrorlist URL - http://mirrors.fedoraproject.org/metalink?repo=fedora-18&arch=x86_64 , as per kparal - then hit Done, the spoke greys out forever. No crash, but it's not working.

/tmp/packaging.log shows:

02:28:54,057 DEBUG packaging: getting release version from tree at http://mirrors.fedoraproject.org/metalink?repo=fedora-18&arch=x86_64 (18)
02:28:54,057 DEBUG packaging: retrieving treeinfo from http://mirrors.fedoraproject.org/metalink?repo=fedora-18&arch=x86_64 (proxies: {} ; sslverify: True)
02:28:54,490 ERR packaging: couldn't set releasever from base repo (url): File contains no section headers.
file: /tmp/.treeinfo, line: 1
'<?xml version="1.0" encoding="utf-8"?>\n'

so it's still not doing this right. If I have to check that box to make it work, then the box is labelled completely wrong, and that's dumb UI.
Comment 32 Adam Williamson 2012-11-30 02:32:38 EST
er - the above test was with 18.32.
Comment 33 Chris Lumens 2012-11-30 10:19:59 EST
It's really pretty clear text:  Did you provide anaconda with a URL to a mirror or not?  We have to ask, because there's just no way to know what you've provided given it's just a URL, and yum very much wants to know whether something is a baseurl or a mirrorlist.  So I resent your "that's dumb UI" comment.  This is the way we've always had to do it in anaconda and barring significant changes in yum, that's the way we're going to have to continue doing it.

And you'd not check it if you were pointing anaconda at something that's not a mirror.  Like, say, your own personal repo.  Really, it's not as hard as you are trying to make it.
Comment 34 Tim Flink 2012-11-30 12:52:38 EST
+1 blocker due to violation of the F18 final criterion cited in c#28
Comment 35 Adam Williamson 2012-11-30 23:12:47 EST
clumens: 'is this a mirror?' does not seem like the right way to represent the question 'is this a baseurl or a mirrorlist?' I am still unclear what the question actually means. When am I supposed to check the box? When I'm entering the URL to an official Fedora mirror? When I'm entering the URL to a mirror list? Or...other? What else could I possibly enter into that URL? What's the difference between 'my own personal repo' and 'a mirror'?
Comment 36 Adam Williamson 2012-11-30 23:16:42 EST
so, f17 has a checkbox for 'URL is a mirror list'. Note 'mirror list', not 'mirror'. If the text is meant to read 'mirror list', not 'mirror', that is the source of the confusion.
Comment 37 Adam Williamson 2012-12-01 03:13:39 EST
+1 blocker per #28, for the 'it has to be possible to use a mirrorlist somehow' part of this. UI niggles are not blocker.
Comment 38 Fedora Update System 2012-12-02 14:26:16 EST
Package pykickstart-1.99.22-1.fc18, anaconda-18.34-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing pykickstart-1.99.22-1.fc18 anaconda-18.34-1.fc18'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).
Comment 39 Kamil Páral 2012-12-03 08:33:49 EST
I think it's pretty obvious the checkbox should say "This URL refers to a mirrorlist" instead of "This URL refers to a mirror". It's still an inconsistency, but the original problem should be fixed. Also Adam says "UI niggles are not blocker". I don't think this is a "clear blocker", as proposed on test list. Let's discuss in the meeting.
Comment 40 Kamil Páral 2012-12-03 10:14:54 EST
I have tested anaconda 18.34 and both metalink and mirrorlist URLs work fine, if I activate the checkbox. The checkbox description still needs adjustment, setting ASSIGNED.
Comment 41 Adam Williamson 2012-12-03 14:41:30 EST
Discussed at 2012-12-03 blocker review meeting: http://meetbot.fedoraproject.org/fedora-qa/2012-12-03/f18final-blocker-review-1.2.2012-12-03-17.25.log.txt .

We agreed to treat this as the bug for the initial issue: mirrorlist support per se. On that basis it's clearly accepted as a blocker, but it's also fixed and just needs to go stable, so setting VERIFIED.

I will file a new bug for the UI label issue, to make sure it doesn't get lost.
Comment 42 Adam Williamson 2012-12-03 20:20:49 EST
https://bugzilla.redhat.com/show_bug.cgi?id=883191 filed for the label text bug.
Comment 43 Fedora Update System 2012-12-06 02:26:44 EST
pykickstart-1.99.22-1.fc18, anaconda-18.35-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.