Bug 1259506 - anaconda not parsing kickstart files containing %include
anaconda not parsing kickstart files containing %include
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: anaconda (Show other bugs)
23
x86_64 Linux
unspecified Severity medium
: ---
: ---
Assigned To: David Shea
Fedora Extras Quality Assurance
AcceptedFreezeException AcceptedBlocker
:
Depends On:
Blocks: F23BetaFreezeException F23FinalBlocker
  Show dependency treegraph
 
Reported: 2015-09-02 16:11 EDT by Mark Montague
Modified: 2016-01-04 16:44 EST (History)
13 users (show)

See Also:
Fixed In Version: 23.19.4-1.fc23
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-09-16 14:35:55 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Main kickstart file containing an %include directive (839 bytes, text/plain)
2015-09-02 16:11 EDT, Mark Montague
no flags Details
Kickstart file included by ks-f23-main.cfg (220 bytes, text/x-matlab)
2015-09-02 16:12 EDT, Mark Montague
no flags Details
Console output from virt-install of machine + ks files to install machine (62.91 KB, application/x-bzip)
2015-11-08 18:10 EST, Peter Lawler
no flags Details
Our ks file which seems to fail int he same way (3.66 KB, text/plain)
2015-11-12 17:40 EST, Jeffrey Lund
no flags Details

  None (edit)
Description Mark Montague 2015-09-02 16:11:17 EDT
Created attachment 1069582 [details]
Main kickstart file containing an %include directive

Description of problem:

Attempting to install Fedora Server 23 Alpha from ISO using a kickstart file provided by a web server fails if the kickstart file contains an %include directive.  If the %include directives are replaced by hand with the contents of the file, then the install succeeds.


Version-Release number of selected component (if applicable):


How reproducible:
always


Steps to Reproduce:
1. Put ks-f23-main.cfg and ks-f23-packages.cfg on a web server accessible to the system being installed.
2. Boot from the Fedora Server 23 Alpha DVD image using the following kernel command line (changing the URL to be the correct URL from step 1, above):
vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-S-23_A-x86_64 ks=http://192.168.4.1/~markmont/ks-f23-main.cfg


Actual results:

Anaconda fails with:
19:13:59,803 INFO anaconda: /sbin/anaconda 23.17-1
19:14:01,039 INFO anaconda: created new libuser.conf at /tmp/libuser.tsihfa97 with instPath="/mnt/sysimage"
19:14:01,040 INFO anaconda: 4161040 kB (4063 MB) are available
19:14:01,047 INFO anaconda: check_memory(): total:4161040, needed:320, graphical:410
19:14:01,053 INFO anaconda: anaconda called with cmdline = ['/sbin/anaconda']
19:14:01,054 INFO anaconda: Default encoding = utf-8
19:14:01,072 ERR anaconda.stdout: Kickstart file /run/install/ks.cfg is missing.

"journalctl -a" shows:
Sep 02 18:57:35 localhost dracut-initqueue[587]: //lib/url-lib.sh@67(curl_fetch_url): curl --globoff --location --retry 3 --fail --show-error --output - -- http://192.168.4.1/~markmont/ks-f23-main.cfg
Sep 02 18:57:35 localhost dracut-initqueue[587]: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Sep 02 18:57:35 localhost dracut-initqueue[587]: Dload  Upload   Total   Spent    Left  Speed
Sep 02 18:57:35 localhost dracut-initqueue[587]: 0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0^M100   839  100   839    0     0   679k      0 --:--:-- --:--:-- --:--:--  819k
Sep 02 18:57:35 localhost dracut-initqueue[587]: //lib/url-lib.sh@73(curl_fetch_url): '[' -f /tmp/ks.cfg ']'
Sep 02 18:57:35 localhost dracut-initqueue[587]: //lib/url-lib.sh@77(curl_fetch_url): '[' -z /tmp/ks.cfg ']'
Sep 02 18:57:35 localhost dracut-initqueue[587]: //lib/dracut/hooks/initqueue/fetch-ks-eno16777736.sh@5(source): parse_kickstart /tmp/ks.cfg
Sep 02 18:57:35 localhost dracut-initqueue[587]: //lib/anaconda-lib.sh@210(parse_kickstart): /sbin/parse-kickstart /tmp/ks.cfg
Sep 02 18:57:36 localhost dracut-initqueue[587]: parse-kickstart DEBUG: ksdevice argument is not available
Sep 02 18:57:36 localhost dracut-initqueue[587]: parse-kickstart INFO: processing kickstart file /tmp/ks.cfg
Sep 02 18:57:36 localhost dracut-initqueue[587]: parse-kickstart DEBUG: kickstart line 3: handling cdrom
Sep 02 18:57:36 localhost dracut-initqueue[587]: parse-kickstart DEBUG: kickstart line 4: handling text
Sep 02 18:57:36 localhost dracut-initqueue[587]: parse-kickstart DEBUG: kickstart line 18: handling bootloader
Sep 02 18:57:36 localhost dracut-initqueue[587]: Traceback (most recent call last):
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/sbin/parse-kickstart", line 604, in <module>
Sep 02 18:57:36 localhost dracut-initqueue[587]: outfile, output = process_kickstart(path)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/sbin/parse-kickstart", line 582, in process_kickstart
Sep 02 18:57:36 localhost dracut-initqueue[587]: parser.readKickstart(processed_file)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/pykickstart/parser.py", line 757, in readKickstart
Sep 02 18:57:36 localhost dracut-initqueue[587]: self.readKickstartFromString(s, reset=False)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/pykickstart/parser.py", line 730, in readKickstartFromString
Sep 02 18:57:36 localhost dracut-initqueue[587]: self._stateMachine (i)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/pykickstart/parser.py", line 680, in _stateMachine
Sep 02 18:57:36 localhost dracut-initqueue[587]: self._handleInclude(args[1])
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/pykickstart/parser.py", line 643, in _handleInclude
Sep 02 18:57:36 localhost dracut-initqueue[587]: self.readKickstart(f, reset=False)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/pykickstart/parser.py", line 753, in readKickstart
Sep 02 18:57:36 localhost dracut-initqueue[587]: s = load_to_str(f)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/pykickstart/load.py", line 42, in load_to_str
Sep 02 18:57:36 localhost dracut-initqueue[587]: return _load_url(location)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/pykickstart/load.py", line 79, in _load_url
Sep 02 18:57:36 localhost dracut-initqueue[587]: request = requests.get(location, verify=SSL_VERIFY)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/requests/api.py", line 69, in get
Sep 02 18:57:36 localhost dracut-initqueue[587]: return request('get', url, params=params, **kwargs)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/requests/api.py", line 50, in request
Sep 02 18:57:36 localhost dracut-initqueue[587]: response = session.request(method=method, url=url, **kwargs)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 451, in request
Sep 02 18:57:36 localhost dracut-initqueue[587]: prep = self.prepare_request(req)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 382, in prepare_request
Sep 02 18:57:36 localhost dracut-initqueue[587]: hooks=merge_hooks(request.hooks, self.hooks),
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/requests/models.py", line 304, in prepare
Sep 02 18:57:36 localhost dracut-initqueue[587]: self.prepare_url(url, params)
Sep 02 18:57:36 localhost dracut-initqueue[587]: File "/usr/lib/python3.4/site-packages/requests/models.py", line 369, in prepare_url
Sep 02 18:57:36 localhost dracut-initqueue[587]: host = host.encode('idna').decode('utf-8')
Sep 02 18:57:36 localhost dracut-initqueue[587]: LookupError: unknown encoding: idna
Sep 02 18:57:36 localhost dracut-initqueue[587]: //lib/anaconda-lib.sh@211(parse_kickstart): unset CMDLINE
Sep 02 18:57:36 localhost dracut-initqueue[587]: //lib/anaconda-lib.sh@212(parse_kickstart): . /tmp/ks.info
Sep 02 18:57:36 localhost dracut-initqueue[587]: /lib/anaconda-lib.sh: line 212: /tmp/ks.info: No such file or directory



Expected results:

Anaconda finds the kickstart file.


Additional info:

If booted from the Fedora Server 23 Alpha DVD with the kernel command line
vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-S-23_A-x86_64 ks=http://192.168.4.1/~markmont/ks-f23-main.cfg rd.shell rd.debug rd.break=cleanup

...the main kickstart file, containing the %include directive, is present as /tmp/ks.cfg and is correct.  The problem can then be reproduced by running

/sbin/parse-kickstart /tmp/ks.cfg

Should the %include have been processed, expanding the kickstart file, before /sbin/parse-kickstart was run, or should /sbin/parse-kickstart handle the %include?
Comment 1 Mark Montague 2015-09-02 16:12:30 EDT
Created attachment 1069583 [details]
Kickstart file included by ks-f23-main.cfg
Comment 2 Harald Hoyer 2015-09-03 06:03:41 EDT
dracut does not handle kickstart files... It's the anaconda part in dracut.
Comment 3 Fedora Blocker Bugs Application 2015-09-08 13:38:51 EDT
Proposed as a Blocker for 23-beta by Fedora user dshea using the blocker tracking app because:

 Proposing under beta criterion "The installer must be able to use all available kickstart delivery methods. "
Comment 4 Adam Williamson 2015-09-09 12:01:39 EDT
The kickstart criteria are somewhat vague and we've had 'specify exactly what bits of kickstart functionality are release blocking' on the todo list approximately forever. So I'm a bit wibbly-wobbly on the blockeriness of this, but I'm definitely +1 freeze exception at least. For blocker status, I might see it more as final than beta blocker.
Comment 5 Adam Williamson 2015-09-10 15:12:55 EDT
Discussed at 2015-09-10 blocker review meeting: https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2015-09-10/f23-blocker-review.2015-09-10-16.00.log.txt . As the kickstart criteria are still not solidly established this came down to a judgement call. We decided this was most appropriate as a freeze exception for Beta and a blocker for Final: it's not severe enough to block Beta.
Comment 6 Fedora Update System 2015-09-14 13:21:58 EDT
python-blivet-1.12.4-1.fc23 anaconda-23.19.4-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-15846
Comment 7 Fedora Update System 2015-09-15 12:50:49 EDT
anaconda-23.19.4-1.fc23, python-blivet-1.12.4-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.\nIf you want to test the update, you can install it with \n su -c 'yum --enablerepo=updates-testing update anaconda python-blivet'. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-15846
Comment 8 Fedora Update System 2015-09-16 14:35:42 EDT
anaconda-23.19.4-1.fc23, python-blivet-1.12.4-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Comment 9 Peter Lawler 2015-11-05 19:14:02 EST
(In reply to awilliam@redhat.com from comment #5)
> Discussed at 2015-09-10 blocker review meeting:
> https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2015-09-10/f23-
> blocker-review.2015-09-10-16.00.log.txt . As the kickstart criteria are
> still not solidly established this came down to a judgement call. We decided
> this was most appropriate as a freeze exception for Beta and a blocker for
> Final: it's not severe enough to block Beta.

I'm hitting the 'localhost dracut-initqueue[xxx]: /lib/anaconda-lib.sh: line 212: /tmp/ks.info: No such file or directory' on F23 Release.

Is ks broken on Fedora 23? I'm not seeing anything on Common Bugs about this.
Comment 10 Adam Williamson 2015-11-05 19:33:15 EST
The fix for this bug was in F23 Final, so either it didn't work, or you're seeing another bug. Please paste or attach your kickstart (minus any sensitive bits, obviously), and anything that looks useful from the log files in /tmp .
Comment 11 Peter Lawler 2015-11-05 19:39:39 EST
Will do so this weekend. FWIW, it would seem 1278260 is a duplicate report from release day, with logs.
Comment 12 Peter Lawler 2015-11-05 19:41:18 EST
(or maybe not a duplicate, certainly same failure at the end with the missing ks file... my bad... will get back to this on Sunday .au time)
Comment 13 Adam Williamson 2015-11-05 19:55:52 EST
Oh, it's entirely possible you're hitting https://fedoraproject.org/wiki/Common_F23_bugs#kickstart-named-repo .
Comment 14 Peter Lawler 2015-11-08 18:10 EST
Created attachment 1091481 [details]
Console output from virt-install of machine + ks files to install machine

(In reply to awilliam@redhat.com from comment #13)
> Oh, it's entirely possible you're hitting
> https://fedoraproject.org/wiki/Common_F23_bugs#kickstart-named-repo .

I've added that updates.img in, still getting the problem. Having a bit more of a poke around in the logs (see attached) I'm now no longer of the belief it's the same cause as the original bug here, however it's the same error out at the end. Along with working out what's causing this, maybe there's a case for a bit more descriptive error than this if it's a catchall idk.

Anyways, attached all the things. No sensitive data included. Am actually rather hopeful it's merely something obvious that I've done that I can't see.. but otoh maybe not. Again, apols for noise - but best to get it looked at I guess.
Comment 15 Jeffrey Lund 2015-11-12 17:40 EST
Created attachment 1093489 [details]
Our ks file which seems to fail int he same way

I inherited our system from someone far more knowledgeable than I, so I don't have much to add other than "me too". However, I've attached our ks file just in case it helps someone see the problem.
Comment 16 bissli82 2016-01-04 05:41:35 EST
issue still exists and was not fixed or fixed correctly in the GA release.
Any KS what soever, with the use of "%include" fails with error:

"racut-initqueue[587]: /lib/anaconda-lib.sh: line 212: /tmp/ks.info: No such file or directory"


Any way to overcome it? or just use ksappend for now?

Please fix it by version 24, it is a major impact for us.
Comment 17 David Shea 2016-01-04 09:46:20 EST
(In reply to bissli82 from comment #16)
> issue still exists and was not fixed or fixed correctly in the GA release.
> Any KS what soever, with the use of "%include" fails with error:
> 
> "racut-initqueue[587]: /lib/anaconda-lib.sh: line 212: /tmp/ks.info: No such
> file or directory"
> 
> 
> Any way to overcome it? or just use ksappend for now?
> 
> Please fix it by version 24, it is a major impact for us.

It is fixed in 24. There was a pykickstart update that did not make it into F23.
Comment 18 Peter Lawler 2016-01-04 16:44:20 EST
(In reply to David Shea from comment #17)

> It is fixed in 24. There was a pykickstart update that did not make it into
> F23.

Thanks for the info, David. I'll mate a note of testing out early releases to see if I can still break it ;)

Note You need to log in before you can comment on or make changes to this bug.