Bug 805316

Summary: anaconda looks for %includes before %pre is run
Product: [Fedora] Fedora Reporter: Orion Poplawski <orion>
Component: anacondaAssignee: Will Woods <wwoods>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 17CC: bfay, cra, g.kaviyarasu, jonathan, myllynen, pasteur, ppinatti, sr, t.h.amundsen, tomasz.kepczynski, tomek, vanmeeuwen+fedora
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-08-01 13:11:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Error from text mode install
none
anaconda.log
none
/tmp/anaconda.log from Fedora17 serial console Kickstart none

Description Orion Poplawski 2012-03-20 21:07:04 UTC
Description of problem:

It appears that in F17 anaconda looks for %includes before %pre is run. 
cobbler uses a setup where a file is generate in /tmp during %pre and %included
in the main anaconda section.  Something like:

install
...
%include /tmp/pre_install_network_config

%pre
echo network  --proto=dhcp > /tmp/pre_install_network_config.

I now see:

parse-kickstart ERROR: The following problem occurred on line 0 of the kickstart file:

Unable to open input kickstart file: Could not open/read file:///tmp/pre_install_network_config


Couple things:
- "line 0" is incorrect, this in on line 34.  Or perhaps it's line 0 of "/tmp/pre_install_network_config"
- The /tmp/pre_install_network_config file does end up getting created.


The install continues, presumably without the proper network configuration.

Version-Release number of selected component (if applicable):
17.13

Comment 1 Trond H. Amundsen 2012-04-17 15:34:25 UTC
I'm seeing this as well, with anaconda-17.20 (F17 Beta).

Comment 2 Will Woods 2012-04-17 17:31:00 UTC
The installer has always parsed the kickstart in the initrd.img, well before %pre.
The old parsing code might have ignored %includes; the new code attempts to follow them. The actual parsing in anaconda is separate and should be basically unchanged.

I suspect this is a harmless (albeit confusing) error message, and that the network configuration is actually correct. Can you please finish the install and confirm?

If that is indeed the case, I'll have it ignore missing %includes while parsing in initrd.img.

Comment 3 Orion Poplawski 2012-04-17 20:16:53 UTC
Yeah, that indeed appears to be the case.  Network configuration works.

Comment 4 Tomasz Kepczynski 2012-05-31 20:19:04 UTC
I seem to have the same problem but in different part of kickstart file.
I keep my %packages and %post in separate files as they are the same for
all kickstart files:

%packages
%include /tmp/rpm-list.cfg
%end

%post
%include /tmp/std-post.sh
%end

Both files above are copied from nfs server in %pre script:

%pre
nfsServer="triss.jot23.org"
mntPoint="/tmp/mnt"
[[ -d "${mntPoint}" ]] || mkdir "${mntPoint}"
mount -n -r -o nolock ${nfsServer}:/srv/nfs/install "${mntPoint}"
cp "${mntPoint}"/linux/cfg/fedora-17/{rpm-list.cfg,std-post.sh} /tmp
umount "${mntPoint}"
rmdir "${mntPoint}"
%end

The above setup used to work up to Fedora 16 and Scientific Linux 6.2 (and
presumably RHEL 6.2 as well). It gave me over 1400 packages in the resulting
OS installation on Fedora 16.

It ceased to work with Fedora 17. Anaconda picked up only 426 packages and
my post install script has not been run.

So - is this intended behaviour? If not - can this be reinstated? If yes -
how are we supposed to use %include directive in kickstart (am I missing
something - I don't think there is a way to say "get this file from the
location where you get this kickstart from")?

Comment 5 Tomasz Kepczynski 2012-05-31 20:34:54 UTC
Created attachment 588187 [details]
Error from text mode install

The screenshot with error message from text mode install. Graphical install gives similar (if not identical) error message.

Comment 6 Steve Rikli 2012-06-01 00:11:02 UTC
Confirmed in Fedora 17 release; I see that the %include files are actually
present as expected in /tmp/ , but maybe not in time for anaconda?

I use %include files in the main command section and in %packages section,
(e.g. /tmp/diskpart.inc and /tmp/packages.inc), both are created in %pre
section, and I can see the files in /tmp/, but anaconda complains they are
not available when ks.cfg is parsed.

E.g from a serial console Kickstart install:

[   37.440190] dracut: anaconda fetching kickstart from nfs:*****/ks-f17.cfg
[   37.668175] FS-Cache: Loaded
[   37.708449] Registering the id_resolver key type
[   37.763691] FS-Cache: Netfs 'nfs' registered for caching
[   38.679067] parse-kickstart ERROR: The following problem occurred on line 0 of the kickstart file:
[   38.679071] 
[   38.679072] Unable to open input kickstart file: Could not open/read file:///tmp/diskpart.inc
[   38.679074]

... and in my case, the installation fails (probably expected at this point)
with "Could not allocate requested partitions: not enough free space on disks."
I expect packages selection would fail too, but it hasn't gotten that far.

This same mechanism worked OK in Fedora 16 and prior.

Comment 7 Will Woods 2012-06-02 17:27:50 UTC
The "Unable to open..." message from dracut is *harmless*.

Fedora 16 and earlier also failed to open the files at that point (because they don't exist yet!), but they suppressed the error message.

Once anaconda actually starts, it loads the kickstart file, runs %pre scripts, then *re-parses the kickstart* specifically for this purpose:

    if opts.ksfile:
        kickstart.preScriptPass(anaconda, opts.ksfile)
        anaconda.ksdata = kickstart.parseKickstart(anaconda, opts.ksfile)

(that's line 776-778 of anaconda)

Can you please attach the anaconda.log from one of these failed attempts?

Comment 8 Tomasz Kepczynski 2012-06-02 19:23:40 UTC
Created attachment 588710 [details]
anaconda.log

Requested log file. Please search for 'no package matching %include'.

Comment 9 Will Woods 2012-06-02 19:58:04 UTC
In my testing, this works fine, despite the "Unable to open input kickstart file" message:

  %pre
  echo "rootpw  --plaintext fedora" > /tmp/rootpw.txt
  %end

  %include /tmp/rootpw.txt

For %packages, this doesn't work:

  %pre
  echo -e "@gnome-desktop" > /tmp/packages.txt
  %end
  
  %packages
  %include /tmp/packages.txt
  %end

But this does:

  %pre
  echo -e "%packages\n@gnome-desktop\n%end" > /tmp/packages.txt
  %end

  %include /tmp/packages.txt

It may be the case that %include is no longer parsed inside the %packages section.

Could you try modifying your kickstart/script to put '%packages ... %end' inside the %included file and see if that fixes your problem?

Comment 10 Tomasz Kepczynski 2012-06-03 10:45:18 UTC
The solution with putting %packages...%end inside package list worked for me.

I also had problems with putting post script in %included file and putting %post...%end inside the included file (and not in kickstart file itself) also fixed this issue.

Comment 11 Steve Rikli 2012-06-03 12:27:01 UTC
(In reply to comment #9)
> In my testing, this works fine, despite the "Unable to open input kickstart
> file" message:
> 
>   %pre
>   echo "rootpw  --plaintext fedora" > /tmp/rootpw.txt
>   %end
> 
>   %include /tmp/rootpw.txt

I'm doing effectively the same steps for e.g. disk partitioning in the main
command section, but it doesn't "just work" like it did in Fedora 16 -- it
fails with disk partitioning errors, which is why I looked into the "unable
to open" messages that lead me here originally.

I suspect it would also fail with other command section %include files, but
isn't getting that far yet.  I'll reproduce and attach anaconda.log .

> It may be the case that %include is no longer parsed inside the %packages
> section.
> 
> Could you try modifying your kickstart/script to put '%packages ... %end'
> inside the %included file and see if that fixes your problem?

I'll try that as well.  But is this a bug or new feature?  I.e. can we expect
a return to the behavior in Fedora 16 and prior, or is this an intentional
change we should prepare to deal with going forward?  I support a multi-OS
and multi-version Kickstart installation, and when a new behavior appears in
the leading edge, it sometimes has unintentional consequences on the others.

Comment 12 Steve Rikli 2012-06-04 21:47:32 UTC
Created attachment 589279 [details]
/tmp/anaconda.log from Fedora17 serial console Kickstart

Comment 13 Steve Rikli 2012-06-04 21:59:20 UTC
anaconda.log from failed disk partitioning in Fedora17 serial console Kickstart
is attached.  For reference, it's created basically like so:

%pre
wget -P/tmp http://172.19.7.11/kickstart/configs/pre/diskpart-biosboot.sh
/bin/sh /tmp/diskpart-biosboot.sh > /tmp/diskpart.inc
%end

and in the main command section, among the other typical things:

zerombr
%include /tmp/diskpart.inc


If it helps, the /tmp/diskpart.inc does get created with the right info (i.e.
same as a Fedora 16 install, including biosboot); so the method we're using
for Fedora 16 and Fedora 17 hasn't changed, but the former works and the
latter fails with this dialog box error:

        Could not allocate requested partitions: 
        not enough free space on disks.
        Press 'OK' to exit the installer.
                   [OK]

I have no trouble believing that isn't due to %include files (not) being
parsed, but from looking at the error msg and the logs in /tmp/ I'm still
at a loss to understand why this works in F16 and prior, but not F17.

Comment 14 Steve Rikli 2012-06-06 21:43:32 UTC
After additional testing on my own, I'm ready to discount the partitioning
error & subsequent failure as unrelated to this %include file topic -- I've
replicated it using the same "part ..." commands directly in ks.cfg instead
of via %include file, so I believe it is due to some change in Fedora 17
partitioning rules/syntax rather than the %include label parsing we seem
to be facing in this bug.

So putting that aside, I would still like to know if we will see a fix for
the F17 behavior, and a return to how Fedora 16 and earlier handled %include
directives inside %packages and %post sections.

Comment 15 Tomasz Kepczynski 2012-07-30 18:10:27 UTC
(In reply to comment #9)
> ...
> But this does:
> 
>   %pre
>   echo -e "%packages\n@gnome-desktop\n%end" > /tmp/packages.txt
>   %end
> 
>   %include /tmp/packages.txt
> 
> It may be the case that %include is no longer parsed inside the %packages
> section.

Please note that %include of a file with %post/%packages...%end will not fully substitute:

  %packages
  %include packages-1.txt
  %include packages-2.txt
  %end

Similar problem with %post. You simple loose the ability to collect parts from different files.

Comment 16 Fedora End Of Life 2013-07-04 04:19:48 UTC
This message is a reminder that Fedora 17 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 17. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '17'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 17's end of life.

Bug Reporter:  Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 17 is end of life. If you 
would still like  to see this bug fixed and are able to reproduce it 
against a later version  of Fedora, you are encouraged  change the 
'version' to a later Fedora version prior to Fedora 17's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 17 Fedora End Of Life 2013-08-01 13:11:51 UTC
Fedora 17 changed to end-of-life (EOL) status on 2013-07-30. Fedora 17 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.