Bug 1578930

Summary: initial-setup fails on IoT images - failed due to invalid kickstart file
Product: [Fedora] Fedora Reporter: Paul Whalen <pwhalen>
Component: initial-setupAssignee: Martin Kolman <mkolman>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: urgent    
Version: 28CC: jkonecny, mkolman, pbrobinson, rvykydal, sbueno, v.podzimek+fedora, vponcova
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-04 14:57:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1269538    
Attachments:
Description Flags
journalctl
none
kickstart
none
Journalctl output with python3-kickstart-3.12-4.fc28
none
journalctl -u initial-setup none

Description Paul Whalen 2018-05-16 15:56:41 UTC
Created attachment 1437453 [details]
journalctl

Description of problem:
Initial-setup fails on the IoT images:

May 11 10:28:52 localhost.localdomain initial-setup[789]: kickstart parsing failed: The following problem occurred on line 42 of the kickstart file:
the following arguments are required: <mntpoint>

Version-Release number of selected component (if applicable):
initial-setup-0.3.58-1.fc28.aarch64

Kickstart and logs attached.

Comment 1 Paul Whalen 2018-05-16 15:58:36 UTC
Created attachment 1437466 [details]
kickstart

Comment 2 Vendula Poncova 2018-05-17 18:55:13 UTC
Fixed in a pull request: https://github.com/clumens/pykickstart/pull/224

Comment 3 Peter Robinson 2018-05-21 15:27:38 UTC
Can we get this pushed to both F-28 and rawhide please?

Comment 4 Martin Kolman 2018-05-22 14:22:22 UTC
(In reply to Peter Robinson from comment #3)
> Can we get this pushed to both F-28 and rawhide please?

for Rawhide this should be fixed once new Pykickstart build is available. 

For F28 the following is needed:
- Pykickstart build with the fix
- Bodhi update for the build so it can get into repos
- making sure the updated Pykickstart package ends up in the environment used for generating the IoT images

There is also an alternative fix in the works, which basically boils down to Initial Setup only parsing the sections is actually cares about. That will likely land in Initial Setup & possibly some small changes Anaconda as well. We are currently targeting the second fix just to Rawhide, but could theoretically backport it to F28 as well if the Pykickstart based fix could not be used for some reason.

Comment 5 Peter Robinson 2018-05-22 14:30:57 UTC
> We are currently targeting the second fix just to Rawhide, but could
> theoretically backport it to F28 as well if the Pykickstart based fix could
> not be used for some reason.

This is _CRITICAL_ for the IoT images and is currently blocking us from announcing IoT nightly images which is causing me and a number of other people stress. We need this fix in F-28 ASAP.

Comment 6 Fedora Update System 2018-05-22 20:25:47 UTC
pykickstart-3.12-4.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-59681406fe

Comment 7 Fedora Update System 2018-05-23 17:37:53 UTC
pykickstart-3.12-4.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-59681406fe

Comment 8 Paul Whalen 2018-05-24 21:07:03 UTC
Still fails, full logs attached

[root@rpi3-2 ~]# rpm -qa | grep kickstart
python3-kickstart-3.12-4.fc28.noarch
[root@rpi3-2 ~]# rpm -q python3-kickstart
python3-kickstart-3.12-4.fc28.noarch
[root@rpi3-2 ~]# systemctl status initial-setup
● initial-setup.service - Initial Setup configuration program
   Loaded: loaded (/usr/lib/systemd/system/initial-setup.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2018-05-24 20:38:50 UTC; 26min ago
  Process: 767 ExecStart=/usr/libexec/initial-setup/run-initial-setup (code=exited, status=1/FAILURE)
  Process: 759 ExecStartPre=/bin/plymouth quit (code=exited, status=0/SUCCESS)
  Process: 751 ExecStartPre=/bin/kill -SIGRTMIN+21 1 (code=exited, status=0/SUCCESS)
 Main PID: 767 (code=exited, status=1/FAILURE)

May 24 20:38:49 rpi3-2.friendly-neighbours.com org.fedoraproject.Anaconda.Boss[1072]: DEBUG:anaconda.modules.boss.module_manager:/org/fedoraproject/Anaconda/Modules/Security has quit.
May 24 20:38:49 rpi3-2.friendly-neighbours.com org.fedoraproject.Anaconda.Boss[1072]: DEBUG:anaconda.modules.boss.module_manager:/org/fedoraproject/Anaconda/Modules/Services is unavailable
May 24 20:38:49 rpi3-2.friendly-neighbours.com org.fedoraproject.Anaconda.Boss[1072]: DEBUG:anaconda.modules.boss.module_manager:/org/fedoraproject/Anaconda/Addons/Baz is unavailable
May 24 20:38:49 rpi3-2.friendly-neighbours.com org.fedoraproject.Anaconda.Boss[1072]: DEBUG:anaconda.modules.boss.module_manager:/org/fedoraproject/Anaconda/Modules/User has quit.
May 24 20:38:49 rpi3-2.friendly-neighbours.com org.fedoraproject.Anaconda.Boss[1072]: DEBUG:anaconda.modules.boss.module_manager:/org/fedoraproject/Anaconda/Modules/Services has quit.
May 24 20:38:49 rpi3-2.friendly-neighbours.com org.fedoraproject.Anaconda.Boss[1072]: DEBUG:anaconda.modules.boss.module_manager:/org/fedoraproject/Anaconda/Addons/Baz has quit.
May 24 20:38:50 rpi3-2.friendly-neighbours.com initial-setup[1143]: Initial Setup failed, keeping enabled
May 24 20:38:50 rpi3-2.friendly-neighbours.com systemd[1]: initial-setup.service: Main process exited, code=exited, status=1/FAILURE
May 24 20:38:50 rpi3-2.friendly-neighbours.com systemd[1]: initial-setup.service: Failed with result 'exit-code'.
May 24 20:38:50 rpi3-2.friendly-neighbours.com systemd[1]: Failed to start Initial Setup configuration program.

Comment 9 Paul Whalen 2018-05-24 21:08:45 UTC
Created attachment 1441244 [details]
Journalctl output with python3-kickstart-3.12-4.fc28

Comment 10 Paul Whalen 2018-05-24 22:59:23 UTC
Commenting line 42 from the kickstart (parted --grow) I was able to get initial-setup to start, but only the userspoke came up:

================================================================================
1) [ ] User creation
       (No user will be created)

Please make a selection from the above ['1' to enter the User creation spoke,
'c' to continue, 'q' to quit, 'r' to refresh]:

Comment 11 Paul Whalen 2018-05-24 22:59:49 UTC
Created attachment 1441300 [details]
journalctl -u initial-setup

Comment 12 Peter Robinson 2018-05-26 11:47:05 UTC
Martin: where do we go from here?

Comment 13 Fedora Update System 2018-05-26 20:43:14 UTC
pykickstart-3.12-4.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 14 Martin Kolman 2018-05-29 10:26:17 UTC
(In reply to Peter Robinson from comment #12)
> Martin: where do we go from here?

The new pykickstart needs to be added to environment that generates the image - adding it to the image itself is not enough & not actually needed. The issue is caused by incorrectly generated kickstart, so the environment that generates it (which in this case is the builder) is what needs the Pykickstart update.

Comment 15 Peter Robinson 2018-08-14 15:36:23 UTC
(In reply to Martin Kolman from comment #14)
> (In reply to Peter Robinson from comment #12)
> > Martin: where do we go from here?
> 
> The new pykickstart needs to be added to environment that generates the
> image - adding it to the image itself is not enough & not actually needed.
> The issue is caused by incorrectly generated kickstart, so the environment
> that generates it (which in this case is the builder) is what needs the
> Pykickstart update.

So the builders have pykickstart-3.12-4.fc28.noarch and we're still seeing just the user creation part so I don't believe this has been fixed

Comment 16 Paul Whalen 2018-12-04 14:57:56 UTC
This is now fixed in IoT, thanks mkolman!

Comment 17 Martin Kolman 2018-12-05 12:23:45 UTC
(In reply to Paul Whalen from comment #16)
> This is now fixed in IoT, thanks mkolman!

I'm glad we managed to track it down. :)