Bug 841822

Summary: pungi can't create installable media with F17 + updates
Product: [Fedora] Fedora Reporter: Jos Vos <jos>
Component: systemdAssignee: Michal Schmidt <mschmidt>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 17CC: dennis, johannbg, jreiser, lnykryn, lpoetter, metherid, mschmidt, msekleta, notting, plautrba, systemd-maint, vpavlin
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-20 15:04:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Ppungi config file
none
Screenshot tty1 DVD F17 release + updates
none
Screenshot tty1 DVD F17 release none

Description Jos Vos 2012-07-20 10:09:07 UTC
Created attachment 599352 [details]
Ppungi config file

Description of problem:
When creating a spin with pungi using the F17 release repo AND the F17 updates repo, the result is a DVD that (a) gives a timeout (after a long time) when trying to spawn a shell on tty2 and (b) runs into a Python exception a few steps later.  Creating a spin with the same parameters and a config file using ONLY the F17 release repo creates a DVD that works fine.

Version-Release number of selected component (if applicable):
pungi-2.11-2.fc17

How reproducible:
pungi --name=Fedora --ver=17 --nosource --nodebuginfo -c ksf17upd.cfg --all-stages --destdir=`pwd`/out00rel

I've attached the following files:

-  The config file ksf17upd.cfg I'm using.  Note that the same config file works fine when commenting out the f17-updates repo.

-  A screenshot f17upd.png of tty1 when installing the wrong DVD, the one with all updates included (booting w/o quiet).

-  A screenshot f17rel.png of tty1 when installing the good DVD, the one with no updates included (booting w/o quiet).

Note the major differences in the screenshots (systemd related?).

I get a Python exception after sekecting the language: GError: Couldn't recignize the image file format for file '/usr/share/system-config-keyboard/...

Comment 1 Jos Vos 2012-07-20 10:10:59 UTC
Created attachment 599353 [details]
Screenshot tty1 DVD F17 release + updates

Comment 2 Jos Vos 2012-07-20 10:12:07 UTC
Created attachment 599354 [details]
Screenshot tty1 DVD F17 release

Comment 3 Jos Vos 2012-07-20 10:16:24 UTC
I forgot to mention that I'm testing the media using KVM virtual machines on a CentOS 6 host.

I'm running pungi on a F17 system with all updates applied.

Others also confirmed that they ran into the same problems using pungi for updated F17 spins.

Comment 4 Dennis Gilmore 2012-07-21 13:09:26 UTC
considering that pungi hasnt changed since ga was made, and that the image does boot, pungi is doing its job. the bug will be in some component installed into the image. something in updates has caused the regression.

Comment 5 John Reiser 2012-07-22 14:49:01 UTC
I nominate systemd-44-17.fc17.x86_64 as the first culprit.  An install DVD built from f17-release plus f17-updates minus {systemd-44-17.fc17, systemd-sysv-44-17.fc17}, does boot and does instantiate the shell on VT2.  This is progress at isolating bugs.  [After choosing the language then I still see the Python exception, but now I can use VT2 to capture the anaconda info.]

Detailed methodoolgy:
1. Compose a DVD using ksf17ga.cfg, which names only
  repo --name=f17-release ...
 Verify that the DVD boots, brings up shell on VT2, and gets beyond choosing the language without a Python exception.
2. Save the list of packages in the anaconda installer system:
   grep 'installed successfully' $DESTDIR/logs/x86_64.log \
 | sed -e 's/.*DEBUG: //' -e 's/ installed successfully.*//' \
 | sed -e 's/[^:]*://' \
 | sort > anaconda-install.release

3. Compose a DVD using ksf17upd.cfg, which names both -release and -updates repos:
  repo --name=f17-release ...
  repo --name=f17-updates ...
 Verify that the DVD boots, but fails to bring up shell on VT2, and gets the Python exception after selecting language.
4. Save the list of packages in the anaconda installer system:
   grep 'installed successfully' $DESTDIR/logs/x86_64.log \
 | sed -e 's/.*DEBUG: //' -e 's/ installed successfully.*//' \
 | sed -e 's/[^:]*://' \
 | sort > anaconda-install.updates

5. Generate the list of updated packages used by the installer:
  comm -13 anaconda-install.release anaconda-install.updates > anaconda-install.updnew

6. Guess that systemd is a culprit.  Generate a new file:// repo which is -updates minus the new systemd and anything that requires it:
 (  cd /var/cache/pungi/f17-updates
    echo                     systemd-sysv-44-17.fc17.x86_64
    echo                     systemd-44-17.fc17.x86_64
    repoquery --whatrequires systemd-44-17.fc17.x86_64
 ) \
 | sort \
 | comm -23 anaconda-install.updnew - \
 | \
 (  mkdir -p /var/cache/pungi/my-repo
    cd /var/cache/pungi/my-repo
    rm -f *.sqlite packages/*
    while read pkg; do
      cp ../f17-updates/packages/$pkg.rpm packages
   done
   createrepo .
 ) 
7. Compose a DVD using -release plus the private file:// repo:
  repo --name=f17-release ...
  repo --name=my-repo --baseurl=file:///var/cache/pungi/my-repo --cost=1
 Verify that the DVD boots, and gives a shell on VT2, but gets a Python exception after choosing language.
8. Save the list of packages in the anaconda installer system:
   grep 'installed successfully' $DESTDIR/logs/x86_64.log \
 | sed -e 's/.*DEBUG: //' -e 's/ installed successfully.*//' \
 | sed -e 's/[^:]*://' \
 | sort > anaconda-install.guess

9. Find the package changes in the anaconda installer system:
 diff -u anaconda-install.updates anaconda-install.guess
@@ -508,8 +508,8 @@
  system-config-keyboard-1.3.1-8.fc17.x86_64
  system-config-keyboard-base-1.3.1-8.fc17.x86_64
  system-config-users-1.2.115-1.fc17.noarch
 -systemd-44-17.fc17.x86_64
 -systemd-sysv-44-17.fc17.x86_64
 +systemd-44-8.fc17.x86_64
 +systemd-sysv-44-8.fc17.x86_64
  system-setup-keyboard-0.8.8-2.fc17.x86_64
  sysvinit-tools-2.88-6.dsf.fc17.x86_64

 which indicts systemd-44-17.

Jos, can you reproduce this?  If so, the re-open this bug, but with systemd sa the component.

Comment 6 John Reiser 2012-07-22 14:54:17 UTC
Jos, you may have to empty the $DESTDIR/logs/x86_64.log before each compose; otherwise the log might just append.  I save all my logs individually:
  SAV=$(date "+%m%d.%H%M")
  ( cd $DESTDIR/logs; mv $ARCH.log $ARCH.log.old$SAV
    mv pylorax.log pylorax.log.old$SAV
  )

Comment 7 Jos Vos 2012-07-22 16:15:37 UTC
> Jos, can you reproduce this?  If so, the re-open this bug, but with systemd
> sa the component.

Thanks for your analysis.  I didn't have time to follow-up on this problem, but I'm definitely planning to do so.

I think I'll have time to check it tomorrow.  Did you look at the screenshots I attached to this bug?  The startup sequence (i.e. green "OK" at the end vs. at the start of a line) looks completely different, which already made me suspect systemd.

BTW, the Python exception might be related to a missing icon in /usr/share/system-config-keyboard/, which might be more easy to solve, although both anaconda and system-config-keyboard do not seem to be updated after F17 GA. Anyway, when the first problem is solved, the second one is probably an easy one.

In any case, this bug should be reopened in some way, as it is a real bug (so NOTABUG does not apply), although the exact component(s) are not known yet.

Comment 8 Jos Vos 2012-07-24 13:51:06 UTC
I reopen this bug, but now for systemd.

When systemd-44-17.fc17.x86_64.rpm and systemd-sysv-44-17.fc17.x86_64.rpm are removed from (a local copy of) the updates repository, pungi creates a DVD (including systemd-44-8.fc17.x86_64.rpm and systemd-sysv-44-8.fc17.x86_64.rpm) that does not have the shell spawning problem.

Comment 9 Michal Schmidt 2012-07-24 15:09:06 UTC
I haven't tested it, but I'm pretty sure it's caused by:
0115-getty-VC-devices-are-always-available-we-don-t-need-.patch

http://pkgs.fedoraproject.org/gitweb/?p=systemd.git;a=blob_plain;f=0115-getty-VC-devices-are-always-available-we-don-t-need-.patch;h=9c9a3ea0089af835ec2db54b6203a434b66aa04b;hb=f17

We can either revert this in systemd, or apply a similar modification to anaconda's anaconda-shell@.service.

Comment 10 Michal Schmidt 2012-07-24 15:15:35 UTC
I see anaconda has applied it upstream already:
http://git.fedorahosted.org/git/?p=anaconda.git;a=commit;h=2e4ab59ebb63c17995a8e1f6b4c37016319f6503

But maybe there were other users of dev-ttyX.device, so I'll revert the udev rule change in F17 to be on the safe side.

Comment 11 Jos Vos 2012-07-25 13:35:09 UTC
For the record: the other problem mentioned here earlier (Python exception) is now handled in bug #843062 (https://bugzilla.redhat.com/show_bug.cgi?id=843062).

Comment 12 John Reiser 2012-08-21 16:39:29 UTC
(In reply to comment #10)

Ping?  It's been four weeks, but no work on this problem has appeared in builds.  The last f17 build was two months ago.  There were builds of systemd-188 for f18 and f19 last week.

Comment 13 Hedayat Vatankhah 2012-10-10 21:21:08 UTC
I confirm that the Anaconda patch will fix the problem. I think an update for Fedora 17's anaconda is more appropriate than a systemd update.

Comment 14 Fedora Update System 2012-10-12 16:46:52 UTC
systemd-44-20.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/systemd-44-20.fc17

Comment 15 Fedora Update System 2012-10-13 00:21:21 UTC
Package systemd-44-20.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-44-20.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-16035/systemd-44-20.fc17
then log in and leave karma (feedback).

Comment 16 Fedora Update System 2012-12-20 15:04:56 UTC
systemd-44-20.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.