Bug 1292215 - System upgrade from Fedora 21 to 23 plymouthd core dump while verifying, failed upgrade
System upgrade from Fedora 21 to 23 plymouthd core dump while verifying, fail...
Status: CLOSED EOL
Product: Fedora
Classification: Fedora
Component: dnf-plugin-system-upgrade (Show other bugs)
23
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Will Woods
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-16 13:24 EST by Raman Gupta
Modified: 2016-11-06 19:05 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-11-06 19:05:23 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
journalctl output for the upgrade (1.90 MB, text/plain)
2015-12-16 13:24 EST, Raman Gupta
no flags Details
journalctl output for upgrade on another system (909.83 KB, text/x-vhdl)
2016-01-06 10:23 EST, Raman Gupta
no flags Details
log from the system-upgrade run (1.32 MB, text/plain)
2016-03-28 08:09 EDT, Thekla Damaschke
no flags Details

  None (edit)
Description Raman Gupta 2015-12-16 13:24:23 EST
Created attachment 1106503 [details]
journalctl output for the upgrade

While doing a system upgrade from F21 to F23, plymouthd dumped core, and the system upgrade failed to complete. The system was left in a reasonably ok state, but I have no idea what upgrade actions did not complete. I have certainly had lots of issues with KDE5, but this might not be related.

I will attach the log for the upgrade boot from journalctl.
Comment 1 Zbigniew Jędrzejewski-Szmek 2015-12-17 08:25:04 EST
The last message from rpm is:

Dec 12 06:02:03 edison dnf[1263]: Verifying: hibernate-validator-5.0.1-2.fc21.noarch 5235/8033

It seems that the upgrade was essentially complete at that point.


Dec 12 06:02:06 edison systemd[1]: plymouth-start.service: Main process exited, code=dumped, status=6/ABRT
Dec 12 06:02:06 edison systemd[1]: plymouth-start.service: Unit entered failed state.
Dec 12 06:02:06 edison systemd[1]: plymouth-start.service: Failed with result 'core-dump'.
Dec 12 06:02:08 edison systemd-coredump[18586]: Process 454 (plymouthd) of user 0 dumped core.
                                                
                                                Stack trace of thread 454:
                                                #0  0x00007efda9c148d7 n/a (/usr/lib64/libc-2.20.so (deleted))

Dec 12 03:20:47 edison kernel: perl[16259]: segfault at 5 ip 00007f0f6b415e0e sp 00007fff9cd6cbd0 error 4 in LibXML.so[7f0f6b3
d3000+69000]
Dec 12 03:20:47 edison kernel: perl[16261]: segfault at 5 ip 00007f508f568e0e sp 00007ffcef84eee0 error 4 in LibXML.so[7f508f5
26000+69000]
Dec 12 03:20:47 edison dnf[1263]: Upgrading   : perl-XML-SAX-0.99-15.fc23.noarch                      1599/8033
Dec 12 03:20:47 edison dnf[1263]: /var/tmp/rpm-tmp.P8SL2P: line 1: 16259 Segmentation fault      (core dumped) perl -MXML::SAX
 -e "XML::SAX->add_parser(q($p))->save_parsers()" 2> /dev/null
Dec 12 03:20:47 edison dnf[1263]: /var/tmp/rpm-tmp.P8SL2P: line 1: 16261 Segmentation fault      (core dumped) perl -MXML::SAX
 -e "XML::SAX->add_parser(q($p))->save_parsers()" 2> /dev/null
Dec 12 03:20:48 edison dnf[1263]: Upgrading   : perl-XML-LibXML-1:2.0121-6.fc23.x86_64                1600/8033
Dec 12 03:20:48 edison systemd-coredump[16260]: Process 16259 (perl) of user 0 dumped core.
                                                
                                                Stack trace of thread 16259:
                                                #0  0x00007f0f6b415e0e n/a (/usr/lib64/perl5/vendor_perl/auto/XML/LibXML/LibXML.so)
Dec 12 03:20:49 edison systemd-coredump[16262]: Process 16261 (perl) of user 0 dumped core.
                                                
                                                Stack trace of thread 16261:
                                                #0  0x00007f508f568e0e n/a (/usr/lib64/perl5/vendor_perl/auto/XML/LibXML/LibXML.so)

This smells like memory corruption issue of some sort. Did you run memtest?
Comment 2 Raman Gupta 2015-12-17 09:14:40 EST
Thanks, I'll run memtest tonight overnight to check.
Comment 3 Raman Gupta 2015-12-18 11:28:58 EST
Ran memtest overnight. In 10 hours and a full pass, there were no errors. Is there anything else that could cause this problem?
Comment 4 Zbigniew Jędrzejewski-Szmek 2015-12-18 11:42:51 EST
I have no futher ideas. I can imagine that if the one of the libraries was corrupted in the filesystem or in memory we could see effect like that, but there's nothing in the logs you attached to support this theory.

The scriptlet for perl-XML-LibXML is:
if [ $1 -eq 0 ] ; then
  for p in XML::LibXML::SAX::Parser XML::LibXML::SAX ; do
    perl -MXML::SAX -e "XML::SAX->remove_parser(q($p))->save_parsers()" \
      2>/dev/null || :
  done
fi

Both this and plymouth segfaulted, and those things seem unrelated.
Comment 5 Raman Gupta 2015-12-18 12:45:56 EST
Yes its very strange. I'm gonna run Prime95 to see if it shows any hardware issues.

Can the upgrade process deal more gracefully with a failure in plymouth? Perhaps rather than simply failing and rebooting, the user could be asked whether they wish to continue, or be dumped to a shell for debugging?
Comment 6 Zbigniew Jędrzejewski-Szmek 2015-12-18 13:49:47 EST
It should be fine with plymouth going away, I don't know what started the shutdown sequence.

There's definitely a bug in the service file, and the upgrade process should have DefaultDependencies=no. It's on my todo list ;)
Comment 7 Raman Gupta 2016-01-06 10:23 EST
Created attachment 1112209 [details]
journalctl output for upgrade on another system

I got the plymouth core dump on a second system being upgraded from F21 to F23.
Comment 8 Thekla Damaschke 2016-03-28 08:09 EDT
Created attachment 1140839 [details]
log from the system-upgrade run

It seems like I have been hit by this as well upgrading a KDE-spin system from F21 to F23. The new bootmeny entry has been created but I cant use it, all I get a is a blck screen with a huge cross as mouse pointer after a couple of minutes.
The old kernel seems to work so far with all the new packages that were installed.
Comment 9 Raman Gupta 2016-11-06 19:05:23 EST
Did not have this issue on upgrade Fedora 23 -> 24. Closing.

Note You need to log in before you can comment on or make changes to this bug.