Bug 752846 - Services don't start after upgrade from Fedora 15 to 16
Summary: Services don't start after upgrade from Fedora 15 to 16
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: 16
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-11-10 15:40 UTC by Nadav Har'El
Modified: 2011-12-19 06:22 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-11-10 15:59:22 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Nadav Har'El 2011-11-10 15:40:40 UTC
I had a Fedora 15 system, running the httpd and mysqld services fine (and of course, starting them on boot).

After upgrading (with "preupgrade") to Fedora 16, these two services did not start on boot. system-config-services didn't even show them. When I manually typed "service httpd start" and "service mysqld start", both services started, and also started showing in system-config-services.

This is a pretty serious bug for server machines, because it means significantly increased downtime during upgrade, while the system administrator tries to figure out how to return these services.

Comment 1 Michal Schmidt 2011-11-10 15:59:22 UTC
systemctl enable mysqld.service httpd.service
(or you can use chkconfig)

The services got converted from SysV initscripts to systemd in F16. It is intentional that service enablement status is not migrated. It is mandated by the packaging guidelines ( http://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Packages_migrating_to_a_systemd_unit_file_from_a_SysV_initscript )
The reason for that is basically that SysV runlevels do not exactly map 1:1 to systemd targets, so automated migration could not be reliable.

I don't know if it is in the release notes, but it should have been.
Since this is likely to surprise more users, I'm going to add it to http://fedoraproject.org/wiki/Common_F16_bugs

Comment 2 Nadav Har'El 2011-11-10 16:14:24 UTC
I think the decision not to migrate service enablement is problematic, because 1. I didn't see it documented (or I missed it while reading the release notes), 2. The upgraded machine "appears" to work, and it takes time for the administrator to suddenly realise that important services which always worked, suddenly don't. And when he does realize that - he doesn't know how to enable it! And why doesn't "system-config-services" let me enable these services and I must use these command lines?? Can you at least consider fixing that?

I've been using Unix heavily for 27 (!) years, and this is the first time I've heard of this "systemctl" command, which is a fairly recent addition to Fedora. I'm sure other people will be surprised as well :-)

By the way, I realize the thing about runlevels, but the current method is even less reliable (it basically means you convert "running in run levels x, y, z" to "never running" in the migration!). If you ask me, my suggestion is to only look at the default run level before and after the migration, and everything that ran on the default run level before the migration, should also run by default after the migration.

Thanks.

Comment 3 Michal Schmidt 2011-11-10 16:21:25 UTC
(In reply to comment #2)
> And why doesn't "system-config-services" let me enable these services and I
> must use these command lines??

I don't know. I do not use this utility. Please file a bug against system-config-services.

> I've been using Unix heavily for 27 (!) years, and this is the first time I've
> heard of this "systemctl" command, which is a fairly recent addition to Fedora.
> I'm sure other people will be surprised as well :-)

The traditional command "chkconfig" still works (as a compatibility wrapper for systemctl).

> my suggestion is to only look at the default run level before and after
> the migration, and everything that ran on the default run level before
> the migration, should also run by default after the migration.

Something like that was considered when creating the guidelines, but in the end FESCo/FPC decided against it. If you're curious, see https://fedorahosted.org/fpc/ticket/31

I have added the note about this issue to the page:
https://fedoraproject.org/wiki/Common_F16_bugs#Upgrade_from_previous_releases_resets_the_enablement_status_of_services

Comment 4 Oliver Henshaw 2011-11-10 16:55:05 UTC
Is there a list of services that have been converted from sysv to systemd in the f15 -> f16 cycle, i.e. those that need checking? The closest I can find is https://bugzilla.redhat.com/showdependencytree.cgi?id=713562&hide_resolved=0 but the bug title don't contain the service or package names, so that's no use.

Comment 5 Michal Schmidt 2011-11-10 17:08:17 UTC
(In reply to comment #4)

From there you can click "view as bug list" and then "Change Columns" to add the column for "Component".
But I think that it is easier for the user to check "systemctl list-unit-files" on his system.

Comment 6 Michal Schmidt 2011-11-10 17:10:56 UTC
Also I'm not sure if that tracker bug is complete. Maintainers may have migrated their packages without filing a bug or adding it to the tracker.

Comment 7 Oliver Henshaw 2011-11-10 17:26:31 UTC
Right, but "systemctl list-unit-files" will also list those services that were already converted in F15.

I think grepping(*) /var/log/boot.log and /var/log/boot.log-$date-before-update' will get me lists I can 'sort' and 'comm'pare:

* 'egrep "Starting " /var/log/boot.log | egrep "SYSV|LSB"' I think should be enough, though I might also need to compare 'egrep -v "SYSV|LSB"' to catch services that have enabled themselves in the upgrade.

Comment 8 Nadav Har'El 2011-11-10 20:41:52 UTC
Unfortunately I found more services which have stopped after the upgrade, after httpd and mysqld, which I already mentioned.

The other services were "cups", "smb", and "nmb".

Sadly I didn't notice these were gone, and when a remote user cried about being unable to print, I had to figure out which services have disappeared; For example, since it's been more than a year since I originally configured this machine, I didn't immediately remember that "nmb", not just "smb", was necessary for having remote Windows users print on the machine.

Comment 9 edwinh 2011-11-11 18:15:47 UTC
This is very annoying.  Sorry to see this just plain "closed wontfix".  Understand the problem/policy as intended, but need to keep in mind the good practice of least surprise for the end user.  

Lots of people upgrade and expect things like services previously enabled to still run for the system to come up correctly (ypbind, nfs..) - as has been the case for any other fedora upgrade.  Have to spend time figuring out whats missing before I go mass-upgrade the rest of my network as usual.

Comment 10 Bill Nottingham 2011-11-11 18:32:07 UTC
The issue is that it can't be fixed in the systemd package - it needs to be fixed in the inidividual packages that ship services to handle the upgrade transition.

Comment 11 Michal Schmidt 2011-11-14 09:02:20 UTC
It needs to, but it mustn't, because FPC forbade it in the packaging guidelines.

Comment 12 Trevor Cordes 2011-12-17 10:49:45 UTC
Perhaps release notes and upgrade wikis should have a big note:

Before upgrading, run:

chkconfig --list |grep on > ~/previously-running-services

That way one can know out of the 50+ services on their box which ones they were previously running after they upgrade.  If you don't do that, you will have to walk through each one, one by one, trying to remember if you were running it!

Comment 13 Oliver Henshaw 2011-12-17 13:32:40 UTC
As noted in comment #7, you can also retrospectively get this information from /var/log/boot.log-$date-before-update (at least if you do it before the old logs are rotated out).

Comment 14 Michal Schmidt 2011-12-19 06:22:41 UTC
You can also take a look at /var/lib/systemd/sysv-convert/database where the previous enablement state of services following the packaging guidelines is saved.


Note You need to log in before you can comment on or make changes to this bug.