Red Hat Bugzilla – Bug 982007
system does not boot with separate /var partition
Last modified: 2014-06-19 19:46:34 EDT
Description of problem:
When I put /var on a separate partition the system hangs at boot.
Version-Release number of selected component (if applicable):
Create two volume groups: vgSystem and vgData.
/etc/fstab like this:
/dev/mapper/vgSystem-root / ext4 defaults 1 1
/dev/mapper/vgData-var /var ext4 defaults 1 1
/dev/mapper/vgData-home /home ext4 defaults 1 1
/dev/mapper/vgSystem-swap swap swap defaults 0 0
UUID=a9c10b79-6d97-4bfb-9ffb-621cc05cc599 /boot ext4 defaults 1 2
UUID=925B-7095 /boot/efi vfat umask=0077,shortname=winnt 0 0
In this scenario vgSystem partitions are on an SSD HDD and vgData partitions on a SATA HDD.
Steps to Reproduce:
1. Boot the PC
System hangs at message:
Binary verified by the vendor certificate
Normal boot ending up on Gnome Desktop
This even happens when only /var/log is put to the separate partition
Also I noticed that after several resets of the PC it sometimes works.
On the other hand: if /var is on the same partition with / the boot works most of the times but some times hangs (so in this case it is just the other way round).
What is your kernel command line?
Does it help, if you add "rd.lvm.vg=vgData" to the kernel command line?
Without the workaround, to debug the issue:
Can you wait 5 minutes and get a login shell?
Here you can see what is in my grub.cfg:
linuxefi /vmlinuz-3.9.9-301.fc19.x86_64 root=/dev/mapper/vgSystem-root ro rd.md=0 rd.dm=0 rd.lvm.lv=vgSystem/swap rd.lvm.lv=vgSystem/root rd.lvm.vg=vgData rd.luks=0 vconsole.font=latarcyrheb-sun16 vconsole.keymap=de rhgb quiet LANG=en_US.utf8
Waiting for 5 min does not bring anything up.
Putting the whole /var partition to another disk makes booting very unlikely. I have to reset my machine up to perhaps ten times in order to make it boot.
As you can see here I have added your suggestion for at test. But unfortunately it is not better than before.
If I put only /var/log to another disk booting becomes more likely. Perhaps ever second attempt fails.
Where exactly does it hang? YOu should see an animation in the textboot progress that tells you what is being waited for.
No unfortuantely it hangs at this message:
"Binary verified by the vendor certificate". At this point there is no GUI.
There is always some probability that it might boot or hang.
What I did not test so far is to disable secure boot with /var on a separate disk. Maybe I will test this. After UEFI update from 1101 to 1204 (ASUS board P8B75-M LE) secure boot is not working anymore. I have opened bug 1008084 for this and also wrote a complaint to ASUS.
I know this is another story - but this gave me idea to test with secure boot disabled. But I still have to do this.
But to be honest I doubt that the issue with the /var partiton has something to do with secure boot because as far as I understood nothing what is needed for secure boot is in /var.
Fixed. I changed to Arch Linux. There it works as expected.
systemd doesn't seem to be mounting additional mount points early enough. I am effected by this as well, but my system usually boots although some processes do not start, or are started before /var has been mounted and therefore are writing to the masked directory.
/dev/mapper/VolGrp01-root / ext4 defaults,_netdev 1 1
UUID=3041eab9-f5b9-4086-bd79-c9db4538481f /boot ext4 defaults,_netdev 1 2
/dev/mapper/VolGrp01-var /var ext4 defaults,_netdev 1 2
macross:/storage /storage nfs4 rw 0 0
macross:/home /home nfs4 rw 0 0
And the boot.log which I had to do a bind-mount in order to retrieve as it was written to the VolGrp01-root partition and not the VolGrp01-var.
[root@htpc log]# cat /mnt/tmp/var/log/boot.log
[ OK ] Started Show Plymouth Boot Screen.
[ OK ] Reached target Paths.
[ OK ] Reached target Basic System.
[ 14.198817] dracut-initqueue: iscsistart: version 184.108.40.2063-1
[ 14.753990] dracut-initqueue: iscsistart: Connection1:0 to [target: iqn.2008-06.localdomain.macross:htpc_fedora.root, portal: 192.168.0.1,3260] through [iface: default] is operational now
[ OK ] Started dracut initqueue hook.
[ OK ] Mounted /sysroot.
[ OK ] Reached target Initrd Root File System.
Starting Reload Configuration from the Real Root...
[ OK ] Started Reload Configuration from the Real Root.
[ OK ] Reached target Initrd File Systems.
[ OK ] Reached target Initrd Default Target.
Welcome to Fedora 19 (Schrödinger’s Cat)!
[ OK ] Stopped Switch Root.
[ OK ] Stopped target Switch Root.
[ OK ] Stopped target Initrd File Systems.
[ OK ] Stopped target Initrd Root File System.
Expecting device dev-disk-by\x2duuid-3041eab9\x2df5b9\x2d4086\x2dbd79\x2dc9db4538481f.device...
[ OK ] Listening on Syslog Socket.
Starting Collect Read-Ahead Data...
Starting Replay Read-Ahead Data...
[ OK ] Listening on Delayed Shutdown Socket.
[ OK ] Listening on /dev/initctl Compatibility Named Pipe.
Starting Create static device nodes in /dev...
[ OK ] Reached target Paths.
Mounting Debug File System...
[ OK ] Set up automount Arbitrary Executable File Formats File System Automount Point.
Mounting POSIX Message Queue File System...
[ OK ] Listening on udev Kernel Socket.
[ OK ] Listening on udev Control Socket.
Starting udev Coldplug all Devices...
Mounting Huge Pages File System...
Expecting device dev-mapper-VolGrp01\x2dvar.device...
[ OK ] Reached target Encrypted Volumes.
[ OK ] Listening on LVM2 metadata daemon socket.
[ OK ] Listening on Device-mapper event daemon FIFOs.
Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
[ OK ] Reached target Swap.
Mounting Temporary Directory...
[ OK ] Stopped Trigger Flushing of Journal to Persistent Storage.
Stopping Journal Service...
[ OK ] Stopped Journal Service.
Starting Journal Service...
[ OK ] Started Journal Service.
[ OK ] Started Collect Read-Ahead Data.
[ OK ] Started Replay Read-Ahead Data.
Starting File System Check on Root Device...
Starting Apply Kernel Variables...
Mounting Configuration File System...
[ OK ] Mounted Temporary Directory.
[ OK ] Mounted Debug File System.
[ OK ] Mounted POSIX Message Queue File System.
[ OK ] Mounted Huge Pages File System.
[ OK ] Started File System Check on Root Device.
Starting Remount Root and Kernel File Systems...
Starting LVM2 metadata daemon...
[ OK ] Mounted Configuration File System.
[ 16.308897] systemd-fsck: /dev/mapper/VolGrp01-root: clean, 70046/640848 files, 449229/2560000 blocks
[ OK ] Started udev Coldplug all Devices.
[ OK ] Started Create static device nodes in /dev.
Starting udev Kernel Device Manager...
[ OK ] Started LVM2 metadata daemon.
[ OK ] Started Apply Kernel Variables.
[ OK ] Started udev Kernel Device Manager.
[ OK ] Started Remount Root and Kernel File Systems.
Starting Configure read-only root support...
Starting Import network configuration from initramfs...
[ OK ] Reached target Local File Systems (Pre).
[ OK ] Started Import network configuration from initramfs.
[ OK ] Started Configure read-only root support.
[ OK ] Started Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
[ OK ] Reached target Local File Systems.
Starting Tell Plymouth To Write Out Runtime Data...
Starting Security Auditing Service...
Starting Recreate Volatile Files and Directories...
[ OK ] Started Recreate Volatile Files and Directories.
[FAILED] Failed to start Security Auditing Service.
See 'systemctl status auditd.service' for details.
[ OK ] Started Tell Plymouth To Write Out Runtime Data.
[ OK ] Reached target System Initialization.
[ OK ] Reached target Timers.
[ OK ] Reached target Basic System.
Starting irqbalance daemon...
[ OK ] Started irqbalance daemon.
Starting Self Monitoring and Reporting Technology (SMART) Daemon...
[ OK ] Started Self Monitoring and Reporting Technology (SMART) Daemon.
Starting Machine Check Exception Logging Daemon...
Starting System Logging Service...
[ OK ] Started System Logging Service.
Starting Hardware RNG Entropy Gatherer Daemon...
[ OK ] Started Hardware RNG Entropy Gatherer Daemon.
[ INFO ] Network Manager is not active.
[DEPEND] Dependency failed for Network Manager Wait Online.
[ OK ] Reached target Network is Online.
[ OK ] Reached target Network.
Starting Sendmail Mail Transport Agent...
Starting OpenSSH server daemon...
[ OK ] Listening on Open-iSCSI iscsiuio Socket.
[ OK ] Listening on Open-iSCSI iscsid Socket.
[ OK ] Started Machine Check Exception Logging Daemon.
[ OK ] Started OpenSSH server daemon.
[ OK ] Mounted /storage.
[FAILED] Failed to start Sendmail Mail Transport Agent.
See 'systemctl status sendmail.service' for details.
[DEPEND] Dependency failed for Sendmail Mail Transport Client.
[ OK ] Mounted /home.
[ OK ] Found device htpc_fedora.
Starting File System Check on /dev/disk/by-uuid/3041eab9-f5b9-4086-bd79-c9db4538481f...
[ 17.099506] systemd-fsck: /dev/sde1: clean, 315/128016 files, 125372/512000 blocks
[ OK ] Started File System Check on /dev/disk/by-uuid/3041eab9-f5b9-4086-bd79-c9db4538481f.
[ OK ] Mounted /boot.
[ OK ] Found device /dev/mapper/VolGrp01-var.
Starting File System Check on /dev/mapper/VolGrp01-var...
[ 17.255581] systemd-fsck: /dev/mapper/VolGrp01-var: clean, 2253/320000 files, 124691/1280000 blocks
[ OK ] Started File System Check on /dev/mapper/VolGrp01-var.
[ OK ] Mounted /var.
[ OK ] Listening on Avahi mDNS/DNS-SD Stack Activation Socket.
[ OK ] Listening on RPCbind Server Activation Socket.
Starting RPC bind service...
[ OK ] Listening on PC/SC Smart Card Daemon Activation Socket.
[ OK ] Reached target Remote File Systems.
Starting Trigger Flushing of Journal to Persistent Storage...
[ OK ] Started RPC bind service.
[ OK ] Started Trigger Flushing of Journal to Persistent Storage.
Starting Permit User Sessions...
[ OK ] Started Permit User Sessions.
Starting Job spooling tools...
[ OK ] Started Job spooling tools.
Starting Command Scheduler...
[ OK ] Started Command Scheduler.
Starting Terminate Plymouth Boot Screen...
Starting Wait for Plymouth Boot Screen to Quit...
Fedora 20 has systemd version 208. This version seems to fix this issue. I just installed Fedora 20 and it works even if the /var partition is on a different disk.
OK, closing then.