Bug 1340555

Summary: Emergency shell after undercloud install and reboot
Product: Red Hat OpenStack Reporter: Rama <rnishtal>
Component: rhosp-directorAssignee: Angus Thomas <athomas>
Status: CLOSED DUPLICATE QA Contact: Arik Chernetsky <achernet>
Severity: high Docs Contact:
Priority: unspecified    
Version: 8.0 (Liberty)CC: dbecker, ibojer, jdonohue, mburns, morazi, rhel-osp-director-maint, sreichar
Target Milestone: ---   
Target Release: 8.0 (Liberty)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-06 12:55:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1194008, 1295530    
Attachments:
Description Flags
journalctl output
none
sosreport none

Description Rama 2016-05-27 21:06:31 UTC
Created attachment 1162545 [details]
journalctl output

Description of problem:
After Installing OSP8 Director undercloud bits and rebooting the director node will take you to emergency shell.

Version-Release number of selected component (if applicable):
osp8

How reproducible:
Consistently

Steps to Reproduce:
1.Install RHEL 7.2 from dvd binary and select server with gui.
2.Run openstack undercloud install ( osp8) 
3.Reboot the node

Actual results:
Node goes to emergency shell 

Expected results:
Node should come up fine.

Additional info:

To reproduce the problem have done sequentially the following.
(1) Install RHEL 7.2 and do yum update. Reboot works fine.
(2) Install python-tripleoclient and yum update. System comes up fine.
(3) Create undercloud.conf and do openstack undercloud install and system does not come up.

Actually the system fails to shutdown also. However this is probably because of bug 1178497. If we apply the fix it shutsdown. 

Whether or not we apply the fix, it fails to come up.
system asks to enter password for maintenance.

===================
-- Unit boot.mount has finished starting up.
-- 
-- The start-up result is done.
May 27 14:47:53 osp8-director.cisco.com systemd[1]: Job dev-mapper-rhel\x2dhome.device/start timed out.
May 27 14:47:53 osp8-director.cisco.com systemd[1]: Timed out waiting for device dev-mapper-rhel\x2dhome.device.
-- Subject: Unit dev-mapper-rhel\x2dhome.device has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit dev-mapper-rhel\x2dhome.device has failed.
-- 
-- The result is timeout.
May 27 14:47:53 osp8-director.cisco.com systemd[1]: Dependency failed for /home.
-- Subject: Unit home.mount has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit home.mount has failed.

=============

Contents of fstab
/dev/mapper/rhel_osp7--director-root /                       xfs     defaults        0 0
UUID=983bca55-31f6-4bb9-a68e-23344886e692 /boot                   xfs     defaults        0 0
/dev/mapper/rhel_osp7--director-home /home                   xfs     defaults        0 0
/dev/mapper/rhel_osp7--director-swap swap                    swap    defaults        0 0

df output shows the following
Filesystem            1K-blocks    Used Available Use% Mounted on
/dev/mapper/rhel-root 104806400 4411172 100395228   5% /
devtmpfs              131837280       0 131837280   0% /dev
tmpfs                 131852668       0 131852668   0% /dev/shm
tmpfs                 131852668    1284 131851384   1% /run
tmpfs                 131852668       0 131852668   0% /sys/fs/cgroup
/dev/sda1                508588  162036    346552  32% /boot

Attaching journalctl.log file

Comment 2 Rama 2016-05-31 14:59:06 UTC
Created attachment 1163270 [details]
sosreport

Comment 3 Ivan Bojer 2016-05-31 21:04:09 UTC
Just confirming that we ran into exactly the same issue as above. After undercloud install and applying fix for BUG #1178497 (systemctl mask dracut-shutdown.service) we can reboot the system but end up in the emergency mode with x2dhome.device failure (therefore /home or anything cannot be mounted).

Comment 4 Steve Reichard 2016-06-01 12:09:26 UTC
I was able to reproduce this by making /home a separate LV and FS.

After install RHEL w/ the /home I reboot without problem, but then did an undercloud install and ran into the issue.

Comment 5 Steve Reichard 2016-06-01 12:30:18 UTC
I looked at the emergency shell and doing a lvscan found the homevol was not active,  I did a lvchange and was able to finish the boot.

Comment 6 Mike Burns 2016-06-06 12:55:10 UTC

*** This bug has been marked as a duplicate of bug 1323024 ***