Bug 524727 - Dracut + encrypted root + networking
Dracut + encrypted root + networking
Status: ASSIGNED
Product: Fedora
Classification: Fedora
Component: dracut (Show other bugs)
rawhide
All Linux
low Severity low
: ---
: ---
Assigned To: dracut-maint
Fedora Extras Quality Assurance
: FutureFeature
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-09-21 18:25 EDT by Maciej Żenczykowski
Modified: 2017-07-15 09:12 EDT (History)
18 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Maciej Żenczykowski 2009-09-21 18:25:53 EDT
It should be possible to boot a system with an encrypted root filesystem without physical console access.

Envision:
* remote power on (or reboot after power failure)
* bios -> hdd bootsector -> grub, timeout -> kernel + initramfs
* initramfs contains:
  - enough to bring up the network (kernel modules + dhclient or static config) + sshd or dropbear, including private host keys, possibly different (configuration option) from the actual systems ssh host keys (possibly running on a different port)
  - possibly include screen so the boot prompts for passwords can be accessed remotely, alternatively provide a different way to pass the password into the boot process
  - once passwords have been provided and root file system (and others) have been mounted, the network configuration/screen can be unconfigured and normal boot can continue.

(obviously the default would have to be to build the initramfs without this sort of support)
Comment 1 Maciej Żenczykowski 2009-09-21 18:31:23 EDT
(I do realize that from a security perspective this is not as safe as providing your password directly via the console, but in many cases the utility wins)
Comment 2 Bug Zapper 2009-11-16 07:44:05 EST
This bug appears to have been reported against 'rawhide' during the Fedora 12 development cycle.
Changing version to '12'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 3 Harald Hoyer 2010-04-15 09:02:52 EDT
Feel free to write such a dracut module and submit it to the initramfs mailing list. I might include it upstream.
Comment 4 Fedora Admin XMLRPC Client 2011-10-20 12:19:31 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 5 James Heather 2011-11-18 06:40:48 EST
Is anything likely to happen about this? It would be a really neat feature.

Looks like there are web pages detailing how to do it on Debian. We already have a network module for dracut, so I would guess an ssh module wouldn't be too hard to add.
Comment 6 Michael Scherer 2012-05-12 07:22:37 EDT
The potential security problem could be solved by :
1) making sure that you can only use a ssh key to log,taken from some location on the server
2) reusing the ssh key from the server, so there is no fingerprinting issue. But then, the initramfs must be secured like would the keys, ie, readable only by root, etc. This could create problem, and for sure, would requires to be cautious.

From what I gathered, Debian does 1, not 2.
http://blog.neutrino.es/2011/unlocking-a-luks-encrypted-root-partition-remotely-via-ssh/
Comment 7 Maciej Żenczykowski 2012-05-13 04:28:49 EDT
Thanks for the pointer.
Using that information I was able to get a *very* hacky - but working - remote unlock.

Here's what I did:

# Create a backup copy of our current initramfs, If we screw it up we can always edit the grub boot line and add an underscore to the initramfs image to use this backup copy
cp /boot/initramfs-`uname -r`.img /boot/initramfs-`uname -r`.img_

# We'll need dropbear to provide sshd
yum install dropbear

# This is where dracut's modules live
cd /usr/share/dracut/modules.d

# We're going to create a module
mkdir 40maze
cd 40maze

# Generate public/private DSS and RSA keys for the server.
dropbearkey -t dss -f dropbear_dss_host_key
dropbearkey -t rsa -f dropbear_rsa_host_key

# Create some trivial config files
echo 'multi on' > host.conf
echo -en '127.0.0.1\tlocalhost\n::1\tlocalhost\n' > host
echo 'root:x:0:0:root:/home/root:/bin/sh' > passwd
echo '/bin/sh' > shells

...now you need to create some less trivial config files, cut and paste each of the following dumps...

=== module-setup.sh ===
#!/bin/bash
# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
# ex: ts=8 sw=4 sts=4 et filetype=sh

check() {
    # do not add this module by default: return 255
    return 0
}

depends() {
    return 0
}

installkernel() {
    instmods eth0
}

install() {
    dracut_install -o ip

    dracut_install /usr/lib64/libnsl.so
    dracut_install /lib64/libnss_compat.so.2
    dracut_install /lib64/libnss_files.so.2
    dracut_install /lib64/libnss_dns.so.2
    dracut_install -o dropbear

    inst_dir "/etc/dropbear"
    inst "${moddir}/dropbear_dss_host_key" "/etc/dropbear/dropbear_dss_host_key"
    inst "${moddir}/dropbear_rsa_host_key" "/etc/dropbear/dropbear_rsa_host_key"

    inst_dir "/home"
    inst_dir "/home/root"
    inst_dir "/home/root/.ssh"
    inst "${moddir}/authorized_keys" "/home/root/.ssh/authorized_keys"
    #inst "/root/.ssh/authorized_keys" "/home/root/.ssh/authorized_keys"

    inst "/etc/localtime"
    inst "${moddir}/nsswitch.conf" "/etc/nsswitch.conf"
    inst "${moddir}/resolve.conf" "/etc/resolve.conf"
    inst "${moddir}/host.conf" "/etc/host.conf"
    inst "${moddir}/hosts" "/etc/hosts"
    inst "${moddir}/passwd" "/etc/passwd"
    inst "${moddir}/shells" "/etc/shells"
    inst_hook pre-trigger 01 "$moddir/remote-ssh.sh"

    inst "${moddir}/auth" "/bin/auth"
    inst "${moddir}/tiocsti" "/bin/tiocsti"

    # the majority of the stuff below here is useful for debugging, but probably not needed
    dracut_install -o ps find lsof grep egrep sed less more cat tac head tail true false mkdir rmdir rm strace touch vi ip ping ping6 traceroute ssh scp

    #dracut_install -o fsck fsck.ext2 fsck.ext4 fsck.ext3 fsck.ext4dev fsck.vfat e2fsck
}

=== nsswitch.conf ===
passwd:     files
shadow:     files
group:      files
initgroups: files

hosts:      files dns

bootparams: files

ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files

automount:  files
aliases:    files

=== resolv.conf === (whatever is appropriate for your network)
search lan
nameserver 192.168.1.1

=== remote-ssh.sh === (note: hardcoded IPs, use whatever is appropriate for your network, I'm use a 192.168.1.1 gateway with an 192.168.1.10 ip for the machine on a network with a /24 netmask (ie. 255.255.255.0))
#!/bin/sh

/sbin/ip link set dev lo up

/sbin/modprobe eth0
/sbin/ip addr add 192.168.1.10/24 broadcast + dev eth0
/sbin/ip link set dev eth0 up
/sbin/ip route add default via 192.168.1.1

mkdir -p /var/log
> /var/log/lastlog

/usr/sbin/dropbear -E -m -s -p 222 -a -K 600

=== authorized_keys === (this one you'll need to use whatever public keys you want to use to login to the initramfs)
ssh-dss A...w== maze@athina
ssh-rsa A...T maze@athina

=== auth.c ===
// gcc -std=gnu99 -O2 -Wall auth.c -o auth
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>

int main (int argc, const char * argv[]) {
  if (argc != 2) {
    printf("Usage: auth 'passwd'\n");
    return 1;
  }

  int fd = open("/dev/console", O_RDONLY);
  if (fd < 0) {
    return 2;
  }

  for (const char * str = argv[1]; *str; ++str)
    ioctl(fd, TIOCSTI, str);
  ioctl(fd, TIOCSTI, "\r");
  return 0;
}

=== tiocsti.c ===
// gcc -std=gnu99 -O2 -Wall tiocsti.c -o tiocsti
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>

void stuff(int fd, const char * str) {
  printf("stuff [%s]\n", str);
  for (; *str; ++str) {
    // printf("(%c)", *str);
    int rv = ioctl(fd, TIOCSTI, str);
    if (rv < 0) perror("ioctl(TIOCSTI)");
  }
}

int main (int argc, const char * argv[]) {
  if (argc < 3) {
    printf("Usage: tiocsti /dev/ttyX text string\n");
    return 1;
  }

  int fd = open(argv[1], O_RDONLY);
  if (fd < 0) {
    perror("open");
    return 2;
  }

  for (int i = 2; i < argc; ++i) {
    if (i != 2) stuff(fd, " ");
    stuff(fd, argv[i]);
  }

  close(fd);
  return 0;
}

# Compile the two utilities
gcc -std=gnu99 -O2 -Wall tiocsti.c -o tiocsti
gcc -std=gnu99 -O2 -Wall auth.c -o auth

# Here's roughly what you should have, if you "ls -l":

-rwxr-xr-x.  1 root root 6909 May 13 00:57 auth
-rw-r--r--.  1 root root  507 May 13 00:57 auth.c
-rw-r--r--.  1 root root 1635 May 12 19:16 authorized_keys
-r--------.  1 root root  458 May 12 17:34 dropbear_dss_host_key
-r--------.  1 root root  426 May 12 17:34 dropbear_rsa_host_key
-rw-r--r--.  1 root root    9 May 12 18:59 host.conf
-rw-r--r--.  1 root root   44 May 12 18:58 hosts
-rwxr-xr-x.  1 root root 1754 May 13 00:48 module-setup.sh
-rw-r--r--.  1 root root  260 May 12 18:41 nsswitch.conf
-rw-r--r--.  1 root root   35 May 12 19:10 passwd
-rwxr-xr-x.  1 root root  361 May 12 19:52 remote-ssh.sh
-rw-r--r--.  1 root root   34 May 12 18:56 resolv.conf
-rw-r--r--.  1 root root   50 May 12 19:53 shells
-rwxr-xr-x.  1 root root 7529 May 12 20:26 tiocsti
-rw-r--r--.  1 root root  755 May 12 20:28 tiocsti.c

# Build the new initramfs
dracut --force

# And this point you should be done.
# You can reboot (and cross your fingers).

# Once the machine is booted and at it's waiting for the password, it should be pingable from the network.

# Login with:
ssh -p 222 root@<machine_name_or_its_ip>

# You can get a dump of the console with:
cat /dev/vcs1

# In particular you can devote an xterm to getting update of the state of the console:
while true; do cat /dev/vcs1; sleep 1; done
# and use a different xterm to actually authenticate

# Low level keyboard buffer stuffing:
tiocsti /dev/console "$(echo -n 'ThePassWord\r')"

# Easier to use password authentication:
auth 'ThePassword'
# (the above simply stuffs 'ThePassword' <enter> into the keyboard buffer of /dev/console, and as such is simpler to type, but equivalent to the previous tiocsti command)

# Once you've done this, machine boot should continue.
# you won't be kicked out of your shell, but the initramfs will be deleted, so very quickly you'll discover nothing works.
# you may also discover that port 222 is firewalled (when service iptables starts) and that will kill your ssh session
# You may wish to add something to kill the dropbear in your main startup scripts, for example adding 'killall dropbear' near the end of /etc/rc.d/rc.local
Comment 8 Maciej Żenczykowski 2012-05-13 04:39:03 EDT
additional notes.
the above assumes eth0 is your nic, might need fixing

the 'echo -n' should perhaps be 'echo -e -n' ??? maybe this is a dash-ism?  The '-e' is needed in bash...

obviously the above hard codes IPs and network config and as such is only relevant for a static ip config, but if you care about remote unlock, your machine is probably stationary, so this shouldn't be a big problem ;-)
Comment 9 Robert Hager 2012-07-18 17:53:52 EDT
Hello Maciej,

should:
echo -en '127.0.0.1\tlocalhost\n::1\tlocalhost\n' > host
not be:
echo -en '127.0.0.1\tlocalhost\n::1\tlocalhost\n' > hosts
?

Additionaly I want to thank you for this guide.

Unfortunately it isn't working for me, I can't get an uplink with my Centos 6 machine and I have no clue why.
Your help would be highly appreciated. If you are interested in helping me, please write a short email or repost here. Thanks.(In reply to comment #8)
Comment 10 Maciej Żenczykowski 2012-07-19 21:07:27 EDT
Yeah, you are right wrt. 'hosts' being a typo, probably didn't highlight the entire line while cut-and-pasting.

The location of the dracut modules changed recently.
After upgrading to Fedora 17 I had to move the directory to:

  /usr/lib/dracut/modules.d/40maze/

instead of

  /usr/share/dracut/modules.d/40maze/


Wrt. Centos 6, I'm afraid I know precious about dracut and the boot process there...  I think RHEL6 is based off of a much older fedora and thus a much older dracut.  Feel free to email me with any information about how far you've gotten.  Is the machine pingable while waiting for the password?  Can you telnet to port 222 on it? etc.
Comment 11 Maciej Żenczykowski 2012-08-05 17:52:20 EDT
Just as an FYI, while attempting to set this up on a new F17 machine, I discovered I had to change eth0 to em1 throughout, and add the following:

/etc/modprobe.d/local.conf:
alias em1 e1000e

(based on output of 'ip addr' and 'ethtool -i em1')

without this the initramfs doesn't have the right module/driver for the network card and networking fails to comeup.

Another interesting note: 
for debuggability I removed 'rhgb quiet' from the kernel commandline, and also discovered that removing SYSFONT=True from the kernel command line results in a system which somehow fails to boot (apparently switching to full gfx/text mode fails in some very weird userspace-lockup like way - kernel responds to pings, telnet to port 222 connects, but you don't get an ssh banner, nothing ever happens on screen)
Comment 12 Harald Hoyer 2012-08-13 05:00:07 EDT
(In reply to comment #11)
> Just as an FYI, while attempting to set this up on a new F17 machine, I
> discovered I had to change eth0 to em1 throughout, and add the following:
> 
> /etc/modprobe.d/local.conf:
> alias em1 e1000e
> 
> (based on output of 'ip addr' and 'ethtool -i em1')
> 
> without this the initramfs doesn't have the right module/driver for the
> network card and networking fails to comeup.
> 

This should not be necessary with biosdevname installed.

> Another interesting note: 
> for debuggability I removed 'rhgb quiet' from the kernel commandline, and
> also discovered that removing SYSFONT=True from the kernel command line
> results in a system which somehow fails to boot (apparently switching to
> full gfx/text mode fails in some very weird userspace-lockup like way -
> kernel responds to pings, telnet to port 222 connects, but you don't get an
> ssh banner, nothing ever happens on screen)

"SYSFONT=True" is a bug in anaconda. It should be something like SYSFONT=latarcyrheb-sun16
Comment 13 hostmaster 2012-09-24 04:21:21 EDT
A modification to auth does avoid echoing passphrase on ssh client console. Due to security considerations passphrase should never be visible, and should not remain in memory.

#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>
#include <string.h>

int main (int argc, const char * argv[]) {

  char *passphrase;
  const char *prompt="Passphrase: ";
  int i;

  int fd = open("/dev/console", O_RDONLY);
  if (fd < 0) {
    return 2;
  }

  passphrase=getpass(prompt);

  for (const char * str = passphrase; *str; ++str)
    ioctl(fd, TIOCSTI, str);
  ioctl(fd, TIOCSTI, "\r");

  int len=strlen(passphrase);
  for (i=0;i<len;i++) { passphrase[i]=0; };
  return 0;
}

If called, it asks for passphrase and does not echo while entering passphrase. After moving this to console buffer it overwrites its own buffer with zero.
Comment 14 Robert 2012-12-24 10:05:24 EST
Instructions for CentOS6 can be found on http://roosbertl.blogspot.de/2012/12/centos6-disk-encryption-with-remote.html
Comment 15 Christopher Meng 2013-10-23 05:56:56 EDT
Are there any news available?
Comment 16 Michael Scherer 2013-12-08 18:41:51 EST
Also found https://github.com/mk-fg/dracut-crypt-sshd
Comment 17 Till Maas 2013-12-09 01:34:11 EST
For the record, there is also a different approach used here: https://bitbucket.org/bmearns/dracut-crypt-wait
Comment 18 Rainier Wolfcastle 2013-12-24 10:08:33 EST
I looked at the above suggestions and also put together a module to unlock remotely. It's forked from the GitHub suggestion above and available at "https://github.com/rlwolfcastle/dracut-crypt-sshd".

I tested it on Fedora 20 and could unlock the volume either at the console or remotely without any problems. In short, the method I used was:

1. Install dracut-network & module to get/set an IP through dracut kernal options.
2. Convert a current ssh RSA host key (so the fingerprint doesn't change) to Dropbear format.
3. Supply my OpenSSH authorized_keys file for logging in.
4. Use console_auth to send the password.

Both steps 2 & 3 rely on already using OpenSSH to login but can be replaced by just using dropbearkey to make a key pair.
Comment 19 xenoson 2014-08-11 17:09:20 EDT
Hi I think it is not necessary to compile binaries that talk to the console to enter the passwords. We can restart the cryptsetup.target and it asks right away.

I saw someone uploaded on this yesterday, a real package would be nice. I wrote my skripts last week and they work on centos 7:
https://github.com/xenoson/dracut-earlyssh/tree/master/modules/60dropbear
Comment 20 Harald Hoyer 2014-08-12 06:33:20 EDT
(In reply to xenoson from comment #19)
> Hi I think it is not necessary to compile binaries that talk to the console
> to enter the passwords. We can restart the cryptsetup.target and it asks
> right away.
> 
> I saw someone uploaded on this yesterday, a real package would be nice. I
> wrote my skripts last week and they work on centos 7:
> https://github.com/xenoson/dracut-earlyssh/tree/master/modules/60dropbear

Why not start dropbear as a systemd service? No hassle with killing and pid and pkill?
Comment 21 xenoson 2014-08-12 07:59:54 EDT
Yes, that's right, it would be the cleanest. Took that part from the old versions posted here. Honestly I don't know enough about systemd internals to write that now.
Comment 22 Florian Bäuerle 2014-09-23 11:22:00 EDT
I tried using xenoson's dracut-earlyssh. Sadly I failed and could not even debug it properly (Trying to do so on a headless server with rescue system).

I rebuilt the initramfs while chrooting from a debian rescue system, maybe that's an issue..

Would be really great to have this working on centos 7.
Comment 23 xenoson 2014-09-28 20:36:14 EDT
Could well be the chroot. Try to built the initrd from the install system.

This is how to boot it with grub2. Put the iso and ks.cfg into an iso directory on the boot partition. Write this into ks.cfg:
sshpw --username=root rootpassword

this into grub's custom.cfg:

isopath="/iso"
bootpartuuid="07f28003-4aa6-474d-b43d-4d9a44c2f3d5"
nameif="ethif0"
mac="11:22:33:44:55:66"
serverip="192.168.1.100"
gatewayip="192.168.1.1"
netmask="255.255.255.0"
hostname="host"
nameserverip1="192.168.1.22"
nameserverip2="192.168.1.33"
nameserverip3="192.168.1.44"

menuentry "CentOS 7 NetInstall vnc ssh" {

    insmod part_gpt
    insmod ext2
    set root='hd0,gpt1'
    if [ x$feature_platform_search_hint = xy ]; then
      search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 $bootpartuuid
    else
      search --no-floppy --fs-uuid --set=root $bootpartuuid
    fi

    set isoname="CentOS-7.0-1406-x86_64-NetInstall.iso"
    set isofile="${isopath}/${isoname}"
    echo "Using ${isofile}..."
    loopback loop $isofile
    linux (loop)/isolinux/vmlinuz inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rd.live.check quiet ifname=$nameif:$mac BOOTIF=$mac ip=$serverip::$gatewayip:$netmask:$hostname:$nameif:none nameserver=$nameserverip1 nameserver=$nameserverip2 nameserver=$nameserverip3 inst.repo=http://mirror.centos.org/centos-7/7.0.1406/os/x86_64 inst.headless nomodeset inst.vnc inst.vncpassword=vncpassword inst.noshell inst.ks=hd:UUID=$bootpartuuid:/iso/ks.cfg inst.sshd iso-scan/filename=${isofile}
    initrd (loop)/isolinux/initrd.img
}
Comment 24 Florian Bäuerle 2014-09-29 12:53:14 EDT
Ahhhw if i knew that earlier... that's a pity..

I just gave up on this and did without encryption (with a bad conscience of course).

At some point maybe i will retry that, i think encrypting a btrfs-raid1 afterwards should be doable. Many thanks for that hint!
Comment 25 Artem Sidorenko 2015-02-05 17:45:04 EST
Created a PR to dracut: https://github.com/haraldh/dracut/pull/39
Comment 26 Michael Scherer 2017-04-08 06:14:59 EDT
So it look like this request went to become https://github.com/dracut-crypt-ssh/dracut-crypt-ssh (just linking here in case someone find that bug).

Note You need to log in before you can comment on or make changes to this bug.