Also need an option to run FIPS-140-2 kernel.

Hi @Mark,

One thing we have to understand is that Ironic kinda limited to what the Nova flavor can offer to us. But that said, Ironic does deploy two types of images:

1. Partition image: Which is an image of the root filesystem, there's no partition table or bootloader in it. With this type of image, Ironic will create a simple partitioning following Nova's flavor: The root partition, and optionals swap and/or ephemeral partitions.

2. Whole disk image: Instead of partitioning the disk and copying the root filesystem onto the root partition, this images already includes it all (the partition table, bootloader etc...). I believe that would be the way forward to you to deploy images with a custom filesystem layout in Ironic.

(You may want to take a look here too: http://docs.openstack.org/developer/ironic/deploy/install-guide.html#image-requirements)

Would that solves it for you ?

...

Also, having a tool to build those images would be great as well, but that's outside of Ironic's scope. Probably it would be an RFE for diskimage-builder or other similar tools.

Changing the component from openstack-ironic to diskimage-builder, it was agreed with @Paul and some other folks involved that this work should be done at image building time.

The upstream work in diskimage-builder to handle this request is https://review.openstack.org/#/c/336946/

Currently there is that change proposed on diskimage-builder to support multiple partitions:
https://review.openstack.org/#/c/375261/

Can this be reviewed and see if that matches our needs?

I did some initial test, using full disk images instead of partition images. That means, I provide overcloud-full.qcow2 and not initrd and vmlinuz images. It is possible for ironic to do that, as documented on:http://docs.openstack.org/project-install-guide/baremetal/draft/configure-integration.html#configure-the-image-service ( it is, i use vm element instead of baremetal one).
However, when deploying TripleO i'm hitting a problem with "openstack overcloud image upload" command.
It expects to have vmlinuz and initrd files. If not present, it throws an error "Required file "./overcloud-full.vmlinuz" does not exist".

Can image upload command be customized to don't require that files?

Related BZ for tripleo: https://bugzilla.redhat.com/show_bug.cgi?id=1381508

Related BZ for ironic: https://bugzilla.redhat.com/show_bug.cgi?id=1381511

A better approach might be to segregate those directories to different (logical) volumes rather than partitions, as this would allow operators to resize later.

Would this also meet DoD requirements?

There is two angles to this requirement; the first one is to ensure that disk space usage does not exceed a specified limit (for example ensure logs does not fill the partition), the second one is to be able to encrypt certain filesystem (for example /home).  
Would the proposed solution meet both those requirements?

Support for advanced partitioning in Ironic won't arrive before Pike (OSP 12), so I'd recommend using whole disk images for now.

Hi Dmitry. Can you clarify a bit more about advanced partitioning in Ironic? What use cases will it cover? Will it be suitable for our security use case?
Also i'm aware of the separated way of working of ironic: whole disk images vs flat partition one. Will advanced partitioning support creation of extra partitions on whole disk images, or will it be limited for the flat partition use case?

> What use cases will it cover?

Hart to tell, we haven't even started planning. I only there is a consistent demand (like this RFE, for example).

> Will it be suitable for our security use case?

Likely yes.

> Will advanced partitioning support creation of extra partitions on whole disk images

We never modify partitions on whole disk images, this is the idea of whole disk images. We will modify the way we deploy partition images.

So I have a working script with libguestfs, to convert a single partition overcloud image to a whole disk image.
That worked when using partitions, but when trying to switch to volumes, I got the problem that we cannot boot from volumes, because lvm modules are not shipped in the ramdisk image.

Console is just stuck on:
[ ***  ] A start job is running for dev-mapp....device (1h 17min 3s / no limit)

It means that is unable to mount the root volume. Also a quick look at the ramdisk image, shows that no lvm modules are loaded.

While the changes for diskimage-builder land, I have created some script that generates a whole disk image from the overcloud flat partition one:

http://teknoarticles.blogspot.com.es/2016/12/start-using-whole-disk-images-with.html
http://teknoarticles.blogspot.com.es/2016/12/how-to-encrypt-your-home-with-guestfs.html

There is a pending article i need to write, about how to expand the filesystem after deployment, to consume the remaining disk space.

Hitting some problems with pre-creating config-drive partition:
https://bugs.launchpad.net/ironic/+bug/1654269

https://blueprints.launchpad.net/tripleo/+spec/build-whole-disk-images

devel_ack+

Code landed for pike-2

This can now be marked verified and tested. Whole Disk Image was tested and deployed into the overcloud containing the requirements of separate partition with the security hardened image created and deployed on many nodes.

openstack overcloud image build --image-name overcloud-hardened-full --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-rhel7.yaml --verbose

cd
mv ~/images/overcloud-full.qcow2 ~/images/overcloud-full-old.qcow2
cp ~/images/overcloud-hardened-full.qcow2 overcloud-full.qcow2

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462