In RHEL 6 beta, in text install mode (still invokable by manually specifying "text" as a boot parameter by pressing tab), there is no "Custom Layout" partitioning option offered during the installation. This same issue exists in Fedora 12 and Fedora 13. The custom layout option IS offered in the graphical install mode, so it should also be there in text mode.
Text mode installation in RHEL-6 has been greatly simplified and all but deprecated. Custom partitioning in RHEL-6 is only available via the graphical installer interface (either on physical hardware sitting in front of you or via VNC) or kickstart. The only partitioning mechanism available during an interactive text mode installation is automatic partitioning. Note that kickstart gives you full control under either interface, text or graphical.
For reference to the original development plan for simplification of text mode:
Removing the custom partitioning is arguably a step too far in dumbing down the text installer, though.
(In reply to comment #4)
> Removing the custom partitioning is arguably a step too far in dumbing down the
> text installer, though.
The problem is maintaining two completely separate code bases for the installer interface. With the addition of many new storage types for RHEL-6 and the likelihood of people having hundreds or thousands of storage devices, the text mode interface simply doesn't scale. To do it correctly there would require a completely separate design and we just don't have the resources to maintain two installer interfaces in anaconda.
Again, if you do not want to use or cannot use the graphical installer directly at the console, you can use vnc install mode which gives you the graphical installer -or- you can use kickstart.