Red Hat Bugzilla – Bug 121876
FC2T3 install boot fails on VMware Workstation
Last modified: 2007-11-30 17:10:41 EST
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040124
Description of problem:
Attempting to install T3 on VMware Workstation 4.5 on FC1 host.
Booting from ISO #1 gets to:
Then dies with the VMware Error:
*** VMware Workstation internal monitor error ***NOT_IMPLEMENTED at
Please report this problem by selecting menu item Help > VMware on
theWeb > Request Support, or by going to the Web
Please provide us with the log file
(/raid/prs/vmware/FC2T2/vmware.log) and the core file
If the problem is repeatable, please set 'Logging level' to 'Debug'
inthe Misc panel of the configuration editor. Then reproduce the
incidentand file it according to the instructions.
We will respond on the basis of your support entitlement.
We appreciate your feedback, -- the VMware Workstation team.
Not entirely unexpected as VMware (on FC1 host) has not been able to
runFC2 test kernels as guest OS since 2.6.5-1.322 (just before the
reiserfs fix) without this problem. Reported to VMware, but this
configuration is (obviously ;-) unsupported.
Version-Release number of selected component (if applicable):
Fedora Core 2 Test 3, anaconda-9.92-8
Steps to Reproduce:
1. Boot from boot.iso or CD #1
Actual Results: Boot fails.
Expected Results: Successfull FC installation.
This is a VMWare bug. There's not much I can do about it.
No, it is your bug, and VMware can only workaround around it. You are
returing with SYSEXIT to segment with limit 0xBFFFFFFF while Intel
documentation explicitly states that segment limit for
sysexit/sysenter must be 0xFFFFFFFF (and hardware always sets limit to
0xFFFFFFFF regardless what GDT says).
This warning is there to only warn you that you wanted to load segment
with limit 0xBFFFFFFF to the processor, but it is not going to happen:
sysexit ALWAYS loads limit 0xFFFFFFFF.
So if you wanted this to do stack no-exec patch, you MUST disable
sysenter, otherwise stack is executable after first sysenter based
syscall in the process occurs, until first interrupt (or taskswitch)
occurs (as interrupts use iret they'll restore proper (shortened)
segment limit on CS).
And if you set limit on CS to 0xBFFFFFFF for some other reason than
stack no-exec (and it does not matter for you that sometime that limit
is 0xFFFFFFFF), then I'd like to know that reason...
Looks like this may warrant reconsideration from Red Hat.
A work-around that seems to work for some, not for others:
On Mon, 2004-05-03 at 11:19 -1000, Warren Togami wrote:
> Running FC2 Test3 and higher in VMWare is broken because VMWare
> implement vdso. For now you can use "vdso=0" boot option. It works
> me when running FC2 Test3 within VMWare 4.5.1 hosted on FC2 Test3
This works for me to run latest FC2/rawhide as a guest OS when the
host is FC1 using the vmware-any-any-update65.tar.gz from
When host if FC2/rawhide with vmware-any-any-update65, and guest is
FC2T3 install attempt, the host reboots during the guest installer boot.
I would like to ask Petr where he gets that information, the docs I
have state explicitly that sysexit must return to a segment with
limits *UPTO* 4Gb
(we have an open interpretation request with intel on this one)
I've used http://folk.uio.no/botnen/intel/vt/reference/vc312.htm, as
it is first reference Google finds when you ask for "sysexit segment
limit". See also
And main problem is not that documentation states it, but that it
works that way.
Just try loading CS's MSR with 0xFFC4, and you'll find that your
system still works until first interrupt (iret will not work, as iret
consults descriptor tables) - and although you have no LDT, you'll
find that CS contains 0xFFC4 (in case of sysenter) or 0xFFD7 (in case
of sysexit), with 4GB limit, executable, non-writeable. So CS's MSR is
used only as a base for visible portion of CS/SS, and invisible
portion is always loaded with base=0, limit=4G-1, big, CS with
readable executable code segment, and SS with writable expand-up.
P4 even works if you'll load values 0xFFE8-0xFFFF to the CS's base,
although then your CS or SS is NULL selector (as Dan Arai found, P3
behaves erratically in this situation, but it is not problem we are
Yes, sysexit can return to the segment with 0 base and limit *upto*
4GB... Only problem is that you'll have set limit to 4GB then, which
does not matter for correct code, but it is killer for stack no-exec...
you are right, the limit is set to 4GB on certain CPUs (but not on all
SEP-capable x86 CPUs - e.g. the limit worked fine on my primary
testbox). I've fixed the exec-shield patch and uploaded a new version.
(Arjan updated the Fedora kernel.)
we enabled sysenter/sysexit (via vdso=1) for Fedora only a couple of
weeks ago, so this bug had minimal exposure so far. The separate
exec-shield patches i have never enabled the vdso.
Seems to be fixed with kernel-2.6.5-1.356 :-)
Currently running FC2 rawhide install with this kernel, VMware
4.5.1/vmware-any-any-update65 on a FC2T3+updates host.
Got quite a few line similar to below when install kernel-2.6.5-1.356
with rpm -ivh kernel-2.6.5-1.356.i686.rpm
/etc/security/selinux/file_contexts: invalid context
system_u:object_r:staff_home_spamassassin_t on line number 1856
Should I get/install mkinitrd-188.8.131.52-2.i386rpm
or generate that from kernel source?
Can you be more specific about how you install FC2 rawhide?