Bug 425411 - [RHEL5.2]: i386 PV guests on x86_64 HV fail "xm save"
[RHEL5.2]: i386 PV guests on x86_64 HV fail "xm save"
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: xen (Show other bugs)
5.2
All Linux
medium Severity medium
: ---
: ---
Assigned To: Jiri Denemark
Virtualization Bugs
:
Depends On:
Blocks: 383211 RHEL5u2_relnotes 448899 480118
  Show dependency treegraph
 
Reported: 2007-12-14 15:31 EST by Chris Lalancette
Modified: 2010-10-22 17:12 EDT (History)
15 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-09-02 06:07:08 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Full xend.log when trying to save an i386 domain (46.97 KB, text/plain)
2007-12-14 15:31 EST, Chris Lalancette
no flags Details
PATCH 1 to fix 32-on-64 save/restore issue (3.01 KB, patch)
2008-02-23 22:40 EST, Chris Lalancette
no flags Details | Diff
PATCH 2 to fix 32-on-64 save/restore issue (41.36 KB, patch)
2008-02-23 22:41 EST, Chris Lalancette
no flags Details | Diff
PATCH 3 to fix 32-on-64 save/restore issue (9.99 KB, patch)
2008-02-23 22:41 EST, Chris Lalancette
no flags Details | Diff
Patch to fix 32-on-64 save/restore issue (43.46 KB, patch)
2009-01-19 05:41 EST, Jiri Denemark
no flags Details | Diff


External Trackers
Tracker ID Priority Status Summary Last Updated
CentOS 2968 None None None Never

  None (edit)
Description Chris Lalancette 2007-12-14 15:31:22 EST
Description of problem:
I've successfully installed RHEL-4.6 i386 and RHEL-5.1 i386 guests on an x86_64
hypervisor with the 5.2 kernel.  However, attempting to do an "xm save <dom>
/var/lib/xen/save/<dom>-save" fails miserably.  In particular, it suspends the
domain, writes an 830 byte header out to disk, and then quits, leaving the guest
suspended.  In /var/log/xen/xend.log, we have the following:

[2007-12-14 15:23:30 xend.XendDomainInfo 3800] INFO (XendDomainInfo:947) Domain
has shutdown: name=migrating-rhel5pvi386 id=5 reason=suspend.
[2007-12-14 15:23:30 xend 3800] INFO (XendCheckpoint:99) Domain 5 suspended.
[2007-12-14 15:23:30 xend 3800] DEBUG (XendCheckpoint:108) Written done
[2007-12-14 15:23:32 xend 3800] INFO (XendCheckpoint:351) ERROR Internal error:
Timed out waiting for frame list updated.
[2007-12-14 15:23:32 xend 3800] INFO (XendCheckpoint:351) ERROR Internal error:
Failed to map/save the p2m frame list
[2007-12-14 15:23:32 xend 3800] INFO (XendCheckpoint:351) Save exit rc=1
[2007-12-14 15:23:32 xend 3800] ERROR (XendCheckpoint:133) Save failed on domain
rhel5pvi386 (5).
Traceback (most recent call last):
  File "/usr/lib64/python2.4/site-packages/xen/xend/XendCheckpoint.py", line
110, in save
    forkHelper(cmd, fd, saveInputHandler, False)
  File "/usr/lib64/python2.4/site-packages/xen/xend/XendCheckpoint.py", line
339, in forkHelper
    raise XendError("%s failed" % string.join(cmd))

And then another error when trying to resume the domain later.  I'll attach the
full xend.log to the bug.
Comment 1 Chris Lalancette 2007-12-14 15:31:22 EST
Created attachment 289421 [details]
Full xend.log when trying to save an i386 domain
Comment 2 Bill Burns 2007-12-17 15:22:24 EST
Setting this as a blocker for bz 383211 as this will prevent fully supporting 32
bit PV guests on x86_64.
Comment 3 Chris Lalancette 2008-02-13 09:07:48 EST
Patch posted today to fix some 32-on-64 save/restore problems:

http://lists.xensource.com/archives/html/xen-devel/2008-02/msg00425.html
Comment 4 Chris Lalancette 2008-02-23 22:39:28 EST
OK.  In order to get this working, we need at least the following upstream
xen-unstable changesets:

15706
15917

Plus the changeset mentioned in the mailing list in the last post.

I was able to take these 3 patches and apply them to the current RHEL5.2 tree
(with some fuzz).  I'll attach the patches here.

Using a libxenguest.so.3.0.0 built with these changes, I was able to
successfully save a RHEL-4 i386 guest.  However, the restore failed (no error
message on the console, of course).  xend.log says:

VmError: (4, 'Out of memory', "xc_dom_boot_mem_init: can't allocate low memory
for domain\n")

So there is still a bug to shake out with the restore stuff, but this is a
better start.

Chris Lalancette
Comment 5 Chris Lalancette 2008-02-23 22:40:46 EST
Created attachment 295727 [details]
PATCH 1 to fix 32-on-64 save/restore issue
Comment 6 Chris Lalancette 2008-02-23 22:41:26 EST
Created attachment 295728 [details]
PATCH 2 to fix 32-on-64 save/restore issue
Comment 7 Chris Lalancette 2008-02-23 22:41:49 EST
Created attachment 295729 [details]
PATCH 3 to fix 32-on-64 save/restore issue
Comment 8 Bill Burns 2008-03-07 15:11:08 EST
This release note is related to the one for bz 383211:

While 32 bit para-virtualized guests can be installed on x86_64 save/restore and
migration are not functional and should not be attempted.

Comment 9 Don Domingo 2008-03-09 20:17:52 EDT
revised note:

<quote>
You can now run 32-bit guests on 64-bit hosts. This capability is now included
as a technology preview. Note, that the ability to save, restore, and migrate
32-bit guests on 64-bit hosts is not functional, and as such should not be
attempted.
</quote>

please advise if any further revisions are required. thanks!
Comment 10 Don Domingo 2008-04-01 22:09:40 EDT
Hi,
the RHEL5.2 release notes will be dropped to translation on April 15, 2008, at
which point no further additions or revisions will be entertained.

a mockup of the RHEL5.2 release notes can be viewed at the following link:
http://intranet.corp.redhat.com/ic/intranet/RHEL5u2relnotesmockup.html

please use the aforementioned link to verify if your bugzilla is already in the
release notes (if it needs to be). each item in the release notes contains a
link to its original bug; as such, you can search through the release notes by
bug number.

Cheers,
Don
Comment 11 RHEL Product and Program Management 2008-06-02 16:26:24 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 13 Jiri Denemark 2009-01-15 08:41:54 EST
Comment on attachment 295727 [details]
PATCH 1 to fix 32-on-64 save/restore issue

This patch is already included in current xen package.
Comment 14 Jiri Denemark 2009-01-15 08:49:52 EST
Comment on attachment 295727 [details]
PATCH 1 to fix 32-on-64 save/restore issue

Included in xen RPM since 3.0.3-64.el5.2_4 and 3.0.3-71.el5
Comment 15 Jiri Denemark 2009-01-19 05:41:53 EST
Created attachment 329322 [details]
Patch to fix 32-on-64 save/restore issue

This is a rebased combination of PATCH 2 and PATCH 3. Works for me with both RHEL-5 and RHEL-4 PV guests.
Comment 16 Chris Lalancette 2009-01-22 06:09:15 EST
Assigning to Jiri since he is working on the 32-on-64 userspace issues.

Chris Lalancette
Comment 17 Jiri Denemark 2009-02-02 08:13:10 EST
This is mainly caused by different size of structures used for 32b and 64b guests. So far libxc ignored this for when saving/restoring a guest and treated all structures as 64b ones. The attached patch makes libxc honor the difference.
The patch is a rebased combination of two xen-unstable changesets:
15917 http://xenbits.xensource.com/xen-unstable.hg?rev/4c8394e3b011
17036 http://xenbits.xensource.com/xen-unstable.hg?rev/0164d924ceba

Tested successfully with both RHEL-4 and RHEL-5 guests with about 6GB of
memory.
Comment 18 Dustin Black 2009-02-02 15:12:19 EST
Tested the patch with CentOS 5.2.  Successfully able to live migrate a 3GB 32-bit CentOS 5.2 guest between two 64-bit hosts.
Comment 22 Jiri Denemark 2009-02-23 06:10:02 EST
A test package which fixes this issue (and several others as well) has been
made available at:

http://people.redhat.com/jdenemar/xen/

Could the reporter try it out and report if it fixes the problem or not?

Thank you for your cooperation.
Comment 23 Jiri Denemark 2009-03-02 05:22:07 EST
Fix built into xen-3.0.3-81.el5
Comment 27 Alain RICHARD 2009-06-29 10:01:36 EDT
I am experiencing the same issue under RHEL 5.3 with current official xen-3.0.3-80.el5_3.3.x86_64

After installing xen-3.0.3-85.el5jd2.x86_64 that incorporates the patches for this bug, I can confirm that the problem is solved.

Regards,

Alain Richard
Comment 28 Matteo Sessa 2009-07-01 07:38:20 EDT
I can confirm xen-3.0.3-85.el5jd2.x86_64.rpm fix this issue. 32-bit guests now save/restore/migrate smoothly.

/Matteo Sessa
Comment 29 Chris Ward 2009-07-03 13:59:30 EDT
~~ Attention - RHEL 5.4 Beta Released! ~~

RHEL 5.4 Beta has been released! There should be a fix present in the Beta release that addresses this particular request. Please test and report back results here, at your earliest convenience. RHEL 5.4 General Availability release is just around the corner!

If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity.

Please do not flip the bug status to VERIFIED. Only post your verification results, and if available, update Verified field with the appropriate value.

Questions can be posted to this bug or your customer or partner representative.
Comment 31 zhanghaiyan 2009-07-29 06:18:19 EDT
Verified on xen-3.0.3-91.el5 and PASS
Comment 33 errata-xmlrpc 2009-09-02 06:07:08 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-1328.html

Note You need to log in before you can comment on or make changes to this bug.