Bug 1356919 - emacs build failure due to high memory consumption during "make bootstrap" on pppc64(le)
Summary: emacs build failure due to high memory consumption during "make bootstrap" on...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: emacs
Version: rawhide
Hardware: powerpc
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jan Synacek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-15 09:27 UTC by Sinny Kumari
Modified: 2016-07-22 11:46 UTC (History)
7 users (show)

Fixed In Version: emacs-25.0.95-4.fc25
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-22 11:46:19 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
emacs patch for powerpc (1.32 KB, patch)
2016-07-15 09:40 UTC, Sinny Kumari
no flags Details | Diff
emacs build fix for ppc64(le) arch (7.24 KB, patch)
2016-07-18 18:53 UTC, Sinny Kumari
no flags Details | Diff
emacs build fix for power architectures (5.10 KB, patch)
2016-07-21 08:31 UTC, Sinny Kumari
no flags Details | Diff

Description Sinny Kumari 2016-07-15 09:27:02 UTC
Description of problem:
emacs-25.0.95 build fails on ppc64 architecture for both BE and LE with following message:
"Memory exhausted--use M-x save-some-buffers then exit and restart Emacs"

Detailed build log is available at koji build http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3537924

Version-Release number of selected component (if applicable): emacs-25.0.95-1

Additional information: Tried building locally on ppc64 BE box and found that emacs needed around 15GB of memory to build successfully which is pretty high than expected.

Comment 1 Sinny Kumari 2016-07-15 09:40:33 UTC
Created attachment 1180084 [details]
emacs patch for powerpc

I am providing a workaround patch which fixes memory issue during build process. This patch runs "make bootstrap" under "setarch -R".
Successful koji scratch build link after applying this patch - http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538290

More detailed discussion around this problem is going on at libc-alpha ML https://sourceware.org/ml/libc-alpha/2016-07/msg00430.html.

Will update here if I have any better solution.

Comment 2 Sinny Kumari 2016-07-15 18:02:52 UTC
Update
------
Emacs builds and works fine on ppc64(le) with patch proposed at https://sourceware.org/ml/libc-alpha/2016-07/msg00481.html 

Koji scratch build with patch applied is available at http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651

Comment 4 Florian Weimer 2016-07-18 10:00:26 UTC
(In reply to Jan Synacek from comment #3)
> This was fixed in
> http://git.savannah.gnu.org/cgit/emacs.git/commit/
> ?id=e95b023163e96538b15f030b7176b7ec59cf86f5.

Based on <https://sourceware.org/ml/libc-alpha/2016-07/msg00505.html>, Emacs still does not build with this patch.

This is not too surprising because the real fix has to be in src/gmalloc.c because it can't deal with huge gaps between .data and the the heap.

Comment 5 Jan Synacek 2016-07-18 11:55:45 UTC
Ok, I must have missed the follow up. I won't apply the patch yet.

Comment 6 Dan Horák 2016-07-18 14:45:11 UTC
Can we get eg. the "setarch"-based workaround in place until there is a proper solution on emacs side? Because the lack of emacs 25.0.95 build blocks the automated builds for Rawhide on ppc64/ppc64le.

Comment 7 Sinny Kumari 2016-07-18 18:53:42 UTC
Created attachment 1181243 [details]
emacs build fix for ppc64(le) arch

So far, I think patch proposed at https://sourceware.org/ml/libc-alpha/2016-07/msg00481.html works well for ppc64(le) in Fedora rawhide. Based on it I have prepared a setarch patch which is available in bugzilla attachment. Patch gets applied only for ppc64(le) and Fedora release > 24. 

With this patch emacs builds fine for all architectures for Fedora, rawhide.

Some successful scratch build links:
F24, ppc64(le) - http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3541129
Fedora, rawhide, ppc64(le) - http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3541133
Fedora rawhide, primary arches - http://koji.fedoraproject.org/koji/taskinfo?taskID=14936002

Comment 8 Jan Synacek 2016-07-19 05:46:59 UTC
@Sinny: You said in https://sourceware.org/ml/libc-alpha/2016-07/msg00505.html that the patch wasn't working in rawhide, so why are you proposing it as a fix?

I'll go with a temporary setarch solution until a proper patch exists.

Comment 9 Sinny Kumari 2016-07-19 05:56:57 UTC
(In reply to Jan Synacek from comment #8)
> @Sinny: You said in
> https://sourceware.org/ml/libc-alpha/2016-07/msg00505.html that the patch
> wasn't working in rawhide, so why are you proposing it as a fix?

https://sourceware.org/ml/libc-alpha/2016-07/msg00505.html is later on proposed patch which fails for rawhide, pcc64(le). What I have proposed is to use a previous patch from https://sourceware.org/ml/libc-alpha/2016-07/msg00481.html which works fine.

Comment 10 Sinny Kumari 2016-07-19 06:09:13 UTC
(In reply to Sinny Kumari from comment #9)
> (In reply to Jan Synacek from comment #8)
> > @Sinny: You said in
> > https://sourceware.org/ml/libc-alpha/2016-07/msg00505.html that the patch
> > wasn't working in rawhide, so why are you proposing it as a fix?
> 
> https://sourceware.org/ml/libc-alpha/2016-07/msg00505.html is later on
> proposed patch which fails for rawhide, pcc64(le). What I have proposed is
> to use a previous patch from
> https://sourceware.org/ml/libc-alpha/2016-07/msg00481.html which works fine.

For reference, my reply to patch proposed at https://sourceware.org/ml/libc-alpha/2016-07/msg00481 is at https://sourceware.org/ml/libc-alpha/2016-07/msg00483.html

Comment 11 Jan Synacek 2016-07-19 10:01:51 UTC
This is now "fixed" in emacs-25.0.95-3.fc25. I will leave this bug open for now.

Comment 12 Sinny Kumari 2016-07-19 10:17:04 UTC
(In reply to Jan Synacek from comment #11)
> This is now "fixed" in emacs-25.0.95-3.fc25. I will leave this bug open for
> now.

Thanks for fixing it.

Note: emacs-25.0.95-3.fc25 build uses patch proposed in c#1 (emacs patch for powerpc) instead from c#6 ( emacs build fix for ppc64(le) arch). With current fix used in emacs-25.0.95-3, on ppc64(le) box user will have to run emacs under setarch -R option to avoid memory consumption issue while running.

Comment 13 Sinny Kumari 2016-07-21 08:31:56 UTC
Created attachment 1182374 [details]
emacs build fix for power architectures

As mentioned in C#12, emacs-25.0.95-3.fc25 fixes only build failure issue. Memory issue still exist when a file is opened using emacs.

Previously, I was not aware of any package which may run emacs during build process. But yesterday came across git package which seems to call emacs during build process and due to which git on ppc64(le) rawhide are failing. Following is the error log:

"emacs -batch -f batch-byte-compile git.el
Loading /usr/share/emacs/site-lisp/site-start.d/desktop-entry-mode-init.el (source)...
>>Error occurred processing git.el: error (("Memory exhausted--use C-x s then exit and restart Emacs")) ".

Good news is that, we have a final patch for this issue which was proposed by Paul Eggert on emacs bugzilla and seems approved to be included in emcas 25 branch http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11. 
Using mentioned fix, I have prepared a patch which can be included in Fedora dist-git rawhide branch which is available in bugzilla attachment.

Successful scratch build with patch included:
Fedora rawhide, Pcc64(le) - http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3546218
Fedora rawhide, Primary arches - http://koji.fedoraproject.org/koji/taskinfo?taskID=14966690

Also, to avoid confusion have marked previously proposed patches in this bugzilla as Obsoletes.


Note You need to log in before you can comment on or make changes to this bug.