This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours

Bug 435767

Summary: [PATCH] frequent and random "Regular expression too big"
Product: [Fedora] Fedora Reporter: Alexandre Oliva <oliva>
Component: emacsAssignee: Chip Coldwell <coldwell>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: rawhideCC: aph, dcantrell, jonstanley, matt_domsch, michal, tagoh
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: NeedsRetesting
Fixed In Version: 22.2-4.fc9 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-05-03 16:36:32 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 235706    
Attachments:
Description Flags
Patch that fixes the bug none

Description Alexandre Oliva 2008-03-03 14:09:57 EST
Description of problem:
I'm a relatively heavy user of gnus within GNU Emacs.  As of a few days ago,
after a couple of weeks running Fedora 8, I went back to rawhide and I started
getting random "Regular experssion too big" errors.  Most often, they occurred
while parsing nnrss files, but sometimes they occur (in the same Emacs process)
doing something as simple as starting an interactive shell.

Version-Release number of selected component (if applicable):
emacs-22.1.50-4.fc9.x86_64

How reproducible:
At random, no reproducer, repeating the same action sometimes works around the
problem.  I set debug-on-errors to t to try to find a pattern, to no avail. 
Sometimes it's open files, sometimes it's starting processes, a few times it's
actually something that has to do with regular expressions.
Comment 1 Alexandre Oliva 2008-03-03 14:26:14 EST
I forgot to mention I couldn't duplicate the problem with XEmacs.
Comment 2 Alexandre Oliva 2008-03-22 07:21:01 EDT
Created attachment 298831 [details]
Patch that fixes the bug

The problem is that emacs invokes undefined behavior in regex.c, computing the
difference between unrelated pointers.	In general, this wouldn't be too much
of a problem, as long as the type used to represent the difference was wide
enough to cover the entire possible range of pointer differences.

Such a type is not even guaranteed to exist, and it can be tricky to get
reasonable results on segmented architectures.	So, the correct code needs to
compute offsets between pointers in the old buffer, and apply the same offset
into the new buffer.  On most cases, the compiler will just optimize the code
to the same we got before on i386, and to something very close, but using a
64-bit offset on x86-64.

The problem seems to have been caused at least in part by heap randomization,
such that buffers were *sometimes* (but somewhat rarely) moved to locations
that were too far away for the offset to be held correctly in an int, and then
Bad Things Happened.  I'm concerned that in this case emacs core memory may be
corrupted.

After the patch, I haven't seen the errors any more.  However, given how
difficult it was to trigger the bug in the first place, I can't be sure it is
gone for good.	But the fix is sound anyway.
Comment 3 Michal Jaegermann 2008-04-01 14:48:20 EDT
> However, given how difficult it was to trigger the bug in the first place ...

That confirms that I am not going crazy.  Thanks, Alexandre! :-)

Maybe this is a coincidence but it appears that it is easier (just easier)
to trigger that bug when logged from a remote.  If 'emacs -nw /some/file'
or just 'emacs /some/file' is used does not seem to matter.  Once that
message starts to appear then the only way out is apparently to logout,
from a remote or a desktop session, to login back and one is good for
a while again.

Sometimes "Regular expression too big" shows up immediately and
emacs does not even attempts to start and sometimes late enough for
something to be written in *Messages* buffer.  Then I found this:
.....
Loading advice...done
Loading edmacro...done
Loading easy-mmode...
do-after-load-evaluation: Invalid regexp: "Regular expression too big"
list-buffers-noselect: Invalid regexp: "Regular expression too big"

and that is it.  On a normal startup I see
.....
Loading advice...done
Loading bytecomp...done
Loading edmacro...done
Loading easy-mmode...done
.....
followed by a series of "Loading" and "Setting" messages as expected.
Comment 4 Daniel Berrange 2008-04-04 17:48:06 EDT
Putting this on the blocker list for F9 since emacs is basically unusable
without it, crashing with "regular expression too big" 50% of the time i start it.
Comment 5 Jon Stanley 2008-04-17 00:26:10 EDT
We are past final freeze.  Are you going to be able to get this patch in soon-ish?
Comment 6 Matt Domsch 2008-04-17 17:30:11 EDT
This is on the blocker list, and the release just slipped out to accommodate
such blocker bugs, especially those with patches.  A koji build/test cycle
followed by a request to tag for f9-final would seem appropriate, if it happens
immediately.
Comment 7 Matt Domsch 2008-04-17 17:35:31 EDT
I note only package maintainers/comaintainers may commit this patch and build,
due to pkg.acl existance.  At the moment that is only coldwell.
Comment 8 Matt Domsch 2008-04-17 23:02:48 EDT
koji scratch build with Alexandre's patch is here:
 http://koji.fedoraproject.org/koji/taskinfo?taskID=571715
Comment 9 Matt Domsch 2008-04-17 23:27:59 EDT
"works for me".
Comment 10 Jesse Keating 2008-04-18 10:43:07 EDT
Chip; ping?  This should be a easy build right?
Comment 11 Chip Coldwell 2008-04-18 14:14:32 EDT
Yeah, it went in upstream on March 26, so I'll just pull in the latest tarball.
Comment 12 Chip Coldwell 2008-04-18 17:03:45 EDT
emacs-23.0.60-2 is built.  This should include the fix (and some fancy new
icons, anti-aliased fonts, and other goodness).

Chip
Comment 13 Jesse Keating 2008-04-18 17:19:08 EDT
Tagging for final (since we're in a freeze, and this is a blocker)
Comment 14 Akira TAGOH 2008-04-20 21:29:56 EDT
How come does new features appear around this time anyway? the font backend
feature are totally broken for me, particularly boldish characters are too wide.
and also uim.el doesn't work on current CVS HEAD. though I'm not sure what's
wrong, but I can't get this fix now due to we are in a freeze and no time to
investigate that. please just apply a pinpoint patch to fix this issue only.
Comment 15 Alexandre Oliva 2008-04-22 00:02:50 EDT
23.0.60-2 does NOT fix the problem, unfortunately.
Comment 16 Alexandre Oliva 2008-04-22 00:13:30 EDT
Sorry, false alarm.  The upgade to 23.* broke a script I had in place to run a
personal build from back when 21.* was in the latest stable release and 22.* was
only available in the development tree.  I was running the personal build
instead of the emacs in /usr/bin.  Oops :-)
Comment 17 Brad Walker 2008-04-24 06:07:16 EDT
Upstream checked this patch into CVS, but not in time for the 22.2 release.
emacs-22.2-1.fc9 suffers from this issue. I get the error 33%-50% of the time
during startup.
Comment 18 Alexandre Oliva 2008-05-01 13:29:58 EDT
Is there anything I can do to help make sure we don't ship a memory-corrupting
and possibly remotely-exploitable GNU Emacs in F9?  The upgrade to 23.* was
rolled back, and what's in place now has the bug again.
Comment 19 Brad Walker 2008-05-02 06:54:10 EDT
emacs-22.2-4.fc9 has this patch.