Bug 435767 - [PATCH] frequent and random "Regular expression too big"
Summary: [PATCH] frequent and random "Regular expression too big"
Alias: None
Product: Fedora
Classification: Fedora
Component: emacs
Version: rawhide
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Chip Coldwell
QA Contact: Fedora Extras Quality Assurance
Whiteboard: NeedsRetesting
Keywords: Reopened
Depends On:
Blocks: F9Blocker
TreeView+ depends on / blocked
Reported: 2008-03-03 19:09 UTC by Alexandre Oliva
Modified: 2013-01-10 04:35 UTC (History)
6 users (show)

Clone Of:
Last Closed: 2008-05-03 20:36:32 UTC

Attachments (Terms of Use)
Patch that fixes the bug (2.03 KB, patch)
2008-03-22 11:21 UTC, Alexandre Oliva
no flags Details | Diff

Description Alexandre Oliva 2008-03-03 19:09:57 UTC
Description of problem:
I'm a relatively heavy user of gnus within GNU Emacs.  As of a few days ago,
after a couple of weeks running Fedora 8, I went back to rawhide and I started
getting random "Regular experssion too big" errors.  Most often, they occurred
while parsing nnrss files, but sometimes they occur (in the same Emacs process)
doing something as simple as starting an interactive shell.

Version-Release number of selected component (if applicable):

How reproducible:
At random, no reproducer, repeating the same action sometimes works around the
problem.  I set debug-on-errors to t to try to find a pattern, to no avail. 
Sometimes it's open files, sometimes it's starting processes, a few times it's
actually something that has to do with regular expressions.

Comment 1 Alexandre Oliva 2008-03-03 19:26:14 UTC
I forgot to mention I couldn't duplicate the problem with XEmacs.

Comment 2 Alexandre Oliva 2008-03-22 11:21:01 UTC
Created attachment 298831 [details]
Patch that fixes the bug

The problem is that emacs invokes undefined behavior in regex.c, computing the
difference between unrelated pointers.	In general, this wouldn't be too much
of a problem, as long as the type used to represent the difference was wide
enough to cover the entire possible range of pointer differences.

Such a type is not even guaranteed to exist, and it can be tricky to get
reasonable results on segmented architectures.	So, the correct code needs to
compute offsets between pointers in the old buffer, and apply the same offset
into the new buffer.  On most cases, the compiler will just optimize the code
to the same we got before on i386, and to something very close, but using a
64-bit offset on x86-64.

The problem seems to have been caused at least in part by heap randomization,
such that buffers were *sometimes* (but somewhat rarely) moved to locations
that were too far away for the offset to be held correctly in an int, and then
Bad Things Happened.  I'm concerned that in this case emacs core memory may be

After the patch, I haven't seen the errors any more.  However, given how
difficult it was to trigger the bug in the first place, I can't be sure it is
gone for good.	But the fix is sound anyway.

Comment 3 Michal Jaegermann 2008-04-01 18:48:20 UTC
> However, given how difficult it was to trigger the bug in the first place ...

That confirms that I am not going crazy.  Thanks, Alexandre! :-)

Maybe this is a coincidence but it appears that it is easier (just easier)
to trigger that bug when logged from a remote.  If 'emacs -nw /some/file'
or just 'emacs /some/file' is used does not seem to matter.  Once that
message starts to appear then the only way out is apparently to logout,
from a remote or a desktop session, to login back and one is good for
a while again.

Sometimes "Regular expression too big" shows up immediately and
emacs does not even attempts to start and sometimes late enough for
something to be written in *Messages* buffer.  Then I found this:
Loading advice...done
Loading edmacro...done
Loading easy-mmode...
do-after-load-evaluation: Invalid regexp: "Regular expression too big"
list-buffers-noselect: Invalid regexp: "Regular expression too big"

and that is it.  On a normal startup I see
Loading advice...done
Loading bytecomp...done
Loading edmacro...done
Loading easy-mmode...done
followed by a series of "Loading" and "Setting" messages as expected.

Comment 4 Daniel Berrange 2008-04-04 21:48:06 UTC
Putting this on the blocker list for F9 since emacs is basically unusable
without it, crashing with "regular expression too big" 50% of the time i start it.

Comment 5 Jon Stanley 2008-04-17 04:26:10 UTC
We are past final freeze.  Are you going to be able to get this patch in soon-ish?

Comment 6 Matt Domsch 2008-04-17 21:30:11 UTC
This is on the blocker list, and the release just slipped out to accommodate
such blocker bugs, especially those with patches.  A koji build/test cycle
followed by a request to tag for f9-final would seem appropriate, if it happens

Comment 7 Matt Domsch 2008-04-17 21:35:31 UTC
I note only package maintainers/comaintainers may commit this patch and build,
due to pkg.acl existance.  At the moment that is only coldwell.

Comment 8 Matt Domsch 2008-04-18 03:02:48 UTC
koji scratch build with Alexandre's patch is here:

Comment 9 Matt Domsch 2008-04-18 03:27:59 UTC
"works for me".

Comment 10 Jesse Keating 2008-04-18 14:43:07 UTC
Chip; ping?  This should be a easy build right?

Comment 11 Chip Coldwell 2008-04-18 18:14:32 UTC
Yeah, it went in upstream on March 26, so I'll just pull in the latest tarball.

Comment 12 Chip Coldwell 2008-04-18 21:03:45 UTC
emacs-23.0.60-2 is built.  This should include the fix (and some fancy new
icons, anti-aliased fonts, and other goodness).


Comment 13 Jesse Keating 2008-04-18 21:19:08 UTC
Tagging for final (since we're in a freeze, and this is a blocker)

Comment 14 Akira TAGOH 2008-04-21 01:29:56 UTC
How come does new features appear around this time anyway? the font backend
feature are totally broken for me, particularly boldish characters are too wide.
and also uim.el doesn't work on current CVS HEAD. though I'm not sure what's
wrong, but I can't get this fix now due to we are in a freeze and no time to
investigate that. please just apply a pinpoint patch to fix this issue only.

Comment 15 Alexandre Oliva 2008-04-22 04:02:50 UTC
23.0.60-2 does NOT fix the problem, unfortunately.

Comment 16 Alexandre Oliva 2008-04-22 04:13:30 UTC
Sorry, false alarm.  The upgade to 23.* broke a script I had in place to run a
personal build from back when 21.* was in the latest stable release and 22.* was
only available in the development tree.  I was running the personal build
instead of the emacs in /usr/bin.  Oops :-)

Comment 17 Brad Walker 2008-04-24 10:07:16 UTC
Upstream checked this patch into CVS, but not in time for the 22.2 release.
emacs-22.2-1.fc9 suffers from this issue. I get the error 33%-50% of the time
during startup.

Comment 18 Alexandre Oliva 2008-05-01 17:29:58 UTC
Is there anything I can do to help make sure we don't ship a memory-corrupting
and possibly remotely-exploitable GNU Emacs in F9?  The upgrade to 23.* was
rolled back, and what's in place now has the bug again.

Comment 19 Brad Walker 2008-05-02 10:54:10 UTC
emacs-22.2-4.fc9 has this patch.

Note You need to log in before you can comment on or make changes to this bug.