Red Hat Bugzilla – Bug 336471
Flex generates "input in flex scanner failed" under variety of conditions.
Last modified: 2015-05-04 21:33:33 EDT
Description of problem: Flex-generated scanner produces same error under a
variety of build options. Example response
$ ./fuzz -v ../tex/exampleUTF8.tex
JPM: passed open_prelude()
JPM: inside read_a_file .. passed yylloc
JPM: entered newline filter zzlex.
JPM: making a call to yylex in zzlex.
--(end of buffer or a NUL)
input in flex scanner failed
It does this with -m32, -m64, or neither gcc flags, and several plausible make
and flex options as well as with or without the flex --nounistd switch. You can
see my printf's but I can't insert them into flex generated code. The error
appears to be an I/O problem, because I get the same error when trying to read
non-existent files. That is, $fuzz nonexistent.tex produces the same "input in
flex scanner failed" message rather than a "can't open" or "can't read" message
that fuzz normally produces. I was able to build the fuzz tool on Mac OS X with
no problems, so its not a fuzz issue, other than fuzz trying to be ANSI.
Version-Release number of selected component (if applicable):
# rpm -qi flex
Name : flex Relocations: (not relocatable)
Version : 2.5.33 Vendor: Fedora Project
Release : 9.fc7 Build Date: Mon 25 Jun 2007 10:34:08
Install Date: Wed 17 Oct 2007 09:13:18 AM EDT Build Host:
Group : Development/Tools Source RPM: flex-2.5.33-9.fc7.src.rpm
Size : 702392 License: BSD
Signature : DSA/SHA1, Mon 25 Jun 2007 11:22:57 AM EDT, Key ID b44269d04f2a6fd2
Packager : Fedora Project
URL : http://flex.sourceforge.net/
# rpm -qi kernel
Name : kernel Relocations: (not relocatable)
Version : 22.214.171.124 Vendor: Fedora Project
Release : 76.fc7 Build Date: Thu 30 Aug 2007 02:59:30
Install Date: Fri 21 Sep 2007 08:19:12 AM EDT Build Host:
Group : System Environment/Kernel Source RPM:
Size : 66412539 License: GPLv2
Signature : DSA/SHA1, Wed 12 Sep 2007 10:49:57 AM EDT, Key ID b44269d04f2a6fd2
Packager : Fedora Project
Download the tarball for the fuzz tool and try to build it. Fuzz is available
from http://spivey.oriel.ox.ac.uk/mike/fuzz/ and takes about 2 secs to build.
Steps to Reproduce:
1. Initial attempts to build causes a unistd.h mismatch over the definition of read.
2. This problem can be removed by several approaches including commenting out,
using the --nounistd switch for flex, and copying in the function prototypes
from unistd.h. All of these compile but all produce the same error.
I tried reverting to flex 5.4 but it still causes the same error. I suspect a
problem with flex's approach to size_t, but that's beyond my knowledge of either
flex or Fedora 7.
Building fuzz does not reproduce this error on Mac OS X or Debian Sarge.
I tried to enter this as an FC7 bug, but Bugzilla would not give me fc7 as an
Created attachment 230201 [details]
Fix for fuzz problem.
The problem is in fuzz source code. They shouldn't declare 'read' function
themselves but instead rely on libc providing it. The attached little patch
(should be applied to fuzz, not flex!) fixes the build for me on x86_64, in
both 32bit and 64bit mode. Build and testsuite then pass.
Hmm, as I'm re-reading your problem, it looks like you didn't generally complain
that you can build it, but that fuzz can't read your tex file. I'll give it
Thanks for looking at this. The patch pretty much does what I did at first,
except I couldn't get a functional fuzz and still don't. If just removing the
spurious read works for you, then my FC7 must be out of date in a way that I
don't understand. I'll try even more updating of my host than I already have,
and see if that makes the problem go away.
Well, without the patch I don't even build it. With the patch, I get few
harmless warnings and a build which passes the testsuite. I've tried F7 as well
as rawhide on x64, and F7 on x86. tex/example.tex gives me seemingly realistic
"tex/example.tex", line 46: Type mismatch in left argument of infix
> Predicate: president \subseteq members
> Arg type: PERSON
> Expected: P ?
"tex/example.tex", line 54: Right argument of operator \cup has wrong
> Expression: members \cup new?
> Arg type: PERSON
> Expected: P PERSON
So I can't seem to be able to reproduce this. Perhaps try to upload the
exampleUTF8.tex file that exposes the failure, it doesn't come with the
distribution of fuzz that I got from http://spivey.oriel.ox.ac.uk/mike/fuzz/.
I worked on this some more and have been able to get fUZZ to work properly. As far as I can tell, some
part of my FC 7 was out-of-date, but I could not figure out which library or tool was the problem. I run
KDE on the box in question, but I don't think KDE caused the problem. Since the problem can't be
reproduced, it clearly is neither a flex nor an FC 7 bug.
The file exampleUTF8.tex was just a re-encoding of example.tex. I was trying various file encodings to see
if that made any difference. I now have fUZZ working with both UTF8 and UTF16.
Ok, in that case I'll close the bug.