Bug 431009 - buffer overflow when using shell script expansion macro syntax
buffer overflow when using shell script expansion macro syntax
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: rpm (Show other bugs)
8
All Linux
low Severity medium
: ---
: ---
Assigned To: Panu Matilainen
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-01-30 22:03 EST by Carl Roth
Modified: 2012-03-14 15:54 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-01-09 02:38:23 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
sample spec file (170 bytes, text/plain)
2008-01-30 22:03 EST, Carl Roth
no flags Details
Sample shell script (needs to be executable) (525 bytes, text/plain)
2008-01-30 22:03 EST, Carl Roth
no flags Details
console output from rpmbuild (4.38 KB, text/plain)
2008-07-13 20:01 EDT, Carl Roth
no flags Details
new sample shell script for rpm 4.9.1.2 (needs to be executable) (458 bytes, text/plain)
2012-03-14 08:19 EDT, Anssi Hannula
no flags Details

  None (edit)
Description Carl Roth 2008-01-30 22:03:03 EST
Description of problem:

I'm trying to define a SPEC file that uses shell-script macro expansion as follows:

  # some rpm SPEC file text
  %{expand:%(./some-shell-script)}
  # some more SPEC file text

rpm and rpmbuild seem to not like it when macro text expanded from shell command
is too long.  I played around a bit with some test cases and it appears that the
limit is around 8K or so before rpm starts reporting SPEC file errors.

Version-Release number of selected component (if applicable):

rpm-4.4.2.2-7.fc8

(may be reproducible with earlier versions)

How reproducible:

Always

Steps to Reproduce:
1. Make a spec file
2. Include a shell-script expansion in it
3. Craft the shell script to generate at least 8k of SPEC syntax as output.
4. Operate on the spec file, e.g.

     rpmbuild -bs foo.spec

Actual results:

RPM reports a syntax error in the SPEC file, usually something like an un-closed
macro, but it varies.

Expected results:

RPM should either process the macro correctly, or report that the macro
expansion is invalid.

Additional info:

I'm attaching a simple SPEC file with a shell script to trigger the error.

I'm not sure about the internal details of what RPM is doing here; if it's a
buffer overflow this this is potentially also a security hole.
Comment 1 Carl Roth 2008-01-30 22:03:03 EST
Created attachment 293532 [details]
sample spec file
Comment 2 Carl Roth 2008-01-30 22:03:55 EST
Created attachment 293533 [details]
Sample shell script (needs to be executable)
Comment 3 Jeff Johnson 2008-02-04 08:12:41 EST
From /usr/share/doc/rpm-X.Y/macros (for many years now):

In addition to the "%{...}" form, shell expansion can be performed
using "%(shell command)". The expansion of "%(...)" is the output of
(the expansion of) ... fed to /bin/sh. For example, "%(date
+%%y%%m%%d)" expands to the string "YYMMDD" (final newline is
deleted). Note the 2nd % needed to escape the arguments to /bin/date.
There is currently an 8K limit on the size that this macro can expand
to.

There is no buffer "overflow"; the expansion buffer is truncated, causing
syntax errors.
Comment 4 Jindrich Novy 2008-03-20 13:18:17 EDT
The 8K limit is likely caused by default buffer size of BUFSIZ (which defaults
to 8K) to which rpmbuild tries to sprintf stuff. We are currently in progress of
removing such artificial limitations and to use asprintf based approach instead,
so that problems like this will be hopefully history since rpm-4.6+ an on.
Comment 5 Panu Matilainen 2008-04-15 06:24:40 EDT
There was an overflow of sorts involved here, expansion didn't leave room for
trailing '\0'. Fixed upstream (including 4.4.2.3)...
Comment 6 Carl Roth 2008-07-13 20:01:09 EDT
Are you claiming that this is fixed?  I tried this in rpm-4.4.2.3 and got the
following:

* Lots of reports of "error: Target buffer overflow"
* rpm aborted with a glibc assertion

This all may be moot now that pre-rpm5 is going through rawhide...
Comment 7 Carl Roth 2008-07-13 20:01:50 EDT
Created attachment 311674 [details]
console output from rpmbuild
Comment 8 Panu Matilainen 2008-07-14 07:58:43 EDT
Hmm, it was supposedly fixed in 4.4.2.3 too but apparently not so.

It is fixed in the new rawhide rpm though, or at least I can't reproduce it. Oh
and it has nothing to do with rpm5(.org), what's in rawhide is rpm.org snapshot.
Comment 9 Bug Zapper 2008-11-26 04:38:58 EST
This message is a reminder that Fedora 8 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 8.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '8'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 8's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 8 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 10 Bug Zapper 2009-01-09 02:38:23 EST
Fedora 8 changed to end-of-life (EOL) status on 2009-01-07. Fedora 8 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.
Comment 11 Anssi Hannula 2012-03-14 08:19:37 EDT
Created attachment 569969 [details]
new sample shell script for rpm 4.9.1.2 (needs to be executable)

This can still be reproduced with 4.9.1.2 by upping the 'some-script' output to ~140kB, new version attached.

We've hit this a few times on Mageia.

$ LC_ALL=C rpmbuild -bb test.spec
Hello from some-script
error: Unclosed %if
$ LC_ALL=C rpm --version
RPM version 4.9.1.2

Can you reopen this bug, or should it be re-reported elsewhere?
Comment 12 Panu Matilainen 2012-03-14 09:28:20 EDT
I suppose by "this" you mean the "unclosed if"-error? That's still caused by largely the same fundamental limitation as in 2008 (see comment #3). The macro engine itself no longer has such limitations as of rpm >= 4.9.x but the spec parser is still using a static-sized buffer and the expanded result is larger, the output gets truncated and you'll get errors like that. 

Removing the spec parser limitation is tracked here already: http://rpm.org/ticket/814, in the meanwhile if you really need such enormous macros for something, bump up the spec lbuf size in build/rpmbuild_internal.h.

As it happens, there is an actual buffer overflow involved with rpm >= 4.9.x, looks like an off-by-one regression introduced somewhere in the process of eliminating the static macro buffer size limit. If you want to track *that* separately then open a new bug (even if by just cloning selected parts of this). Reopening old bugs for new issues just because they look similar only confuses things.
Comment 13 Anssi Hannula 2012-03-14 15:54:45 EDT
That makes sense, thank you.

Note You need to log in before you can comment on or make changes to this bug.