Bug 2164068

Summary: unzip-6.0-59.fc38: unzip -t: *** buffer overflow detected ***: terminated
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: unzipAssignee: Jakub Martisko <jamartis>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 38CC: fweimer, jamartis, kdudka, lucilanga, mattdm, pstodulk, python-maint, sipoyare
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: unzip-6.0-60.fc38 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-31 16:47:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2158232    
Attachments:
Description Flags
archive.zip harvested from the failing Python test none

Description Miro Hrončok 2023-01-24 16:38:28 UTC
Description of problem:
When unzip is upgraded from unzip-6.0-58.fc38 to unzip-6.0-59.fc38, Python tests are failing.

Version-Release number of selected component: unzip-6.0-59.fc38


How reproducible: Always.


Steps to Reproduce:
1. $ python3 -m test -v -j0 test_shutil

E.g. currently in mock:


$ mock -r fedora-rawhide-x86_64 init
...
$ mock -r fedora-rawhide-x86_64 install python3-test
...
$ mock -r fedora-rawhide-x86_64 shell
...
<mock-chroot> sh-5.2# rpm -q unzip
unzip-6.0-58.fc38.x86_64
<mock-chroot> sh-5.2# python3 -m test -v -j0 test_shutil
...
Ran 165 tests in 0.193s

OK (skipped=26)

== Tests result: SUCCESS ==

1 test OK.

Total duration: 288 ms
Tests result: SUCCESS

$ mock -r fedora-rawhide-x86_64 --enablerepo=local --update unzip
...
$ mock -r fedora-rawhide-x86_64 shell
...
<mock-chroot> sh-5.2# rpm -q unzip
unzip-6.0-59.fc38.x86_64
<mock-chroot> sh-5.2# python3 -m test -v -j0 test_shutil
...
======================================================================
FAIL: test_unzip_zipfile (test.test_shutil.TestArchives.test_unzip_zipfile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.11/test/test_shutil.py", line 1514, in test_unzip_zipfile
    subprocess.check_output(zip_cmd, stderr=subprocess.STDOUT)
  File "/usr/lib64/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['unzip', '-t', '/tmp/test_python_23æ/test_python_worker_28æ/tmpoqqrv126/archive.zip']' returned non-zero exit status 80.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.11/test/test_shutil.py", line 1520, in test_unzip_zipfile
    self.fail(msg.format(exc, details))
AssertionError: Command '['unzip', '-t', '/tmp/test_python_23æ/test_python_worker_28æ/tmpoqqrv126/archive.zip']' returned non-zero exit status 80.

**Unzip Output**
Archive:  /tmp/test_python_23æ/test_python_worker_28æ/tmpoqqrv126/archive.zip
*** buffer overflow detected ***: terminated


----------------------------------------------------------------------
Ran 165 tests in 0.189s

FAILED (failures=1, skipped=26)
test test_shutil failed

== Tests result: FAILURE ==

1 test failed:
    test_shutil

Total duration: 280 ms
Tests result: FAILURE



Additional info:
https://koschei.fedoraproject.org/package/python3.12?collection=f38
https://koschei.fedoraproject.org/package/python3.11?collection=f38
https://koschei.fedoraproject.org/package/python3.10?collection=f38
https://koschei.fedoraproject.org/package/python3.9?collection=f38
https://koschei.fedoraproject.org/package/python3.8?collection=f38
https://koschei.fedoraproject.org/package/python3.7?collection=f38
https://koschei.fedoraproject.org/package/python3.6?collection=f38

Comment 1 Miro Hrončok 2023-01-24 16:51:36 UTC
Created attachment 1940228 [details]
archive.zip harvested from the failing Python test

Reproduce the buffer overflow with:

<mock-chroot> sh-5.2# rpm -q unzip
unzip-6.0-59.fc38.x86_64

<mock-chroot> sh-5.2# unzip -t archive.zip 
Archive:  archive.zip
*** buffer overflow detected ***: terminated

Comment 2 Miro Hrončok 2023-01-24 17:02:09 UTC
OK, this fails with arbitrary zips as well:

<mock-chroot> sh-5.2# touch a
<mock-chroot> sh-5.2# zip test.zip a
  adding: a (stored 0%)
<mock-chroot> sh-5.2# unzip -t test.zip 
Archive:  test.zip
*** buffer overflow detected ***: terminated

Comment 3 Miro Hrončok 2023-01-25 10:57:48 UTC
I've rebuilt unzip in mock and I was still able to reproduce the problem.

I've added `%global _fortify_level 2` and rebuilt it again. The problem was gone:

<mock-chroot> sh-5.2# touch a
<mock-chroot> sh-5.2# zip test.zip a
  adding: a (stored 0%)
<mock-chroot> sh-5.2# unzip -t test.zip 
Archive:  test.zip
    testing: a                        OK
No errors detected in compressed data of test.zip.


Hence, blocking bz2158232 here and will open  PR shortly to set the fortify level to 2 until this is solved.

Comment 5 Florian Weimer 2023-01-25 11:17:29 UTC
The code in unzip:

2884            /* Convert filtered wide chars back to multi-byte. */
2885            woslen = wcstombs( NULL, wostring, 0);
2886            if ((newraw = malloc(woslen + 1)) == NULL) {
2887                free(wstring);
2888                free(wostring);
2889                strcpy( (char *)space, raw);
2890                return (char *)space;
2891            }
2892            woslen = wcstombs( newraw, wostring, (woslen * MB_CUR_MAX) + 1);

Siddhesh says that this is an unzip bug as it passes the wrong buffer size. I tend to agree, although overflow is not possible in this case (because unzip is single-threaded etc.).

Comment 6 Siddhesh Poyarekar 2023-01-25 12:16:13 UTC
So this looks like a thinko; one really only needs woslen here, not woslen * MB_CUR_MAX.  I've sent a PR:

https://src.fedoraproject.org/rpms/unzip/pull-request/7

For additional context, the fortified wcstombs (which predates _FORTIFY_SOURCE=3, the new level just happens to succeed more often) expects the passed length to be within bounds of the size of newraw, which makes sense for the function to be safe because any larger size essentially is a pass to read beyond bounds.  An actual buffer overflow check would be quite expensive and is avoided because there's no non-buggy reason to do this.

Comment 7 Siddhesh Poyarekar 2023-01-25 12:39:02 UTC
Please test:

https://bodhi.fedoraproject.org/updates/FEDORA-2023-cee3cccdb9

Comment 8 Miro Hrončok 2023-01-25 13:29:27 UTC
Build in progress in https://koschei.fedoraproject.org/package/python3.12?collection=f38

Manual check:

$ koji wait-repo f38-build --build unzip-6.0-60.fc38
Successfully waited 0:00 for unzip-6.0-60.fc38 to appear in the f38-build repo

$ mock -r fedora-rawhide-x86_64 init
...
$ mock -r fedora-rawhide-x86_64 --enablerepo=local --update unzip
...
Upgraded:
  unzip-6.0-60.fc38.x86_64
...
$ mock -r fedora-rawhide-x86_64 shell
...
<mock-chroot> sh-5.2# touch a
<mock-chroot> sh-5.2# zip test.zip a
  adding: a (stored 0%)
<mock-chroot> sh-5.2# unzip -t test.zip
Archive:  test.zip
    testing: a                        OK
No errors detected in compressed data of test.zip.

Comment 9 Lucian Langa 2023-01-30 16:03:17 UTC
Same issue exists for zip package, the reported error is the same, even the fix (fortify_source  2) works for it.

I was able to reproduce it with:

zip a52.zip -r Inbox/
*** buffer overflow detected ***: terminated


zip error: Interrupted (aborting)

Comment 10 Siddhesh Poyarekar 2023-01-30 16:19:16 UTC
(In reply to Lucian Langa from comment #9)
> Same issue exists for zip package, the reported error is the same, even the
> fix (fortify_source  2) works for it.
> 
> I was able to reproduce it with:
> 
> zip a52.zip -r Inbox/
> *** buffer overflow detected ***: terminated
> 
> 
> zip error: Interrupted (aborting)

It's likely a different bug because zip does not have any calls to wcstombs and doesn't appear to share that code with unzip.  I'll try to reproduce it.

Comment 11 Siddhesh Poyarekar 2023-01-30 16:44:30 UTC
Not reproducible with a simple local test on rawhide:

# ls somedir
NETBOOT_METHOD.TXT  RECIPE.TXT
# zip test.zip -r somedir/
updating: somedir/ (stored 0%)
  adding: somedir/NETBOOT_METHOD.TXT (stored 0%)
  adding: somedir/RECIPE.TXT (stored 0%)

I even tried compressing a glibc source tree and the zip30 source tree, but no crash.

Can you please file a separate bug with a minimal reproducer for zip?

Comment 12 Ben Cotton 2023-02-07 15:14:20 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 38 development cycle.
Changing version to 38.

Comment 13 Matthew Miller 2023-05-31 16:47:28 UTC
This does indeed appear to be fixed in 6.0-60