Bug 1770464
| Summary: | regression in binutils strip | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Ali Akcaagac <aliakc> |
| Component: | binutils | Assignee: | Nick Clifton <nickc> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 32 | CC: | aoliva, dominik, dvlasenk, fweimer, jakub, jaruga, jin, nickc, ol+redhat, quantum.analyst |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-05-03 17:36:04 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hi Ali, (2-32.26 is a Fedora 31 release, not a rawhide release...) Yes, sorry about that. I have "improved" strip's ability to merge the data held in the .gnu.build.attributes section which has started to take up quite a large amount of space in built executables. But this improvement did come at the cost of speed. I have however now tweaked strip so that it does not perform this merge by default, so if you use 2-32.27 or 2.32-28, you should see the delay go away. Could you try one of them out, and update this BZ with the results ? Cheers Nick I upgraded to -28 today and confirm, that the empty "st*" files are still remaining. I ended up manually deleting 1300+ of those files from the directory. Also the time needed increased from once ~13 mins, to ~45 mins and with the recent update to ~63 minutes (all three had the same set of files to process). (In reply to Ali Akcaagac from comment #2) > I upgraded to -28 today and confirm, that the empty "st*" files are still > remaining. Darn - OK, i t was not the problem that I thought it was. Let me see if I can find a way to reproduce the problem locally, and then I can investigate. Koschei tells me that this update has broken all (that have been rebuilt so far) Haskell packages in Fedora 31 due to strip failing. (In reply to Elliott Sales de Andrade from comment #4) > Koschei tells me that this update has broken all (that have been rebuilt so > far) Haskell packages in Fedora 31 due to strip failing. OK - is there a simple way to reproduce the problem ? The files are temporary files that strip creates to hold its output. Once the strip has completed the file is renamed to whatever the correct output file name should be, or deleted if the strip failed. Since the files are not being renamed or deleted, something must be causing strip to terminate early. Are there any error messages, or core files being created ? > OK - is there a simple way to reproduce the problem ?
Yes! At least for me...
But before I tell you the steps... Now that I think further... The null byte st[0-9a-zA-Z]+ files might be related to "non-binary" files, where strip creates emtpy st[0-9a-zA-Z]+ files and forget to delete them ?
Easy reproduction steps:
1) Grab Fedora 31 Live Image x64_x86 and copy the entire contents to /.cdrom (create that folder for example).
2) You should now have a direct copy of /boot /bin /usr etc... inside the /.cdrom dir.
3) Open a Terminal (if not already done...)
4) execute this as root:
find /.cdrom -type f -exec strip {} \;
5) To get rid of all the empty st[0-9a-zA-Z]+ files:
find /.cdrom -type f -name "st[0-9a-zA-Z]*" -empty -delete
(In reply to Ali Akcaagac from comment #6) Hi Ali > 1) Grab Fedora 31 Live Image x64_x86 and copy the entire contents to /.cdrom > (create that folder for example). I think that we have different definitions of the term "easy" :-) Anyway I found that I could reproduce the problem with just one binary file, like this: % strip /usr/bin/col double free or corruption (!prev) Aborted (core dumped) This explains why the st... files are left lying around, the strip program is being terminated abnormally. Fortunately, it also turns out that this problem has already been reported (#1774507) and fixed. So please try out binutils-2.32-29.fc31, and let me know if this time the issue is resolved. Well actually - this should fix the temp-files-left-around issue, but I am not sure if it will fix the strip-takes-much-longer problem. So please let me know if there is any speed up in the strip process as well. Thanks. Cheers Nick Ok this looks interesting: Fri 22 Nov 2019 06:25:06 PM CET --- fkt_strip_binaries --- strip: "/.cdrom/" --- fkt_status_small --- time: "998 sec." Fri 22 Nov 2019 06:41:44 PM CET I belive that the coredumps caused a lot of writing on the HD, which caused the huge time consuming here. But this fix seem to have solved it and the time is back to "normality". Thanks! Also no st[0-9a-zA-Z]+ files created anymore. Hi, Today I faced an issue with Fedora 31: bunutils: 2.32-26.fc31 strip command when doing scratch build for samtools. [1] Let me comment here, as I think the issue is related to this ticket. How to reproduce the strip error. ``` $ fedpkg co samtools $ cd samtools $ git checkout f31 $ fedpkg srpm $ fedpkg scratch-build --srpm --nowait ``` The scratch build for f31. https://koji.fedoraproject.org/koji/taskinfo?taskID=39214153 https://kojipkgs.fedoraproject.org//work/tasks/4249/39214249/build.log Here is the error. Do you know what's the reason? build.log ``` + strip libbam.so.1 strip:libbam.so.1[.gnu.build.attributes]: corrupt GNU build attribute note: wrong note type: bad value error: Bad exit status from /var/tmp/rpm-tmp.korOzb (%install) ``` This error happened for f31 scratch build. But did not did not happen for rawhide, f30 and f29 scratch build using same samtools.spec file today. Following binutils and gcc were used as dependencies. * rawhide (ok) * binutils: 2.33.1-8.fc32 * gcc: 9.2.1-1.fc32.3 * f31 (error) * binutils: 2.32-26.fc31 * gcc: 9.2.1-1.fc31 * f30 (ok) * binutils: 2.31.1-34.fc30 * gcc: 9.2.1-1.fc30 * [1] https://src.fedoraproject.org/rpms/samtools/ Hi, I am wondering, is the fix from rawhide is going to be backported to Fedora 31? Currently it is not possible to do any Yocto project builds on Fedora 31 due to this issue... I tried recompiling gcc and binutils src rpms from the above mentioned rawhide versions under Fedora 31 and that did not really work out. I was also not able to install only these two packages from the rawhide repo. Is there any workaround for Fedora 31? (In reply to Sergey Bostandzhyan from comment #11) > I am wondering, is the fix from rawhide is going to be backported to Fedora 31? Please have a look at the buildsystem page: https://koji.fedoraproject.org/koji/packageinfo?packageID=310 Grab the -29.fc31 packages for your system. Thank you, I totally missed that! The following fixed the problem for me on Fedora 31: dnf install https://kojipkgs.fedoraproject.org//packages/binutils/2.32/29.fc31/x86_64/binutils-2.32-29.fc31.x86_64.rpm https://kojipkgs.fedoraproject.org//packages/binutils/2.32/29.fc31/x86_64/binutils-gold-2.32-29.fc31.x86_64.rpm (In reply to Sergey Bostandzhyan from comment #13) FYI - I have a Bodhi update request in place to move the fixed binutils packages into the F31 and F30 stable releases. This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle. Changing version to 32. The issue has been resolved for Fedora 32 and thus the issue can be closed. |
There seems to be a huge regression with binutils 2.32-26 strip: 1) strip takes noticable longer to strip than the strip that came delivered with binutils before above version. I have a directory here with around 15k files and directories mixed up. Stripping the executable files (binary) usually takes around 13 minutes on my underperformed system. I usually run something like "find <path> -type f -exec strip {} \;" over that path and let it work (the find line above is just an example). With the recent update I end up getting thousands of 0 byte files starting with st[a-zA-Z0-9]* in binary related paths. These files never occoured before, so there must be a regression. I had to manually remove those files. See example of a diff from two paths comparing before stripping and after stripping. I also belive that this is what's causing the huge performance issue. Stripping went from 13 minutes to over 45 minutes on my system. +-rw------- 0 root root 0 Nov 0 00:00 st01KEdv +-rw------- 0 root root 0 Nov 0 00:00 st04J1R2 +-rw------- 0 root root 0 Nov 0 00:00 st0DYQ99 +-rw------- 0 root root 0 Nov 0 00:00 st0E0cah +-rw------- 0 root root 0 Nov 0 00:00 st0e0IQJ +-rw------- 0 root root 0 Nov 0 00:00 st0qnnDw +-rw------- 0 root root 0 Nov 0 00:00 st0qYJtK +-rw------- 0 root root 0 Nov 0 00:00 st0ZMBN4 +-rw------- 0 root root 0 Nov 0 00:00 st11eHUr +-rw------- 0 root root 0 Nov 0 00:00 st18IT7o +-rw------- 0 root root 0 Nov 0 00:00 st1aTItV +-rw------- 0 root root 0 Nov 0 00:00 st1bZ26O +-rw------- 0 root root 0 Nov 0 00:00 st1DzEof +-rw------- 0 root root 0 Nov 0 00:00 st1KDgBn +-rw------- 0 root root 0 Nov 0 00:00 st1NkzP4 +-rw------- 0 root root 0 Nov 0 00:00 st1RZOdE +-rw------- 0 root root 0 Nov 0 00:00 st1Slkjd +-rw------- 0 root root 0 Nov 0 00:00 st1tqjo5 +-rw------- 0 root root 0 Nov 0 00:00 st1U22gG +-rw------- 0 root root 0 Nov 0 00:00 st1wo2jZ +-rw------- 0 root root 0 Nov 0 00:00 st1yotMQ +-rw------- 0 root root 0 Nov 0 00:00 st21qPce +-rw------- 0 root root 0 Nov 0 00:00 st22mKy3 +-rw------- 0 root root 0 Nov 0 00:00 st27xzLv +-rw------- 0 root root 0 Nov 0 00:00 st2b42P0 +-rw------- 0 root root 0 Nov 0 00:00 st2cJh1d +-rw------- 0 root root 0 Nov 0 00:00 st2HY1Ug +-rw------- 0 root root 0 Nov 0 00:00 st2i8Xf2 +-rw------- 0 root root 0 Nov 0 00:00 st2KDYXf +-rw------- 0 root root 0 Nov 0 00:00 st2oV0IW +-rw------- 0 root root 0 Nov 0 00:00 st2Z8EGS +-rw------- 0 root root 0 Nov 0 00:00 st33Uw7r