Bug 434168
| Summary: | exceptions.OverflowError when up2date'ing rpm bigger than 2GB | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 4 | Reporter: | Jan Pazdziora <jpazdziora> |
| Component: | up2date | Assignee: | Sayli Karmarkar <skarmark> |
| Status: | CLOSED ERRATA | QA Contact: | desktop-bugs <desktop-bugs> |
| Severity: | low | Docs Contact: | |
| Priority: | low | ||
| Version: | 4.6 | CC: | cperry |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | RHBA-2008-0771 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2008-07-24 20:05:58 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
If I wrap that fd.read() with try except and print the exception, I get: requested number of bytes is more than a Python string can hold An error has occurred: exceptions.OverflowError So ... we use that SmartIO thingy to download the file to a temporary file in tmp, and then try to read the whole content of that tmp file into memory. Is that correct? Hmmm. Plus, that f.write(fd.read()) in /usr/share/rhn/up2date_client/repoBackends/up2dateRepo.py:214 does not look like a good approach either. It slurps the whole file into memory and then writes it to the filePath. Why can't the self.s.getPackage get another argument, the directory to which to store that temporary file, and then just do a rename in that directory? This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release. If you need to generate some 2+GB rpm to test, look into bug 433795 for a spec file. An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2008-0771.html |
Description of problem: When I run up2date -i huge-rpm, up2date fails with An error has occurred: exceptions.OverflowError Version-Release number of selected component (if applicable): # rpm -qf /usr/lib/python2.3/site-packages/rhn/transports.py rhnlib-2.1.1-9.el4_6 # rpm -qf /usr/sbin/up2date up2date-4.6.2-7.el4_6.1 How reproducible: Deterministic. Steps to Reproduce: 1. Create rpm bigger than 2GB. 2. Push it to Satellite. 3. Register RHEL 4 system to that Satellite (and to the channel to which you pushed that huge rpm). 4. Run up2date -i huge-rpm. Actual results: Testing package set / solving RPM inter-dependencies... ######################################## huge-rpm-1.0-20080221.14221 ############################################ 11187 k/sec, 00:02:57 rem########################################################################################## 11259 k/sec, 00:01:37 rem################################################################################################### 11278 k/sec, 00:00:19 rem####################################################### Done. An error has occurred: exceptions.OverflowError See /var/log/up2date for more information Expected results: No error, rpm installed. Additional info: # tail -25 /var/log/up2date [Fri Feb 22 16:27:15 2008] up2date successfully retrieved authentication token from up2date server [Fri Feb 22 16:27:15 2008] up2date availablePackageList from network [Fri Feb 22 16:31:15 2008] up2date File "/usr/sbin/up2date", line 1273, in ? sys.exit(main() or 0) File "/usr/sbin/up2date", line 791, in main fullUpdate, dryRun=options.dry_run)) File "/usr/sbin/up2date", line 1127, in batchRun batch.run() File "/usr/share/rhn/up2date_client/up2dateBatch.py", line 92, in run self.__getPackages() File "/usr/share/rhn/up2date_client/up2dateBatch.py", line 223, in __getPackages up2date.getPackage(pkg, wrapperUtils.printPkg, wrapperUtils.printRetrieveHash) File "/usr/share/rhn/up2date_client/up2date.py", line 180, in getPackage buf = rpcServer.doCall(repos.getPackage, pkg, msgCallback, progressCallback) File "/usr/share/rhn/up2date_client/rpcServer.py", line 238, in doCall ret = apply(method, args, kwargs) File "/usr/share/rhn/up2date_client/repoDirector.py", line 37, in getPackage return self.handlers[channel['type']].getPackage(pkg, msgCallback, progressCallback) File "/usr/share/rhn/up2date_client/rpmSource.py", line 186, in getPackage package = source.getPackage(pkg, MsgCallback, progressCallback) File "/usr/share/rhn/up2date_client/repoBackends/up2dateRepo.py", line 214, in getPackage f.write(fd.read()) File "/usr/lib/python2.3/site-packages/rhn/transports.py", line 793, in read return fd.read() Strace shows: 3703 read(7, "n\0a\357\36\255\223)x\235!r\363\256\226\345s\233MfU\362"..., 15904) = 15904 3703 write(9, "\34\243)\240+\357h \203U\333\356i\324\274\264\200\336\337"..., 4096) = 4096 3703 write(9, "\223\265_0\245\340\32\267v\246\324w\t\\!\322d.\266r H\36"..., 4096) = 4096 3703 gettimeofday({1203694275, 326623}, NULL) = 0 3703 write(9, "\353\2232\331R1\'\306c\326\336\352\233\31=\320:\220\224"..., 4096) = 4096 3703 write(9, "\247\231\275\33\276\243\253\340\177\367d\321\351\1\226"..., 4096) = 4096 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, "#", 1) = 1 3703 write(1, " Done. ", 25) = 25 3703 write(1, "\n", 1) = 1 3703 write(9, "7\5\204g\320l\332\374\330\3248}\t2\264H\205g\362\322N\v"..., 3584) = 3584 3703 _llseek(9, 0, [0], SEEK_SET) = 0 3703 _llseek(9, 0, [0], SEEK_SET) = 0 3703 _llseek(9, 0, [0], SEEK_SET) = 0 3703 fstat64(9, {st_mode=S_IFREG|0755, st_size=2622242304, ...}) = 0 3703 _llseek(9, 0, [0], SEEK_CUR) = 0 3703 write(1, "An error has occurred:", 22) = 22 3703 write(1, "\n", 1) = 1 3703 write(1, "exceptions.OverflowError", 24) = 24 3703 write(1, "\n", 1) = 1 3703 gettimeofday({1203694275, 336605}, NULL) = 0 A lsof run while the up2date was still running shows that the file was actually correctly downloaded from the Satellite: up2date 3703 root 0u CHR 136,0 2 /dev/pts/0 up2date 3703 root 1u CHR 136,0 2 /dev/pts/0 up2date 3703 root 2u CHR 136,0 2 /dev/pts/0 up2date 3703 root 3r REG 253,0 8581120 340040 /var/lib/rpm/Packages up2date 3703 root 4r REG 253,0 12288 340046 /var/lib/rpm/Name up2date 3703 root 5r REG 253,0 94208 340042 /var/lib/rpm/Providename up2date 3703 root 6r REG 253,0 0 340250 /var/spool/up2date/huge-rpm-1.0-20080221.142211.noarch.rpm up2date 3703 root 7u IPv4 8482 TCP vmware166.englab.brq.redhat.com:32798->xen34.englab.brq.redhat.com:https (ESTABLISHED) up2date 3703 root 8u REG 253,0 0 340250 /var/spool/up2date/huge-rpm-1.0-20080221.142211.noarch.rpm up2date 3703 root 9u REG 253,0 2589831168 275287 /tmp/_rhn_transports-3703 (deleted) ^^ note the size of the /tmp/_rhn_transports-3703.