Bug 522838

Summary: RFE: buildinstall caches headers/*.hdr on media
Product: [Fedora] Fedora Reporter: John Reiser <jreiser>
Component: pungiAssignee: Dennis Gilmore <dennis>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 18CC: dennis, fedora, vanmeeuwen+fedora
Target Milestone: ---Keywords: Reopened, Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-02-05 22:38:58 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description John Reiser 2009-09-11 17:52:14 UTC
Description of problem: During an install from DVD, the phase "Starting installation process" (just before installing the first package) seeks to every package on the media and creates a new file of its RPM headers in /mnt/sysimage/var/cache/yum/anaconda-InstallationRepo-*/headers/<package>.hdr.  The seeking takes a couple minutes (average seek time is about 140 milliseconds: 7 per second).  Rapid seeking causes excessive wear on the drive, and encourages scratches on the media.  Instead, buildinstall should create this cache headers/*.hdr on the media itself, and the installer should use it.  For 1081 packages in a minimal install on i686 (uncheck Office and Productivity) the space required is 29.4MB  (about 15 buffers full with the usual 2MB RAM buffer in a common DVD drive.)


Version-Release number of selected component (if applicable):
anaconda-12.23

How reproducible: every time


Steps to Reproduce:
1. boot from DVD for install
2. proceed to "Starting installation process"
3. listen for seeks on optical drive; use VT2 to inspect the directory /mnt/sysimage/var/cache/yum/anaconda-InstallationRepo-*/headers 
  
Actual results: A new file headers/<package>.hdr is created during installation for every package on the media.


Expected results: The headers/*.hdr files are created on the media by buildinstall.


Additional info:

Comment 2 Manuel Faux 2010-08-19 17:25:09 UTC

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 3 Dennis Gilmore 2012-08-03 05:53:23 UTC
closing as not a bug. there is no .hdr files any longer and has not been for a long time

Comment 4 John Reiser 2012-08-04 02:52:07 UTC
I am re-opening this bug because I still see *.hdr files during a fresh install of the current release (Fedora 17) from DVD.

The *.hdr files are created during the "Installation Starting" phase.  A dialog box displays a progress bar with the title "Starting installation process".  This is just after Dependency checking, which follows choosing the installation repos and clicking [Next].  There is one .hdr file created in /mnt/sysimage/var/cache/yum/x86_64/17/<repo_name>/headers for each package in that repo.

I booted a physical install DVD on bare hardware, and proceeded to the point of choosing which kind of installation (default Graphical Internet) and which repos (installation repo [the DVD], Fedora 17, and Fedora 17 updates).  Before clicking [Next], I switched to VT2 and started an strace of anaconda:
   # ps ax  |  grep anaconda
   # cd /tmp; strace -f -o "|gzip -c >strace.out.gz" -e trace=file -p <PID-of-anaconda> 2>/dev/null &
I saw two 'anaconda' processes, and chose the process with more accumulated CPU time.

After install finished and before rebooting, I copied the file /tmp/strace.out.gz to a safe place.  I inserted a flash memory device with pre-existing FAT filesystem, created a directory /my_flash for it, mounted the device on that directory, and copied the file into the FAT filesystem.

Examining the strace output, these are all the references to gnome-contacts-3.4.1-1.fc17.x86_64.hdr:
----- :g/gnome-contacts-3.4.1-1.fc17.x86_64.*hdr/p
645   stat("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr", 0x7fff96703810) = -1 ENOENT (No such file or directory)
645   open("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 55
645   utimes("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr", {{1335801449, 0}, {1335801449, 0}}) = 0
645   open("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr", O_RDONLY) = 55
645   open("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr", O_RDONLY) = 56
645   stat("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr", {st_mode=S_IFREG|0644, st_size=18157, ...}) = 0
645   open("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr", O_RDONLY) = 55
645   stat("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr", {st_mode=S_IFREG|0644, st_size=18157, ...}) = 0
645   unlink("/mnt/sysimage/var/cache/yum/x86_64/17/anaconda-0/headers/gnome-contacts-3.4.1-1.fc17.x86_64.hdr") = 0
-----
The strace reveals that a file of .st_size=18157 was created in /mnt/sysimage/var/acache/yum/x86_64/17/anaconda-0/headers.  Later, the file was open()ed three times and stat()ed twice before being removed.


As mentioned in the original Description of this RFE, the motivation for caching explicit *.hdr files, is to save clock time during installation, and to reduce wear on the drive and the platter.  If the *.hdr files already exist on the platter in one directory and their sectors are contiguous [the default physical arrangement], then copies can be transfered to harddrive using just one seek, and taking full advantage of the 2MB RAM buffer in the drive, and the default read-ahead strategy.  This is a couple _minutes_ faster than seeking to each individual .rpm, where the sectors containing the .hdr data are not contiguous, and interact poorly with the device buffering and read-ahead.

Comment 5 John Reiser 2012-08-31 03:16:56 UTC
The creation and use of .hdr files during install persists in Fedora 18 Alpha pre-TC4.  I change Product Version to 18.

While watching for some other problem, I noticed strace lines such as these during "Performing post-install ...":
-----
1125  stat("/tmp/yum.cache/fedora/packages/libtranslit-icu-0.0.2-3.fc18.x86_64.rpm", 0x7f77680ac0b0) = -1 ENOENT (No such file or directory)
1125  stat("/tmp/yum.cache/fedora/headers/libtranslit-icu-0.0.2-3.fc18.x86_64.hdr", {st_mode=S_IFREG|0644, st_size=3100, ...}) = 0
1125  unlink("/tmp/yum.cache/fedora/headers/libtranslit-icu-0.0.2-3.fc18.x86_64.hdr") = 0
-----

Comment 6 Fedora End Of Life 2013-12-21 14:51:59 UTC
This message is a reminder that Fedora 18 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 18. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '18'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 18's end of life.

Thank you for reporting this issue and we are sorry that we may not be 
able to fix it before Fedora 18 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior to Fedora 18's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 7 Fedora End Of Life 2014-02-05 22:38:58 UTC
Fedora 18 changed to end-of-life (EOL) status on 2014-01-14. Fedora 18 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.