This service will be undergoing maintenance at 20:00 UTC, 2017-04-03. It is expected to last about 30 minutes
Bug 353171 - non-existence of *.sqlite-journal not cached during install
non-existence of *.sqlite-journal not cached during install
Product: Fedora
Classification: Fedora
Component: yum (Show other bugs)
All Linux
low Severity low
: ---
: ---
Assigned To: Seth Vidal
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2007-10-25 16:10 EDT by John Reiser
Modified: 2008-01-25 15:40 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-01-25 15:40:52 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description John Reiser 2007-10-25 16:10:10 EDT
Description of problem: The installer does not remember that sqlite journaling
is not in effect.  These filenames are looked up much too often:
  16469 /tmp/cache/anaconda-base-200710250923.x86_64/primary.sqlite-journal
   1064 /tmp/cache/anaconda-base-200710250923.x86_64/filelists.sqlite-journal
(the number is the number of times that the lookup failed.)

Version-Release number of selected component (if applicable):

How reproducible: always

Steps to Reproduce:
1. as soon as vtty2 becomes available:
strace -f -o '|gzip' -p <pid-of-anaconda>  > strace.out &
(you have to provide strace on a USB flash memory device before boot, and mount
the device yourself.)
2. check strace.out (63MB compressed, 1GB uncompressed) for ENOENT involving
Actual results:
  16469 /tmp/cache/anaconda-base-200710250923.x86_64/primary.sqlite-journal
   1064 /tmp/cache/anaconda-base-200710250923.x86_64/filelists.sqlite-journal

Expected results: Only two failing lookups (once per filename.)

Additional info:
Comment 1 Chris Lumens 2007-10-26 11:44:12 EDT
Is there a way we can tell sqlite that there's not going to be a journal file so
it should not bother looking for one over and over again?
Comment 2 Panu Matilainen 2007-12-21 07:03:34 EST
Checking for journal existence is part of sqlite locking mechanism (see, "dealing with hot journals"). AFAICT the
only way to avoid checking for the journal is grabbing an exclusive lock on the
db (doable at least with "begin exclusive" SQL-statement) but I suppose that'd
have to be done on yum-level and that in turn would probably have some other,
not necessarily wanted, implications...
Comment 3 Jeremy Katz 2007-12-21 09:10:39 EST
Actually, I don't know why we wouldn't want that.  At the yum level, we already
grab a global lock and don't allow other (correct :-) API users to do operations
at the same time.  So locking the sqlite dbs also doesn't seem unfair.
Comment 4 Panu Matilainen 2007-12-22 09:36:05 EST
Well, a quick experiment of sticking the below sledgehammer-approach patch to
yum-metadata-parser cuts down the tests for journal existence on "yum
--enablerepo=updates-testing update" on my box ATM from 5488 to 7 :)

---        2007-12-22 15:58:48.000000000 +0200
+++     2007-12-22 16:20:44.000000000 +0200
@@ -31,6 +31,9 @@
         con = sqlite.connect(filename)
         if sqlite.version_info[0] > 1:
             con.row_factory = sqlite.Row
+        cur = con.cursor()
+        cur.execute("pragma locking_mode = EXCLUSIVE")
+        del cur
         return con
     def getPrimary(self, location, checksum):

With sufficiently large number of accesses, it starts to even show up in
wallclock times: in apt-rpm usage patterns exclusive access to sqlite db cuts
something like ~7% from a full cache rebuild time. Yum's usage patterns are
wildly different but it can't hurt there either...
Comment 5 Seth Vidal 2008-01-25 15:40:52 EST
panu, thank you for the patch, I've applied it to y-m-p and I'm going to check
put it in rawhide shortly.

Note You need to log in before you can comment on or make changes to this bug.