Bug 1091404 - Make cache writing and locking more robust
Summary: Make cache writing and locking more robust
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-nitrate
Version: rawhide
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
Assignee: Petr Šplíchal
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1094105 (view as bug list)
Depends On:
Blocks: 798710 798713 798718
TreeView+ depends on / blocked
 
Reported: 2014-04-25 14:06 UTC by Ales Zelinka
Modified: 2016-06-01 01:47 UTC (History)
4 users (show)

Fixed In Version: python-nitrate-1.1-1.el7
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-15 08:13:05 UTC


Attachments (Terms of Use)
patch for the lock breaking (2.04 KB, patch)
2014-04-29 14:06 UTC, Ales Zelinka
no flags Details | Diff
atomic cache write patch (1.61 KB, patch)
2014-04-29 14:40 UTC, Ales Zelinka
no flags Details | Diff

Description Ales Zelinka 2014-04-25 14:06:27 UTC
I interrupt my script often during development. This almost always leads to cache being locked and corrupted. I then have to remove both the lock and the cache itself.

>  WARNING  Found lock /home/zelialx/.cache/nitrate.lock, opening read-only
>  ERROR  Failed to load the cache (CRC check failed 0x7274696e != 0x0L)
>  ERROR  Cache file: /home/zelialx/.cache/nitrate
>  WARNING  Going on but switching to CACHE_OBJECTS level



I suggest 

 * breaking the lock if the PID from the lockfile is no longer running. There's no point in keeping the cache locked for an already finished process.

 * writing the cache into a temporary file and only once the write is successfully finished, renaming this file to the official cache filename. This way the cache should never get corrupted when the script is interrupted mid-dump.

Comment 1 Ales Zelinka 2014-04-29 14:06:27 UTC
Created attachment 890809 [details]
patch for the lock breaking

Comment 3 Ales Zelinka 2014-04-29 14:40:36 UTC
Created attachment 890842 [details]
atomic cache write patch

Comment 4 Petr Šplíchal 2014-05-17 10:17:19 UTC
Thanks for the idea and patches, Aleš! These should definitely
make the persistent caching more reliable. I've committed a
slightly modified version of your patch for breaking stale locks:
https://github.com/psss/python-nitrate/commit/2874c9ce50f03c3a33b

Applied your patch for limiting the cache writing window:
https://github.com/psss/python-nitrate/commit/da2a9ac17adc96e9aae

And finally did some cleanup and moved the locking/unlocking code
into separate methods and enhanced the cache load error detection
to gracefully handle corrupted cache files as well:
https://github.com/psss/python-nitrate/commit/bef96f3dba50088d195

Now when the cache is corrupted an error is displayed and the
cache level is switched to CACHE_OBJECTS instead of bailing out.

Comment 5 Petr Šplíchal 2014-05-17 10:20:29 UTC
*** Bug 1094105 has been marked as a duplicate of this bug. ***

Comment 6 Fedora Update System 2016-02-10 17:20:10 UTC
python-nitrate-1.3-1.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-687d19409b

Comment 7 Fedora Update System 2016-02-10 18:22:45 UTC
python-nitrate-1.3-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-2bb9364c32

Comment 8 Fedora Update System 2016-02-10 18:50:49 UTC
python-nitrate-1.3-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-7474e957a9

Comment 9 Fedora Update System 2016-02-10 23:54:18 UTC
python-nitrate-1.3-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-35a106d076

Comment 10 Fedora Update System 2016-02-21 02:22:10 UTC
python-nitrate-1.3-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Comment 11 Fedora Update System 2016-02-21 16:27:24 UTC
python-nitrate-1.3-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 12 Fedora Update System 2016-02-27 02:01:12 UTC
python-nitrate-1.3-1.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.

Comment 13 Fedora Update System 2016-02-27 02:07:59 UTC
python-nitrate-1.3-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.