Bug 682247 - Repo update is not atomic
Summary: Repo update is not atomic
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: user-experience
Version: 1.0.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: Sprint 29
Assignee: Jason Connor
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-03-04 15:35 UTC by Jay Dobies
Modified: 2014-03-31 01:39 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)

Description Jay Dobies 2011-03-04 15:35:05 UTC
<jdob> jortel: what's the strategy on repo update, is it atomic in the sense that if one of the new pieces of data fails nothing should be updated or is it a best effort approach where we update what we can?
<jortel> I was taking an all or nothing approach

Looking at the code, the update will fail if the caller attempts to change the relative path. But before that happens, it's possible things about the repo will have been altered:

Certs:
  value = self._write_certs_to_disk(id, {key:value})

Sync Schedule:
  if value:
    self._validate_schedule(value)
    repo_sync.update_schedule(repo)
  else:
    repo_sync.delete_schedule(repo)


In both cases, it's possible that the relative path code later will cause the entire call into the update API to raise an exception, however the repo data will still have changed.

Arguably, both of the above should come after the .save() call to the DB, though that still raises the potential for corrupt data if either of these calls fail.

Whichever tradeoff is chosen, we should write a comment in the code to mention that we explicitly acknowledge the possibility of the inconsistency so new eyes don't think it's an oversight.

Comment 1 Jason Connor 2011-09-09 23:13:17 UTC
I've added comments to the RepoApi.update method. However, this does not cure the issue. Updating a repo's schedule has a number of side-effects in the tasking system, so a valid schedule change has to bundled with these side-effects.

Perhaps we should move schedule adjustments out of the general repo update call and into it's own mechanism.

Comment 2 Jeff Ortel 2011-09-14 20:27:08 UTC
build: 0.231

Comment 3 Preethi Thomas 2011-09-23 19:27:36 UTC
fails_qa

<linear> preethi_lt: hmmm, this second one is tricky, I basically added comment saying that we won't fix it
<linear> preethi_lt: however
<linear> would you mind failing it, and adding the comment that if scheduling changes are not atomic, we need to move them into their own commands

Comment 4 Jay Dobies 2011-09-23 19:30:17 UTC
With the generic content repo API changes, this will be a non-issue. Scheduling is being pulled into its own API away from update.

Comment 5 Jason Connor 2011-10-25 00:10:15 UTC
schedule changes are now broken out from repo create and update

Comment 6 Jeff Ortel 2011-10-27 03:22:29 UTC
build 0.242-2

Comment 7 Preethi Thomas 2011-11-11 16:35:03 UTC
verified

[root@preethi pulp]# pulp-admin repo create --help
Usage: pulp-admin <options> repo create <options>

Options:
  -h, --help            show this help message and exit
  --id=ID               repository id (required)
  --name=NAME           common repository name
  --arch=ARCH           package arch the repository should support
  --feed=FEED           url feed to populate the repository
  --feed_ca=FEED_CA     path location to the feed's ca certificate
  --feed_cert=FEED_CERT
                        path location to the feed's entitlement certificate
  --feed_key=FEED_KEY   path location to the feed's entitlement certificate
                        key
  --consumer_ca=CONSUMER_CA
                        path location to the ca certificate used to verify
                        consumer requests
  --consumer_cert=CONSUMER_CERT
                        path location to the entitlement certificate consumers
                        will be provided at bind to grant access to this repo
  --consumer_key=CONSUMER_KEY
                        path location to the consumer entitlement certificate
                        key
  --relativepath=RELATIVEPATH
                        relative path where the repository is stored and
                        exposed to clients; this defaults to feed path if not
                        specified
  --groupid=GROUPID     a group to which the repository belongs; this is just
                        a string identifier
  --gpgkeys=KEYS        a ',' separated list of directories and/or files
                        containing GPG keys
  --checksum_type=CHECKSUM_TYPE
                        checksum type to use when yum metadata is generated
                        for this repo; default:sha256
  --notes=NOTES         additional information about repo in a dictionary form
                        inside a string
  --preserve_metadata   Preserves the original metadata; only works with feed
                        repos
  --content_type=CONTENT_TYPE
                        content type allowed in this repository; default:yum;
                        supported: [yum, file]
[root@preethi pulp]# pulp-admin repo update --help
Usage: pulp-admin <options> repo update <options>

Options:
  -h, --help            show this help message and exit
  --id=ID               repository id (required)
  --name=NAME           common repository name
  --arch=ARCH           package arch the repository should support
  --feed=FEED           url feed to populate the repository (repository must
                        be empty to change path component of the url)
  --feed_ca=FEED_CA     path location to the feed's ca certificate
  --feed_cert=FEED_CERT
                        path location to the feed's entitlement certificate
  --feed_key=FEED_KEY   path location to the feed's entitlement certificate
                        key
  --remove_feed_cert    if specified, the feed certificate information will be
                        removed from this repo
  --consumer_ca=CONSUMER_CA
                        path location to the ca certificate used to verify
                        consumer requests
  --consumer_cert=CONSUMER_CERT
                        path location to the entitlement certificate consumers
                        will be provided at bind to grant access to this repo
  --consumer_key=CONSUMER_KEY
                        path location to the consumer entitlement certificate
                        key
  --remove_consumer_cert
                        if specified, the consumer certificate information
                        will be removed from this repo
  --addgroup=ADDGROUP   group id to be added to the repository
  --rmgroup=RMGROUP     group id to be removed from the repository
  --addkeys=ADDKEYS     a ',' separated list of directories and/or files
                        containing GPG keys
  --rmkeys=RMKEYS       a ',' separated list of GPG key names

Comment 8 Preethi Thomas 2012-02-24 20:19:06 UTC
Pulp v1.0 is released
Closed Current Release.

Comment 9 Preethi Thomas 2012-02-24 20:19:49 UTC
Pulp v1.0 is released.


Note You need to log in before you can comment on or make changes to this bug.