Bug 1016838 - Grinder has the potential to not update the symlink for a file when it's checksum has changed.
Grinder has the potential to not update the symlink for a file when it's chec...
Status: CLOSED ERRATA
Product: Red Hat Update Infrastructure for Cloud Providers
Classification: Red Hat
Component: RHUA (Show other bugs)
2.1.2
Unspecified Unspecified
medium Severity medium
: ---
: 2.1.3
Assigned To: John Matthews
mkovacik
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-08 15:22 EDT by John Matthews
Modified: 2013-12-17 15:10 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
This update fixes an issue with grinder not updating the symbolic links when a different version is downloaded. This problem would become evident when two repositories were synchronized that shared the same package. On the remote source the file changes with a different checksum and the two repositories would be re-synchronize. The first repository will download the new file and update the symbolic link then the second repo would see that the file has been downloaded and not update the broken symbolic link. This meant the yum repository would end up serving the wrong contents for the file. A fix has been placed in the grinder logic to ensure it always looks at the symbolic link and updates it it when needed. This means that the yum repositories are now serving the correct file.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-12-17 15:10:37 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 627063 None None None Never

  None (edit)
Description John Matthews 2013-10-08 15:22:07 EDT
Description of problem:

Grinder has a bug in checking when to update the symlink of a package, it's possible to change the checksum of the package, yet Grinder may not update the symlink, leaving the symlink of the package pointing to the older (incorrect) version.

To recreate:

You need 2 repos:
 Repo-A
 Repo-B

Both Repos need to initially have the same file.
 foo-1.0 with a checksum of say:  'value-1'

Sync both Repos.
 Repo-A and Repo-B contain a symlink pointing to
 foo-1.0 -> ../../../packages/foo/1.0/0/noarch/value-1

Now...on both repos replace foo-1.0 with new contents, but leave the filename the same.  (Try signing RPM with different key, or rebuild RPM but leave filename the same).

Re-run 'createrepo' so metadata is updated.

Now Repo-A and Repo-B contain foo-1.0 with checksum of 'value-2'

Here's the tricky part.

Sync both repos.
Assume Repo-A syncs first.
Repo-A will have an entry of:
 foo-1.0 -> ../../../packages/foo/1.0/0/noarch/value-2

Repo-B then will sync, 
 Grinder will see that the foo-1.0 with checksum 'value-2' already exists under "../../packages/foo/1.0/0/noarch/value-2", grinder will not re-download.  

 **Here is the bug....->
 Grinder should at this point update the symlink to point to the new checksum'd version, yet Grinder does not and the symlink is pointing to the original, "../../packages/foo/1.0/0/noarch/value-1" entry.
Comment 1 John Matthews 2013-10-08 15:24:52 EDT
Below is area in Grinder we need to fix.

https://git.fedorahosted.org/cgit/grinder.git/tree/src/grinder/BaseFetch.py#n200

        if os.path.exists(filePath) and \
            verifyExisting(filePath, itemSize, hashtype, checksum, verify_options) and not force:
            LOG.debug("%s exists with expected information, no need to fetch." % (filePath))
            if repofilepath is not None and not os.path.exists(repofilepath):
                relFilePath = GrinderUtils.get_relative_path(filePath, repofilepath)
                LOG.info("Symlink missing in repo directory. Creating link %s to %s" % (repofilepath, relFilePath))
                if not os.path.islink(repofilepath):
                    self.makeSafeSymlink(relFilePath, repofilepath)
            return (BaseFetch.STATUS_NOOP,None)
Comment 2 John Matthews 2013-10-11 17:14:51 EDT
Fixed in below commit.

https://git.fedorahosted.org/cgit/grinder.git/commit/?h=rhui&id=63537e7a4945545854876c8408cd21da458c51c9

QE:

This BZ is a bit subtle, below gives more details and one way to setup a test environment to verify this works.


The heart of the issue is that you need:
 - A HTTP repo which keeps the file path the same for a file, yet the underlying checksum changes
 - Grinder must be using a common storage directory for packages, (packages_location= is the term in the code)
 - Grinder must have already downloaded both versions to the common directory
 - Grinder must have an older symlink in the repo directory, pointing to the prior checksum version.


The original issue is that if the file contents exist locally in the common storage directory for packages, grinder would _only_ check if the symlink exists, if it did not exist it would be created and things were good...yet if the symlink _did_ exist then grinder would do nothing...it would never check the symlink was indeed correct.


One way to setup a test environment:

You want to create 2 directories each with the same filename, yet the file should have a different checksum.  

1) Grab 2 RPMs that have different checksums.
2) Copy one of the rpms to a new directory: dir_a
  cp foo-1.0.rpm dir_a
3) Copy other rpm to a new directory: dir_b
  cp foo-2.0.rpm dir_b/foo-1.0.rpm
Note:  We want to the same exact file name, yet we want the contents to be different.
4) Now run "createrepo" in both directories
5) Serve these repos from a local apache
6) Create a symlink for what you are serving so that the URL of the external repo will remain the same, yet you can change the internal contents.
  cd /var/www/html/pub
  ln -s dir_a repo

7) Make a common area to store the files
  mkdir -p /tmp/test/common

8) Run a grinder sync using the --packages_path option.
  grinder yum -U http://127.0.0.1/pub/repo --label changesym --packages_path /tmp/test/common

9) Verify your file exists under /tmp/test/common

10) Change the /var/www/html/pub/repo link to now point to "dir_b"

11) Resync the repo:
  grinder yum -U http://127.0.0.1/pub/repo --label changesym --packages_path /tmp/test/common

 Notice that the symlink is updated to the new version.  (This is good, yet it's not the bug...grinder prior to this fix would have done this.)

12) Change the /var/www/html/pub/repo link back to "dir_a"

13) Here is the actual test we want to prove the BZ works.
 Resync the repo:
 grinder yum -U http://127.0.0.1/pub/repo --label changesym --packages_path /tmp/test/common

Now check the link under the repo directory of your current working directory, we want to see that the symlink points to the checksum matching whatever you have exported through apache.
Comment 3 Vitaly Kuznetsov 2013-11-13 08:36:38 EST
Verified with grinder-0.0.138.3-1.el6_4.rhui.noarch

Note: to perform test sequence from Comment#2 custom patch for CLI is requred: https://git.fedorahosted.org/cgit/grinder.git/patch/?id=e1b0654bcd4899c8bab66b7d05789968f315d83b

1) Reproducing sequence with grinder-0.0.138.2-1.el6_4.rhui.noarch.rpm:

Afrer step 8:
[root@ip-10-49-122-58 ~]# ls -la changesym/
total 16
drwxr-xr-x. 3 root root 4096 Nov 13 08:17 .
dr-xr-x---. 5 root root 4096 Nov 13 08:17 ..
lrwxrwxrwx. 1 root root  142 Nov 13 08:17 relengo-1.2-6.fc19.noarch.rpm -> ../../tmp/test/common/relengo/1.2/6.fc19/noarch/725470f1182a05224fc79d2ab5434ab238fbb5412c7374fe98e62022d9eda58a/relengo-1.2-6.fc19.noarch.rpm
drwxr-xr-x. 2 root root 4096 Nov 13 08:17 repod

After step 11:
# ls -la changesym/
total 16
drwxr-xr-x. 3 root root 4096 Nov 13 08:19 .
dr-xr-x---. 5 root root 4096 Nov 13 08:17 ..
lrwxrwxrwx. 1 root root  142 Nov 13 08:19 relengo-1.2-6.fc19.noarch.rpm -> ../../tmp/test/common/relengo/1.2/6.fc19/noarch/f0f8b4a1790f6319c296ef5df9f7f795fe64a1c568a353dd0ff1e4b30522826d/relengo-1.2-6.fc19.noarch.rpm

After step 13:
# ls -la changesym/
total 16
drwxr-xr-x. 3 root root 4096 Nov 13 08:20 .
dr-xr-x---. 5 root root 4096 Nov 13 08:17 ..
lrwxrwxrwx. 1 root root  142 Nov 13 08:19 relengo-1.2-6.fc19.noarch.rpm -> ../../tmp/test/common/relengo/1.2/6.fc19/noarch/f0f8b4a1790f6319c296ef5df9f7f795fe64a1c568a353dd0ff1e4b30522826d/relengo-1.2-6.fc19.noarch.rpm


2) Verifying with grinder-0.0.138.3-1.el6_4.rhui.noarch:
After step 8:
# ls -la changesym/
total 16
drwxr-xr-x. 3 root root 4096 Nov 13 08:33 .
dr-xr-x---. 5 root root 4096 Nov 13 08:33 ..
lrwxrwxrwx. 1 root root  142 Nov 13 08:33 relengo-1.2-6.fc19.noarch.rpm -> ../../tmp/test/common/relengo/1.2/6.fc19/noarch/725470f1182a05224fc79d2ab5434ab238fbb5412c7374fe98e62022d9eda58a/relengo-1.2-6.fc19.noarch.rpm
drwxr-xr-x. 2 root root 4096 Nov 13 08:33 repodata

After step 11:
# ls -la changesym/
total 16
drwxr-xr-x. 3 root root 4096 Nov 13 08:34 .
dr-xr-x---. 5 root root 4096 Nov 13 08:33 ..
lrwxrwxrwx. 1 root root  142 Nov 13 08:34 relengo-1.2-6.fc19.noarch.rpm -> ../../tmp/test/common/relengo/1.2/6.fc19/noarch/f0f8b4a1790f6319c296ef5df9f7f795fe64a1c568a353dd0ff1e4b30522826d/relengo-1.2-6.fc19.noarch.rpm

After step 13:
# ls -la changesym/
total 16
drwxr-xr-x. 3 root root 4096 Nov 13 08:34 .
dr-xr-x---. 5 root root 4096 Nov 13 08:34 ..
lrwxrwxrwx. 1 root root  142 Nov 13 08:34 relengo-1.2-6.fc19.noarch.rpm -> ../../tmp/test/common/relengo/1.2/6.fc19/noarch/725470f1182a05224fc79d2ab5434ab238fbb5412c7374fe98e62022d9eda58a/relengo-1.2-6.fc19.noarch.rpm
Comment 6 errata-xmlrpc 2013-12-17 15:10:37 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1854.html

Note You need to log in before you can comment on or make changes to this bug.