Description of problem: Adding Ivan's comments from bug 795819 Ivan Necas 2012-02-24 05:58:49 EST I can confirm this error, but I'm not sure it's related directly to the description of the original bug. When reproducing this one, I've seen the packages not being positioned correctly, but I haven't seen the 'No such file or directory' error. Also on the machine with the original bug, there were no packages in incorrect place. The original bug appears in packages synchronized from remote repo.The strange thing is that for some packages synchronized from remote repo the link is not created. However it seems the package should be there (when cloning, it shouts that the package is missing). So at the end, most (but not all) links are created. Another observation: Pulp says: 2012-02-21 17:58:16,785 11788:140544395171584: pulp.server.api.synchronizers:INFO : synchronizers:783 YumSynchronizer reported 5632 successes, 4729 downloads, 1 er rors But in grinder there are only 4729 lines with line like: grinder.log:2012-02-21 17:55:35,435 11788:140544374191872: grinder.BaseFetch:INFO: activeobject:160 Create a link in repo directory for the package at /var/lib/pulp//repos/Seattle/Library/content/dist/rhel/server/6/6.1/x86_64/os//Packages/zlib-1.2.3-25.el6.i686.rpm to ../../../../../../../../../../../../packages/zlib/1.2.3/25.el6/i686/36a/zlib-1.2.3-25.el6.i686.rpm So it seems like the link was not created for packages, that were already downloaded for another repo (e.g. 6.2) Althought there are also lines in grinder.log like: grinder.log:2012-02-21 17:54:09,017 11788:140544374191872: grinder.BaseFetch:INFO: activeobject:160 Symlink missing in repo directory. Creating link /var/lib/pulp//repos/Seattle/Library/content/dist/rhel/server/6/6.1/x86_64/os//Packages/xulrunner-1.9.2.20-2.el6_1.x86_64.rpm to ../../../../../../../../../../../../packages/xulrunner/1.9.2.20/2.el6_1/x86_64/dd4/xulrunner-1.9.2.20-2.el6_1.x86_64.rpm There are only 177 for the repo. So we still missing links for over 700 packages I suspect this lines starting grinder/BaseFetch.py:193 to cause this troubles: # Acquire a write lock so no other process duplicates the effort grinder_write_locker = GrinderLock(filePath + '.lock') existing_lock_pid = grinder_write_locker.readlock() new_pid = os.getpid() if existing_lock_pid and int(existing_lock_pid) != new_pid and grinder_write_locker.isvalid(existing_lock_pid): # If there is an existing write pid # and if the pid is not same as the current pid # and pid is valid there is another process alive # and handling this, exit here. LOG.debug("another process is already handling this path [%s] and is alive; no need to process this again " % filePath) return (BaseFetch.STATUS_NOOP,None) Bare in mind that synchronization of 6.1 and 6.2 repos runs concurrently, there might be situations, that the package is being downloaded for another repo. In this case it's skipped and the symlink is no longer created.
Added a new status of REQUEUE, which allows us to add this item back to the item queue so the symlink can be created after the package download has completed. http://git.fedorahosted.org/git/?p=grinder.git;a=commitdiff;h=ba659f8ab36398f55f6fd17ecaa64b1aedc4858d Will be in Grinder 0.139
build: 0.270
verified [root@preethi-el6-pulp ~]# rpm -q pulp pulp-0.0.271-1.el6.noarch [root@preethi-el6-pulp ~]# ls -l /var/lib/pulp/repos/released/RHEL-6/6.1/Server/x86_64/os/Packages/ Display all 3480 possibilities? (y or n) [root@preethi-el6-pulp ~]# ls -l /var/lib/pulp/repos/released/RHEL-6/6.2/Server/x86_64/os/Packages/ 6.2/ [root@preethi-el6-pulp ~]# ls -l /var/lib/pulp/repos/released/RHEL-6/6.2/Server/x86_64/os/Packages/ Display all 3529 possibilities? (y or n) [root@preethi-el6-pulp ~]# pulp-admin repo list +------------------------------------------+ List of Available Repositories +------------------------------------------+ Id rhel6-2 Name rhel6-2 Repo URL https://preethi-el6-pulp.usersys.redhat.com/pulp/repos/released/RHEL-6/6.2/Server/x86_64/os/Server/ Feed URL http://download.devel.redhat.com/released/RHEL-6/6.2/Server/x86_64/os/Server/ Feed Type remote Content Type yum Feed Certs CA:No Cert:No Consumer Certs CA:No Cert:No Architecture noarch Sync Schedule 2012-02-29T16:20:00-05:00/PT2M Packages 3529 Files 0 Distributions None Publish True Clones [] Groups None Filters [] Notes {} Preserve Metadata False Checksum Type sha256 Id rhel5-8 Name rhel5-8 Repo URL https://preethi-el6-pulp.usersys.redhat.com/pulp/repos/released/RHEL-5-Client/U8/x86_64/os/Client/ Feed URL http://download.devel.redhat.com/released/RHEL-5-Client/U8/x86_64/os/Client/ Feed Type remote Content Type yum Feed Certs CA:No Cert:No Consumer Certs CA:No Cert:No Architecture noarch Sync Schedule 2012-02-29T16:20:00-05:00/PT2M Packages 2141 Files 0 Distributions None Publish True Clones [] Groups None Filters [] Notes {} Preserve Metadata False Checksum Type sha Id rhel6-1 Name rhel6-1 Repo URL https://preethi-el6-pulp.usersys.redhat.com/pulp/repos/released/RHEL-6/6.1/Server/x86_64/os/Server/ Feed URL http://download.devel.redhat.com/released/RHEL-6/6.1/Server/x86_64/os/Server/ Feed Type remote Content Type yum Feed Certs CA:No Cert:No Consumer Certs CA:No Cert:No Architecture noarch Sync Schedule 2012-02-29T16:20:00-05:00/PT2M Packages 3480 Files 0 Distributions None Publish True Clones [] Groups None Filters [] Notes {} Preserve Metadata False Checksum Type sha256
I verified the update fixed this problem on katello also.
Pulp v1.1 Release