Bug 760458

When repositories with relative_path are created simultaneously, sometimes there occurs an error while creating links from /var/lib/pulp/published to /var/lib/pulp.

I had 3 repositories relative_paths:

ACME_Corporation/Locker/Red_Hat_Enterprise_Linux_Server/Red_Hat_Enterprise_Linux_6_Server_-_Optional_RPMs_61_x86_64

ACME_Corporation/Locker/Red_Hat_Enterprise_Linux_Server/Red_Hat_Enterprise_Linux_6_Server_-_Optional_Beta_RPMs_61_x86_64

ACME_Corporation/Locker/Red_Hat_Enterprise_Linux_Server/Red_Hat_Enterprise_Linux_6_Server_-_Supplementary_RPMs_61_x86_64

I tried to clone them into this relative paths:

ACME_Corporation/Development/Red_Hat_Enterprise_Linux_Server/Red_Hat_Enterprise_Linux_6_Server_-_Optional_RPMs_61_x86_64

ACME_Corporation/Development/Red_Hat_Enterprise_Linux_Server/Red_Hat_Enterprise_Linux_6_Server_-_Optional_Beta_RPMs_61_x86_64

ACME_Corporation/Development/Red_Hat_Enterprise_Linux_Server/Red_Hat_Enterprise_Linux_6_Server_-_Supplementary_RPMs_61_x86_64


Unfortunately only for the first one a link was created in /var/lib/pulp/published/repos

In Pulp log there are the following errors:

2011-12-02 10:16:10,340 3983:140232896804608: pulp.server.api.repo:ERROR: repo:303 [Errno 17] File exists: '/var/lib/pulp/published/repos/ACME_Corporation/Development/Red_Hat_Enterprise_Linux_Server'
2011-12-02 10:16:10,342 3983:140233247008512: pulp.server.api.repo:ERROR: repo:303 [Errno 17] File exists: '/var/lib/pulp/published/repos/ACME_Corporation/Development/Red_Hat_Enterprise_Linux_Server'

It seems to me like a race condition, when creating directories for the link, something like [1]

The code that caused it is probably here:

     def _create_published_link(self, repo):
        if not os.path.isdir(self.published_path):
            os.makedirs(self.published_path)

By the time the condition is evaluated the directory doesn't exist, but it gets created before the makedirs call. In the makedirs implementation it's solved by caching excpetion with errno.EEXISTS, but that's only for subdirs, and not the final desired directory:

    head, tail = path.split(name)
    if not tail:
        head, tail = path.split(head)
    if head and tail and not path.exists(head):
        try:
            makedirs(head, mode)
        except OSError, e:
            # be happy if someone already created the path
            if e.errno != errno.EEXIST:
                raise
        if tail == curdir:           # xxx/newdir/. exists if xxx/newdir exists
            return
    mkdir(name, mode)


There are more places where this could cause a trouble.

[1] - http://bugs.python.org/issue1675