run following command at least in 2 parallel processes $ pulp-admin content upload --repoid=<repo> <rpm> (I used input duplication in Konsole to start the commands at the exactly same time) Result: at least one process fails I believe it's because Pulp can't recover from re-uploading the same file, this should pass when started over and over again: upload_id = upload_api.upload(pkg_path, checksum, chunksize) uploaded = upload_api.import_content(pkginfo, upload_id) import_content fails when a package is already imported
Consoles get the same input -> commands are executed at the same time. CONSOLE #1 ---------- $ pulp-admin content upload --repoid=<repo> test_data/rpm1/testpkg-1-1.el5.noarch.rpm --nosig * Starting Package Upload operation. See /var/log/pulp/client.log for more verbose output * Performing Package Uploads to Pulp server * Performing Repo Associations * Content Upload complete. CONSOLE #2 ---------- $ pulp-admin content upload --repoid=<repo> test_data/rpm1/testpkg-1-1.el5.noarch.rpm --nosig * Starting Package Upload operation. See /var/log/pulp/client.log for more verbose output * Performing Package Uploads to Pulp server * Performing Repo Associations error: operation failed: $ echo $? 244
The log says: IOError: [Errno 2] No such file or directory: '/var/lib/pulp//repos/<repo>/.repodata/other.xml.gz' Looks like 2 createrepo commands are running at the same time and one of them fails...
For this sprint, we are going to tighten up the checking/functionality to support simultaneous uploads of the same content. However, we do recognize a short-coming when it comes to "locking" across our model. So at this point, simultaneous operations on repositories is going to lead to potential issues. We will prioritize this in our next sprint, but will not be available for 3/15. So in the above scenario, I'd recommend refraining from simultaneous repository operations. Meaning, we should be able to handle: CONSOLE #1: $ pulp-admin content upload --repoid=<repoA> test_data/rpm1/testpkg-1-1.el5.noarch.rpm --nosig CONSOLE #2: $ pulp-admin content upload --repoid=<repoB> test_data/rpm1/testpkg-1-1.el5.noarch.rpm --nosig -Todd
Eliminated a race condition (related to package import) in package upload. I was able to produce the race condition and verify the fix by running: pulp-admin content upload python-urlgrabber-3.9.1-6.fc13.noarch.rpm -r foo1 & \ pulp-admin content upload python-urlgrabber-3.9.1-6.fc13.noarch.rpm -r foo2 & * Performing Package Uploads to Pulp server * Starting Package Upload operation. See /var/log/pulp/client.log for more verbose output * Performing Package Uploads to Pulp server * Performing Repo Associations * Performing Repo Associations * Content Upload complete. * Content Upload complete. [1]- Done pulp-admin content upload python-urlgrabber-3.9.1-6.fc13.noarch.rpm -r foo1 [2]+ Done pulp-admin content upload python-urlgrabber-3.9.1-6.fc13.noarch.rpm -r foo2 This test is probably not as close to parallel as dmach's test using Konsole but should be pretty close.
build: 0.146
[root@preethi ~]# pulp-admin content upload /root/upload/gofer-0.20-1.fc14.noarch.rpm -r upload1 --nosig -v & pulp-admin content upload /root/upload/gofer-0.20-1.fc14.noarch.rpm -r upload --nosig -v [1] 21700 * Starting Package Upload * Performing Package Uploads to Pulp server * Starting Package Upload * Performing Package Uploads to Pulp server Successfully uploaded [gofer-0.20-1.fc14.noarch.rpm] to server * Performing Repo Associations Successfully uploaded [gofer-0.20-1.fc14.noarch.rpm] to server * Performing Repo Associations Package association Complete for Repo [upload]: Packages: gofer-0.20-1.fc14.noarch.rpm Files: None * Content Upload complete. [root@preethi ~]# Package association Complete for Repo [upload1]: Packages: gofer-0.20-1.fc14.noarch.rpm Files: None * Content Upload complete. [1]+ Done pulp-admin content upload /root/upload/gofer-0.20-1.fc14.noarch.rpm -r upload1 --nosig -v [root@preethi ~]# echo $? 0 [root@preethi ~]#
Closing with Community Release 15 pulp-0.0.223-4.