Bug 682226

Fixed in this commit.

http://git.fedorahosted.org/git/?p=pulp.git;a=commit;h=892986f4e5e3d3e981fa0a86c4103dac3e041212

Build: 0.145

[root@pulp-qe verify]# pulp-admin repo create --id=foo
Successfully created repository [ foo ]

[root@pulp-qe verify]# rpm -qp --queryformat '%{NAME}-%{EPOCH}-%{VERSION}-%{RELEASE}-%{ARCH}\n' 2/test-0.1-1.noarch.rpm 
test-(none)-0.1-1-noarch
[root@pulp-qe verify]# rpm -qp --queryformat '%{NAME}-%{EPOCH}-%{VERSION}-%{RELEASE}-%{ARCH}\n' 1/test-0.1-1.noarch.rpm 
test-(none)-0.1-1-noarch
[root@pulp-qe verify]# rpm -qp --queryformat '%{NAME}-%{EPOCH}-%{VERSION}-%{RELEASE}-%{ARCH}\n' 3/test-0.1-1.noarch.rpm 
test-(none)-0.1-2-noarch

[root@pulp-qe verify]# cat test.csv 
test-0.1-1.noarch.rpm,4ce7ba37d5f922d6011cff6a125e4e34b86a3490014f3910f413432dc5836945
test-0.1-1.noarch.rpm,b76fced218dd3d7b8e09fd6defd64cdac682f570c0fd61ac9f4aeee64782aa52
test-0.1-1.noarch.rpm,c86f8e45b2d0261ce13542f2af8bbcc0d0042ae5171ceebdaa916c4372e1752e

[root@pulp-qe verify]# pulp-admin repo add_package --id=foo --csv=test.csv 
No Source repo specified, skipping dependency lookup
Successfully added packages [u'test-0.1-1.noarch', u'test-0.1-1.noarch', u'test-0.1-2.noarch'] to repo [foo].

============================
/var/log/pulp/pulp.log 

2011-03-08 14:15:05,073 [INFO][Dummy-4] get_packages_by_filename() @ repo.py:732 - looking up pkg filename [[u'test-0.1-1.noarch.rpm']] in repo [foo]
2011-03-08 14:15:05,075 [INFO][Dummy-4] add_package() @ repo.py:797 - Added: test-0.1-1.noarch.rpm to repo: foo, progress 0/3
2011-03-08 14:15:05,077 [ERROR][Dummy-4] add_package() @ repo.py:785 - Package with same NVREA [{'release': u'1', 'epoch': u'0', 'version': u'0.1', 'arch': u'noarch', 'name': u'test'}] already exists in repo [foo]
2011-03-08 14:15:05,078 [INFO][Dummy-4] get_packages_by_filename() @ repo.py:732 - looking up pkg filename [[u'test-0.1-1.noarch.rpm']] in repo [foo]
2011-03-08 14:15:05,080 [ERROR][Dummy-4] add_package() @ repo.py:791 - Package with same filename [test-0.1-1.noarch.rpm] already exists in repo [foo]
2011-03-08 14:15:05,081 [INFO][Dummy-4] add_package() @ repo.py:810 - inside of repo.add_packages() adding packages took 0.00978207588196 seconds
2011-03-08 14:15:05,327 [INFO][Dummy-4] create_repo() @ util.py:349 - [createrepo --checksum sha256 --update /var/lib/pulp//repos/foo] on /var/lib/pulp//repos/foo finished


Looks like the error needs to be raised all the way back up to the CLI.

We added errors to the API call, need to modify client to display the errors.

Changed repo::add_packages webservice call to return a list of errors, on success nothing is returned.


This is what the error looks like:

$ sudo pulp-admin repo add_package --id test_682226f -p zsh-4.3.10-4.fc12.x86_64.rpm -p zsh-4.3.10-4.fc12.x86_64.rpm  
No Source repo specified, skipping dependency lookup
Error unable to add: zsh-4.3.10-4.fc12.x86_64.rpm with sha256sum of 9f74bb00a2dd3e7486db617e6d10dbc1d1def131329faabe0d1a22ae7e2ebc5a
Errors occurred see /var/log/pulp/pulp.log for more info
Note: any packages not listed in error output have been added
1 packages added to repo [test_682226f]


This is an example of a successful add:

$ sudo pulp-admin repo add_package --id test_682226g -p zsh-4.3.10-4.fc12.x86_64.rpm 
No Source repo specified, skipping dependency lookup
Successfully added packages [u'zsh-4.3.10-4.fc12.x86_64'] to repo [test_682226g].
1 packages added to repo [test_682226g]


This is an example of a duplicate add (packages already existed in repo):
[jmatthews@jwm-devel demo_f12]$ sudo pulp-admin repo add_package --id test_682226e -p zsh-4.3.10-4.fc12.x86_64.rpm 
Package [zsh-4.3.10-4.fc12.x86_64.rpm] are already part of repo [test_682226e]. skipping


Commit is here
http://git.fedorahosted.org/git/?p=pulp.git;a=commitdiff;h=970550d97b31b39aaf95f9fba1cfc66cb8049771

Build: 0.151

Build: 0.151

verified

[root@preethi ~]# rpm -q pulp
pulp-0.0.151-1.fc14.noarch


[root@preethi ~]# pulp-admin repo add_package --id=foo --csv=test.csv 
Package [test-0.1-1.noarch.rpm] are already part of repo [foo]. skipping
No Source repo specified, skipping dependency lookup
Error unable to add: test-0.1-1.noarch.rpm with sha256sum of 4ce7ba37d5f922d6011cff6a125e4e34b86a3490014f3910f413432dc5836945
Error unable to add: test-0.1-1.noarch.rpm with sha256sum of 4ce7ba37d5f922d6011cff6a125e4e34b86a3490014f3910f413432dc5836945
Errors occurred see /var/log/pulp/pulp.log for more info
Note: any packages not listed in error output have been added
-1 packages added to repo [foo]


[root@preethi ~]# pulp-admin repo add_package --id=foo -p zenity-2.32.0-1.fc14.x86_64.rpm --source=f14

Following dependencies are suggested. ['glib2-2.26.0-2.fc14.x86_64.rpm', 'glibc-2.12.90-17.x86_64.rpm', 'pango-1.28.1-4.fc14.x86_64.rpm', 'gdk-pixbuf2-2.22.0-1.fc14.x86_64.rpm', 'freetype-2.4.2-3.fc14.x86_64.rpm', 'gtk2-2.22.0-1.fc14.1.x86_64.rpm', 'libpng-1.2.44-1.fc14.x86_64.rpm', 'cairo-1.10.0-1.fc14.x86_64.rpm', 'atk-1.30.0-5.fc14.x86_64.rpm', 'dbus-libs-1.4.0-1.fc14.x86_64.rpm', 'libnotify-0.5.1-1.fc14.x86_64.rpm', 'libX11-1.3.4-3.fc14.x86_64.rpm', 'dbus-glib-0.86-4.fc14.x86_64.rpm', 'fontconfig-2.8.0-2.fc14.x86_64.rpm'] 
Would you like us to add these?(Y/N/Q):N
Skipping dependencies
Successfully added packages ['zenity-2.32.0-1.fc14.x86_64'] to repo [foo].
1 packages added to repo [foo]

[root@preethi ~]# 
[root@preethi ~]# 
[root@preethi ~]# pulp-admin repo add_package --id=foo -p zenity-2.32.0-1.fc14.x86_64.rpm --source=f14
Package [zenity-2.32.0-1.fc14.x86_64.rpm] are already part of repo [foo]. skipping


[root@preethi ~]# pulp-admin repo add_package --id=foo --csv=test.csv 
No Source repo specified, skipping dependency lookup
Error unable to add: test-0.1-1.noarch.rpm with sha256sum of b76fced218dd3d7b8e09fd6defd64cdac682f570c0fd61ac9f4aeee64782aa52
Error unable to add: test-0.1-1.noarch.rpm with sha256sum of c86f8e45b2d0261ce13542f2af8bbcc0d0042ae5171ceebdaa916c4372e1752e
Errors occurred see /var/log/pulp/pulp.log for more info
Note: any packages not listed in error output have been added
1 packages added to repo [foo]

The constraint seems to be in application level (not sure if it is feasible in mongodb).

I managed to "add" 2 different copies of a package to a repo using content upload.
I think the package wasn't actually added, but no error was reported.


$ pulp-admin content upload --repoid=<repo> --nosig test_data/rpm1/testpkg-1-1.el5.noarch.rpm
* Starting Package Upload operation. See /var/log/pulp/client.log for more verbose output
* Performing Package Uploads to Pulp server
* Performing Repo Associations 
* Content Upload complete.

$echo $?
0


$ pulp-admin content upload --repoid=<repo> --nosig test_data/rpm1_signed_foo/testpkg-1-1.el5.noarch.rpm 
* Starting Package Upload operation. See /var/log/pulp/client.log for more verbose output
* Performing Package Uploads to Pulp server
* Performing Repo Associations 
* Content Upload complete.

$ echo $?
0

Daniel,

I'll get this addressed ASAP, thank you for the feedback.

Pulp was doing the right thing on the server api call (repo.add_packages).
The client code wasn't displaying the returned error messages.

I added more debug info to the error messages returned and the client will now display them for the content call, as well as return a non-zero exit code when an error is detected.

Example using 2 packages, same NEVRA and same filename but different contents.
Using test data from pulp/test/unit/data/sameNEVRA_differenceChecksums


Create an empty repo

$ sudo pulp-admin content upload --repoid test_b --nosig A/repo/pulp-test-package-same-nevra-0.1.0-1.x86_64.rpm --nosig 
* Starting Content Upload operation. See /var/log/pulp/client.log for more verbose output

* Performing Content Uploads to Pulp server

* Performing Repo Associations 

* Content Upload complete.

$ sudo pulp-admin content upload --repoid test_b --nosig B/repo/pulp-test-package-same-nevra-0.1.0-1.x86_64.rpm --nosig 
* Starting Content Upload operation. See /var/log/pulp/client.log for more verbose output

* Performing Content Uploads to Pulp server

* Performing Repo Associations 
Package with same NVREA [(u'pulp-test-package-same-nevra', u'0', u'0.1.0', u'1', u'x86_64')] already exists in repo [test_b]
Package with same filename [pulp-test-package-same-nevra-0.1.0-1.x86_64.rpm] already exists in repo [test_b]

* Content Upload complete.

Commit:
http://git.fedorahosted.org/git/?p=pulp.git;a=commitdiff;h=73823873b65a8df1ee72308f9b1a94496ca32b78

build: 0.177

[root@pulp-qe ~]# pulp-admin repo create --id=foo
^[[ASuccessfully created repository [ foo ]


[root@pulp-qe ~]# pulp-admin content upload --repoid=foo --nosig /tmp/sameNEVRA_differentChecksums/A/repo/pulp-test-package-same-nevra-0.1.0-1.x86_64.rpm 
* Starting Content Upload operation. See /var/log/pulp/client.log for more verbose output

* Performing Content Uploads to Pulp server

* Performing Repo Associations 

* Content Upload complete.

[root@pulp-qe ~]# pulp-admin content upload --repoid=foo --nosig /tmp/sameNEVRA_differentChecksums/B/repo/pulp-test-package-same-nevra-0.1.0-1.x86_64.rpm 
* Starting Content Upload operation. See /var/log/pulp/client.log for more verbose output

* Performing Content Uploads to Pulp server

* Performing Repo Associations 
Package with same NVREA [(u'pulp-test-package-same-nevra', u'0', u'0.1.0', u'1', u'x86_64')] already exists in repo [foo]
Package with same filename [pulp-test-package-same-nevra-0.1.0-1.x86_64.rpm] already exists in repo [foo]

* Content Upload complete.

Closing with Community Release 15

pulp-0.0.223-4.