Bug 1016612 - Pulp needs to have support for uploading srpms
Pulp needs to have support for uploading srpms
Product: Pulp
Classification: Community
Component: rpm-support (Show other bugs)
Unspecified Unspecified
medium Severity medium
: ---
: 2.4.0
Assigned To: Barnaby Court
Preethi Thomas
: Triaged
Depends On:
  Show dependency treegraph
Reported: 2013-10-08 08:10 EDT by Laurent Bigonville
Modified: 2014-09-29 19:03 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-08-09 02:57:13 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Laurent Bigonville 2013-10-08 08:10:12 EDT

Description of problem:
Today I tried to update pulp to 2.2 from 2.1 and pulp-manage-db is failing with a backtrace

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Update pulp RPM's to 2.2
2. run pulp-manage-db

Actual results:
It fails with the following backtrace:

# pulp-manage-db 
Beginning database migrations.
Migration package pulp.server.db.migrations is up to date at version 4
Migration package pulp_puppet.plugins.migrations is up to date at version 0
Applying pulp_rpm.migrations version 8
Migration to pulp_rpm.migrations version 8 complete.
Applying pulp_rpm.migrations version 9
Migration to pulp_rpm.migrations version 9 complete.
Applying pulp_rpm.migrations version 10
Migration to pulp_rpm.migrations version 10 complete.
Applying pulp_rpm.migrations version 11
Applying migration pulp_rpm.migrations.0011_new_importer failed.  See log for details.
2013-10-08 11:52:52,924 db:CRITICAL: Applying migration pulp_rpm.migrations.0011_new_importer failed.
2013-10-08 11:52:52,925 db:CRITICAL: 'NoneType' object has no attribute 'text'
2013-10-08 11:52:52,998 db:CRITICAL: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 79, in migrate_database
    update_current_version=not options.test)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/migrate/models.py", line 161, in apply_migration
  File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0011_new_importer.py", line 29, in migrate
  File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0011_new_importer.py", line 56, in _migrate_collection
    package['sourcerpm'] = format_element.find('sourcerpm').text
AttributeError: 'NoneType' object has no attribute 'text'

Database migrations complete.
Loading content types.
Content types loaded.

Expected results:
Upgrade being complete

Additional info:

After some investigation it seems that I have packages with arch=src and with (obviously) and empty "sourcerpm" in the units_rpm collection.

> db.units_rpm.find({arch: 'src', sourcerpm: ''}).count()

Is this expected? Shouldn't they be in the units_srpm instead.

To work around this issue, I changed the following line in 0011_new_importer.py:

package['sourcerpm'] = format_element.find('sourcerpm').text


    package['sourcerpm'] = format_element.find('sourcerpm').text
except AttributeError:
    package['sourcerpm'] = ''

I'm not really sure this is the proper way to fix this
Comment 1 Michael Hrivnak 2013-10-09 12:02:08 EDT
This turned out to be that the user had SRPMs in the RPM collection. This does point out that we lack a srpm upload command, so we should add that to resolve this issue.
Comment 2 Sayli Karmarkar 2013-10-09 12:13:30 EDT
Changing the summary according to #c1
Comment 3 Barnaby Court 2013-12-02 07:43:34 EST
We already support uploading RPMs manually using "pulp-admin rpm repo uploads srpm --repo-id <repoid> --file <filename>"

I verified this command using the current master branch.
Comment 4 Preethi Thomas 2014-04-16 10:37:52 EDT
[root@pulp-24-server upload]# rpm -qa pulp-server
[root@pulp-24-server upload]# 

[root@pulp-24-server upload]# pulp-admin rpm repo uploads srpm  --repo-id srpm -f /root/upload/test-srpm01-1.0-1.src.rpm -v
                              Unit Upload

Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: test-srpm01-1.0-1.src.rpm
... completed

Files to be uploaded:

Creating upload requests on the server...
[==================================================] 100%
Initializing: test-srpm01-1.0-1.src.rpm
... completed

Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
cancelled entirely using the cancel command.

Uploading: test-srpm01-1.0-1.src.rpm
[==================================================] 100%
1607/1607 bytes
... completed

Importing into the repository...
Import postponed due to queued operations against the repository. The progress
of this import can be viewed in the repository tasks list.

[root@pulp-24-server upload]#
Comment 5 Randy Barlow 2014-08-09 02:57:13 EDT
This has been fixed in Pulp 2.4.0-1.
Comment 6 Josh Baird 2014-09-29 19:03:40 EDT
I think I may be hitting the same bug.  I'm migrating from 2.3.1 to 2.4.1.

The pulp-manage-db fails:

Applying pulp_rpm.plugins.migrations version 9
Migration to pulp_rpm.plugins.migrations version 9 complete.
Applying pulp_rpm.plugins.migrations version 10
Migration to pulp_rpm.plugins.migrations version 10 complete.
Applying pulp_rpm.plugins.migrations version 11
Applying migration pulp_rpm.plugins.migrations.0011_new_importer failed.

Halting migrations due to a migration failure.  See log for details.
'NoneType' object has no attribute 'text'

The logs:

 pulp: pulp.server.db.manage:CRITICAL: Applying migration pulp_rpm.plugins.migrations.0011_new_importer failed.
 pulp: pulp.server.db.manage:CRITICAL:
 pulp: pulp.server.db.manage:CRITICAL: Halting migrations due to a migration failure.
 pulp: pulp.server.db.manage:CRITICAL: 'NoneType' object has no attribute 'text'
 pulp: pulp.server.db.manage:CRITICAL: Traceback (most recent call last):
 pulp: pulp.server.db.manage:CRITICAL:   File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 111, in main
 pulp: pulp.server.db.manage:CRITICAL:     _auto_manage_db(options)
 pulp: pulp.server.db.manage:CRITICAL:   File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 157, in _auto_manage_db
 pulp: pulp.server.db.manage:CRITICAL:     migrate_database(options)
 pulp: pulp.server.db.manage:CRITICAL:   File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 86, in migrate_database
 pulp: pulp.server.db.manage:CRITICAL:     update_current_version=not options.test)
 pulp: pulp.server.db.manage:CRITICAL:   File "/usr/lib/python2.6/site-packages/pulp/server/db/migrate/models.py", line 161, in apply_m

 pulp: pulp.server.db.manage:CRITICAL:     migration.migrate()
 pulp: pulp.server.db.manage:CRITICAL:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0011_new_importer.py", line

 pulp: pulp.server.db.manage:CRITICAL:     _migrate_collection(type_id)
 pulp: pulp.server.db.manage:CRITICAL:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0011_new_importer.py", line

 pulp: pulp.server.db.manage:CRITICAL:     package['sourcerpm'] = format_element.find('sourcerpm').text
 pulp: pulp.server.db.manage:CRITICAL: AttributeError: 'NoneType' object has no attribute 'text'
 pulp: pulp.server.db.manage:CRITICAL:

Note You need to log in before you can comment on or make changes to this bug.