Bug 850840 - repo uploads list/resume errors when you have paused uploads
repo uploads list/resume errors when you have paused uploads
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: user-experience (Show other bugs)
2.2 Beta
Unspecified Unspecified
medium Severity high
: ---
: 2.3.0
Assigned To: Barnaby Court
Preethi Thomas
: Reopened, Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-22 10:11 EDT by Preethi Thomas
Modified: 2013-12-09 09:31 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-12-09 09:31:19 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Preethi Thomas 2012-08-22 10:11:34 EDT
Description of problem:

repo uploads list/resume errors when you have paused uploads

Version-Release number of selected component (if applicable):

[root@pulp-candidate ~]# rpm -qa pulp-rpm-server
pulp-rpm-server-2.0.1-1.fc17.noarch
[root@pulp-candidate ~]# 

How reproducible:


Steps to Reproduce:
1. create a repo
2. run repo uploads pulp-admin repo uploads 
3. ctrl+c to pause uploads
4. run uploads list
5. run uploads resume
  
Actual results:
[root@pulp-candidate ~]# pulp-admin repo uploads  list
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

[ Paused  ] cat-1.0-1.noarch.rpm
[ Paused  ] camel-0.1-1.noarch.rpm
An unexpected error has occurred. More information can be found in the client
log file ~/.pulp/admin.log.


[root@pulp-candidate ~]# pulp-admin repo uploads resume
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

An unexpected error has occurred. More information can be found in the client
log file ~/.pulp/admin.log.




Expected results:


Additional info:


2012-08-22 09:55:08,144 - ERROR - Client-side exception occurred
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/client/extensions/core.py", line 475, in run
    exit_code = Cli.run(self, args)
  File "/usr/lib/python2.7/site-packages/okaara/cli.py", line 891, in run
    exit_code = command_or_section.execute(self.prompt, remaining_args)
  File "/usr/lib/python2.7/site-packages/okaara/cli.py", line 184, in execute
    return self.method(*arg_list, **clean_kwargs)
  File "/usr/lib/pulp/admin/extensions/pulp_upload/pulp_cli.py", line 125, in list
    message = template % (state, os.path.basename(upload.source_filename))
  File "/usr/lib64/python2.7/posixpath.py", line 112, in basename
    i = p.rfind('/') + 1
AttributeError: 'NoneType' object has no attribute 'rfind'
2012-08-22 09:56:24,418 - ERROR - Client-side exception occurred
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/client/extensions/core.py", line 475, in run
    exit_code = Cli.run(self, args)
  File "/usr/lib/python2.7/site-packages/okaara/cli.py", line 891, in run
    exit_code = command_or_section.execute(self.prompt, remaining_args)
  File "/usr/lib/python2.7/site-packages/okaara/cli.py", line 184, in execute
    return self.method(*arg_list, **clean_kwargs)
  File "/usr/lib/pulp/admin/extensions/pulp_upload/pulp_cli.py", line 78, in resume
    source_filenames = [os.path.basename(u.source_filename) for u in non_running_uploads]
  File "/usr/lib64/python2.7/posixpath.py", line 112, in basename
    i = p.rfind('/') + 1
AttributeError: 'NoneType' object has no attribute 'rfind'
Comment 1 Jay Dobies 2012-09-05 11:01:58 EDT
I can't seem to reproduce this. Did you do something in between the pause and resume, such as uninstall or upgrade your Pulp version? Do you see this consistently?


╭─[jdob]─[hinterlands]─[~/code/pulp]─[master ✔]─●
╰─➤ pulp-admin repo uploads list
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

[ Paused  ] thor-1-1.elfake.noarch.rpm
[ Paused  ] capt-1-1.elfake.noarch.rpm
[ Paused  ] widow-1-1.elfake.noarch.rpm
[ Paused  ] hawkeye-1-1.elfake.noarch.rpm
[ Paused  ] hulk-1-1.elfake.noarch.rpm
[ Paused  ] capt-1-1.elfake.noarch.rpm
[ Paused  ] hawkeye-1-1.elfake.noarch.rpm
[ Paused  ] ironman-1-1.elfake.noarch.rpm
[ Paused  ] hulk-1-1.elfake.noarch.rpm

╭─[jdob]─[hinterlands]─[~/code/pulp]─[master ✔]─●
╰─➤ pulp-admin repo uploads resume                                                  
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

Select one or more uploads to resume: 
  -  1 : thor-1-1.elfake.noarch.rpm
  -  2 : capt-1-1.elfake.noarch.rpm
  -  3 : widow-1-1.elfake.noarch.rpm
  -  4 : hawkeye-1-1.elfake.noarch.rpm
  -  5 : hulk-1-1.elfake.noarch.rpm
  -  6 : capt-1-1.elfake.noarch.rpm
  -  7 : hawkeye-1-1.elfake.noarch.rpm
  -  8 : ironman-1-1.elfake.noarch.rpm
  -  9 : hulk-1-1.elfake.noarch.rpm
Enter value (1-9) to toggle selection, 'c' to confirm selections, or '?' for
more commands:
Comment 2 Preethi Thomas 2012-09-05 11:47:41 EDT
Yup looks like this has been resolved
[root@preethi-el6-pulp ~]# rpm -q pulp-rpm-server
pulp-rpm-server-2.0.2-1.el6.noarch
[root@preethi-el6-pulp ~]# 

[root@preethi-el6-pulp ~]# pulp-admin repo create --repo-id rpmSuccessfully created repository [rpm]

[root@preethi-el6-pulp ~]# pulp-admin repo uploads rpm --repo-id rpm  -d /root/rpm/ -v
+----------------------------------------------------------------------+
                               RPM Upload
+----------------------------------------------------------------------+

RPMs to be uploaded:
  pulp-large_1mb_test-packageA-0.1.1-1.fc14.noarch.rpm
  pulp-large_1mb_test-packageB-0.1.1-1.fc14.noarch.rpm

Extracting necessary metdata for each RPM...
[==================================================] 100%
Analyzing: pulp-large_1mb_test-packageB-0.1.1-1.fc14.noarch.rpm
... completed

Creating upload requests on the server...
[==================================================] 100%
Initializing: pulp-large_1mb_test-packageB-0.1.1-1.fc14.noarch.rpm
... completed

Starting upload of selected packages. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
cancelled entirely using the cancel command.

Uploading: pulp-large_1mb_test-packageA-0.1.1-1.fc14.noarch.rpm
^CUploading paused

[root@preethi-el6-pulp ~]# ^C
[root@preethi-el6-pulp ~]# ^C
[root@preethi-el6-pulp ~]# ^C
[root@preethi-el6-pulp ~]# ^C
[root@preethi-el6-pulp ~]# ^C
[root@preethi-el6-pulp ~]# pulp-admin repo uploads list
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

[ Paused  ] pulp-large_1mb_test-packageA-0.1.1-1.fc14.noarch.rpm
[ Paused  ] pulp-large_1mb_test-packageB-0.1.1-1.fc14.noarch.rpm

[root@preethi-el6-pulp ~]# pulp-admin repo uploads resume
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

Select one or more uploads to resume: 
  -  1 : pulp-large_1mb_test-packageA-0.1.1-1.fc14.noarch.rpm
  -  2 : pulp-large_1mb_test-packageB-0.1.1-1.fc14.noarch.rpm
Enter value (1-2) to toggle selection, 'c' to confirm selections, or '?' for
more commands: 1

Select one or more uploads to resume: 
  x  1 : pulp-large_1mb_test-packageA-0.1.1-1.fc14.noarch.rpm
  -  2 : pulp-large_1mb_test-packageB-0.1.1-1.fc14.noarch.rpm
Enter value (1-2) to toggle selection, 'c' to confirm selections, or '?' for
more commands: c

Resuming upload for: pulp-large_1mb_test-packageA-0.1.1-1.fc14.noarch.rpm

Starting upload of selected packages. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
cancelled entirely using the cancel command.

Uploading: pulp-large_1mb_test-packageA-0.1.1-1.fc14.noarch.rpm
[==================================================] 100%
1051164/1051164 bytes
... completed

Importing into the repository...
... completed

Deleting the upload request...
... completed

[root@preethi-el6-pulp ~]# pulp-admin repo uploads list
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

[ Paused  ] pulp-large_1mb_test-packageB-0.1.1-1.fc14.noarch.rpm

[root@preethi-el6-pulp ~]# pulp-admin repo uploads resume
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

Select one or more uploads to resume: 
  -  1 : pulp-large_1mb_test-packageB-0.1.1-1.fc14.noarch.rpm
Enter value (1-1) to toggle selection, 'c' to confirm selections, or '?' for
more commands:
Comment 3 Preethi Thomas 2013-08-29 09:54:05 EDT
Looks like this is an issue on large uploads

[ Paused  ] nonaesthetical-5.7.1-1.noarch.rpm
[ Paused  ] undisconcerted-7.0.5-1.noarch.rpm
An unexpected error has occurred. More information can be found in the client
log file ~/.pulp/admin.log.

2013-08-29 09:52:16,731 - ERROR - Client-side exception occurred
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/client/extensions/core.py", line 478, in run
    exit_code = Cli.run(self, args)
  File "/usr/lib/python2.6/site-packages/okaara/cli.py", line 974, in run
    exit_code = command_or_section.execute(self.prompt, remaining_args)
  File "/usr/lib/python2.6/site-packages/pulp/client/extensions/extensions.py", line 224, in execute
    return self.method(*arg_list, **clean_kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/upload.py", line 452, in run
    message = template % (state, os.path.basename(upload.source_filename))
  File "/usr/lib64/python2.6/posixpath.py", line 111, in basename
    i = p.rfind('/') + 1
AttributeError: 'NoneType' object has no attribute 'rfind'
[root@cloud-qe-9 ~]#
Comment 4 Randy Barlow 2013-09-06 16:50:39 EDT
Since this won't be an MVP2 blocker, I'm going to lower the priority to medium.
Comment 5 Barnaby Court 2013-09-10 16:50:16 EDT
Preethi, I just tested with a 79 MB RPM upload and was able to pause and resume properly.  Can you provide details on either how big an RPM should be used or duplicate this issue on the current 2.3 build?  Thanks!
Comment 6 Jeff Ortel 2013-09-12 18:35:41 EDT
build: 2.3.0-0.10.alpha
Comment 7 Preethi Thomas 2013-09-18 14:08:07 EDT
verified
[root@pulp-v2-server ~]# rpm -q pulp-server
pulp-server-2.3.0-0.12.alpha.el6.noarch
[root@pulp-v2-server ~]# 

uploaded 5000 rpms ctrl+c to pause and ran upload resume

[root@pulp-v2-server ~]# pulp-admin rpm repo uploads rpm --repo-id=test_repo -d  /tmp/rpms/ 
+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: auguster-8.1.9-1.noarch.rpm
... completed

Creating upload requests on the server...
[======                                            ] 13%
Initializing: Michiganian-5.2.10-1.noarch.rpm
^CTraceback (most recent call last):
  File "/usr/bin/pulp-admin", line 9, in <module>
    load_entry_point('pulp-client-admin==2.3.0', 'console_scripts', 'pulp-admin')()
  File "/usr/lib/python2.6/site-packages/pulp/client/admin/__init__.py", line 35, in main
    config_files, exception_handler_class=AdminExceptionHandler
  File "/usr/lib/python2.6/site-packages/pulp/client/launcher.py", line 110, in main
    code = cli.run(args)
  File "/usr/lib/python2.6/site-packages/pulp/client/extensions/core.py", line 478, in run
    exit_code = Cli.run(self, args)
  File "/usr/lib/python2.6/site-packages/okaara/cli.py", line 974, in run
    exit_code = command_or_section.execute(self.prompt, remaining_args)
  File "/usr/lib/python2.6/site-packages/pulp/client/extensions/extensions.py", line 224, in execute
    return self.method(*arg_list, **clean_kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/upload.py", line 231, in run
    file_bundle.unit_key, file_bundle.metadata)
  File "/usr/lib/python2.6/site-packages/pulp/client/upload/manager.py", line 159, in initialize_upload
    response = self.bindings.uploads.initialize_upload().response_body
  File "/usr/lib/python2.6/site-packages/pulp/bindings/upload.py", line 26, in initialize_upload
    return self.server.POST(url)
  File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 106, in POST
    return self._request('POST', path, body=body, ensure_encoding=ensure_encoding)
  File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 150, in _request
    response_code, response_body = self.server_wrapper.request(method, url, body)
  File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 298, in request
    response = connection.getresponse()
  File "/usr/lib64/python2.6/httplib.py", line 990, in getresponse
    response.begin()
  File "/usr/lib64/python2.6/httplib.py", line 391, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.6/httplib.py", line 349, in _read_status
    line = self.fp.readline()
  File "/usr/lib64/python2.6/socket.py", line 433, in readline
    data = recv(1)
  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 228, in read
    return self._read_bio(size)
  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 213, in _read_bio
    return m2.ssl_read(self.ssl, size, self._timeout)
KeyboardInterrupt
[root@pulp-v2-server ~]# 

[root@pulp-v2-server ~]# pulp-admin rpm repo uploads list |more
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

[ Paused  ] cadgers-10.1.10-1.noarch.rpm
[ Paused  ] younghearted-8.8.4-1.noarch.rpm
[ Paused  ] benzopyrazolone-6.4.5-1.noarch.rpm
[ Paused  ] courts-10.2.7-1.noarch.rpm
[ Paused  ] imperatrice-3.5.4-1.noarch.rpm
[ Paused  ] witchmonger-0.1.3-1.noarch.rpm
[ Paused  ] platinize-10.10.5-1.noarch.rpm
[ Paused  ] oxidase-9.0.0-1.noarch.rpm



[root@pulp-v2-server ~]# pulp-admin rpm repo uploads resume |more
+----------------------------------------------------------------------+
                            Upload Requests
+----------------------------------------------------------------------+

Select one or more uploads to resume: 
  -  1 : cadgers-10.1.10-1.noarch.rpm
  -  2 : younghearted-8.8.4-1.noarch.rpm
  -  3 : benzopyrazolone-6.4.5-1.noarch.rpm
  -  4 : courts-10.2.7-1.noarch.rpm
Comment 8 Preethi Thomas 2013-12-09 09:31:19 EST
Pulp 2.3 released.

Note You need to log in before you can comment on or make changes to this bug.