Bug 857528 - sync command should error out with appropriate error message for feed less repos.
sync command should error out with appropriate error message for feed less re...
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: rpm-support (Show other bugs)
2.0.6
Unspecified Unspecified
unspecified Severity unspecified
: ---
: Sprint 42
Assigned To: Jay Dobies
Preethi Thomas
: Triaged
: 877795 (view as bug list)
Depends On:
Blocks: 887372
  Show dependency treegraph
 
Reported: 2012-09-14 14:27 EDT by Preethi Thomas
Modified: 2013-01-09 12:08 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 887372 (view as bug list)
Environment:
Last Closed: 2013-01-09 12:08:37 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Preethi Thomas 2012-09-14 14:27:56 EDT
Description of problem:

feedless repos run sync command and then error out with 

Error during repository synchronization

[u'PulpExecutionException: Importer indicated a failed response\n']


It should error out with message before running the sync.

Version-Release number of selected component (if applicable):
[root@preethi-el6-pulp ~]# rpm -qa pulp-rpm-server
pulp-rpm-server-2.0.3-1.el6.noarch


How reproducible:


Steps to Reproduce:
1. create a feed less repo
2. run repo sync on the repo
3.
  
Actual results:
[root@preethi-el6-pulp ~]# pulp-admin repo sync  run  --repo-id preethi
+----------------------------------------------------------------------+
                   Synchronizing Repository [preethi]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... failed

Publishing packages...
[==================================================] 100%
Packages: 0/0 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[|]
... completed

Publishing repository over HTTP
[-]
... skipped

Publishing repository over HTTPS
[-]
... completed

Error during repository synchronization

[u'PulpExecutionException: Importer indicated a failed response\n']


Expected results:


Additional info:
Comment 1 Jay Dobies 2012-09-17 09:14:36 EDT
I think I know what you're getting at, but wanted to double check.

You say "It should error out with message before running the sync.". Are you suggesting that after the "... failed" in the first step it should not perform the remaining steps?
Comment 2 Preethi Thomas 2012-09-17 10:38:12 EDT
I was thinking more on the lines of what we did for v1.

In v1 what it does for feedless repo is as follows.

[root@pulp-v1-server ~]# pulp-admin -u admin -p admin repo sync --id test -F
error:  operation failed: Repo [test] is not setup for sync. Please add packages using upload.
Comment 3 Jay Dobies 2012-09-17 10:41:48 EDT
Ok, that's fair. There's two things here we need to fix:

1. The task subsystem needs to be fixed to not run the publish if the sync fails. That's being worked on this sprint elsewhere.

2. The progress report should indicate more than just failed for the sync case if there's no feed, that way we can show that in the CLI.
Comment 4 Jay Dobies 2012-11-27 15:52:06 EST
Prad - Please address point #2 in the above comment about having a better error message when trying to sync a repo that doesn't have a feed.
Comment 5 Pradeep Kilambi 2012-11-28 15:42:17 EST
https://bugzilla.redhat.com/show_bug.cgi?id=877795#c1 should address this bug. Moving it to modified.
Comment 6 Jeff Ortel 2012-11-29 16:30:14 EST
build: 2.0.6-0.10.beta
Comment 7 Preethi Thomas 2012-12-10 22:07:27 EST
fails-qa

feedless repo sync fails but there is no indication on what happened unless you are looking in the pulp.log

[root@preethi-el6-pulp ~]# pulp-admin rpm repo sync run --repo-id feedless
+----------------------------------------------------------------------+
                  Synchronizing Repository [feedless]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.



pulp.log

2012-12-10 19:07:23,239 pulp.plugins.yum_importer.importer:ERROR: Cannot perform repository sync on a repository with no feed
2012-12-10 19:07:23,252 pulp.server.dispatch.task:ERROR: Importer indicated a failed response
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/dispatch/task.py", line 123, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 144, in sync
    raise PulpExecutionException(_('Importer indicated a failed response'))
PulpExecutionException: Importer indicated a failed response
2012-12-10 19:07:23,254 pulp.server.dispatch.task:INFO: FAILURE: Task 4cb7658e-a01c-4419-aabc-f04f6d97147c: CallRequest: RepoSyncManager.sync(u'feedless', sync_config_override=None, importer_config={}, importer_instance=<yum_importer.importer.YumImporter object at 0x7f3bbe080a90>)
Comment 8 Jay Dobies 2012-12-11 08:23:10 EST
I was working on this yesterday as part of another bug. I'll close that one out as a dupe.
Comment 9 Jay Dobies 2012-12-11 08:23:45 EST
*** Bug 877795 has been marked as a duplicate of this bug. ***
Comment 10 Jay Dobies 2012-12-11 09:47:20 EST
CLI output after the fix:

╭─[jdob]─[hinterlands]─[~]─[]─●
╰─➤ pulp-admin rpm repo sync run --repo-id empty
+----------------------------------------------------------------------+
                    Synchronizing Repository [empty]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[-]
... failed

Cannot perform repository sync on a repository with no feed
Comment 11 Jay Dobies 2012-12-11 09:48:37 EST
RPM Git Repository: 141ca16e2037e205a450915ac843ac27ee8bca4c
Comment 12 Jeff Ortel 2012-12-13 10:17:13 EST
build: 2.0.6-0.15.beta
Comment 13 Preethi Thomas 2012-12-14 13:00:58 EST
failing this for puppet repo

[root@preethi ~]# pulp-admin puppet repo create --repo-id feedless1
Successfully created repository [feedless1]

[root@preethi ~]# pulp-admin rpm repo sync run --repo-id feedless1
+----------------------------------------------------------------------+
                  Synchronizing Repository [feedless1]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

[root@preethi ~]# 



From the log

2012-12-14 13:03:36,054 pulp_puppet.plugins.importers.sync:INFO: Beginning sync for repository <feedless1>
2012-12-14 13:03:36,055 pulp_puppet.plugins.importers.sync:INFO: Beginning metadata retrieval for repository <feedless1>
2012-12-14 13:03:36,055 pulp_puppet.plugins.importers.downloaders.factory:ERROR: Exception parsing feed type for feed <None>
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/downloaders/factory.py", line 106, in _determine_feed_type
    proto, netloc, path, params, query, frag = urlparse.urlparse(feed)
  File "/usr/lib64/python2.7/urlparse.py", line 135, in urlparse
    tuple = urlsplit(url, scheme, allow_fragments)
  File "/usr/lib64/python2.7/urlparse.py", line 174, in urlsplit
    i = url.find(':')
AttributeError: 'NoneType' object has no attribute 'find'
2012-12-14 13:03:36,055 pulp_puppet.plugins.importers.sync:ERROR: Exception while retrieving metadata for repository <feedless1>
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/sync.py", line 101, in _parse_metadata
    downloader = self._create_downloader()
  File "/usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/sync.py", line 339, in _create_downloader
    self.config, self.is_cancelled_call)
  File "/usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/downloaders/factory.py", line 65, in get_downloader
    feed_type = _determine_feed_type(feed)
  File "/usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/downloaders/factory.py", line 110, in _determine_feed_type
    raise InvalidFeed(feed)
InvalidFeed: None
2012-12-14 13:03:36,191 pulp.server.dispatch.task:ERROR: Importer indicated a failed response
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/dispatch/task.py", line 123, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py", line 158, in sync
    raise PulpExecutionException(_('Importer indicated a failed response'))
PulpExecutionException: Importer indicated a failed response
2012-12-14 13:03:36,192 pulp.server.dispatch.task:INFO: FAILURE: Task 0d40fa6d-f17e-4ca4-bb28-c885d5c1a172: CallRequest: RepoSyncManager.sync(u'feedless1', sync_config_override=None, importer_config={}, importer_instance=<pulp_puppet.plugins.importers.importer.PuppetModuleImporter object at 0x7fb19c53f6d0>)



Yum repo works fine though

[root@preethi ~]# pulp-admin rpm repo create --repo-id feedless
Successfully created repository [feedless]

[root@preethi ~]# pulp-admin rpm repo sync run --repo-id feedless
+----------------------------------------------------------------------+
                  Synchronizing Repository [feedless]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[-]
... failed

Cannot perform repository sync on a repository with no feed

[root@preethi ~]# 
[root@preethi ~]#
Comment 14 Jay Dobies 2012-12-14 15:06:43 EST
The original bug was technically filed against the yum plugins. I totally didn't think to look at the puppet ones. For organizational purposes, can you consider this fixed as per the yum plugins (based on your comment above) and file a separate bug that we're not clean for the puppet repos? While similar functionality, they are entirely separate.

Also align it to master, this won't be in for 2.0. Not ideal, but not catastrophic either.
Comment 15 Preethi Thomas 2012-12-14 15:13:36 EST
Moving to verified  as per the above 2 comments
Comment 16 Preethi Thomas 2013-01-09 12:08:37 EST
Pulp v2.0 released

Note You need to log in before you can comment on or make changes to this bug.