Bug 846730

Summary: Exception when resolving dependencies using the yum importer
Product: [Retired] Pulp Reporter: Jay Dobies <jason.dobies>
Component: z_otherAssignee: Pradeep Kilambi <pkilambi>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 2.0.6CC: skarmark
Target Milestone: ---Keywords: Triaged
Target Release: Sprint 39   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-01-09 17:09:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jay Dobies 2012-08-08 14:22:40 UTC
The exception comes out of the yum importer but I'm not 100% sure it's not my server-side code or the REST request, so this may fall back on my plate to fix.


== Setup ==
Id:                 pulp-rhel6
Display Name:       pulp-rhel6
Description:        None
Content Unit Count: 35
Notes:              
Auto Publish:       True
Sync Config:        
  Feed: http://repos.fedorapeople.org/repos/pulp/pulp/v2/testing/fedora-17/x86_6
        4/
Publish Config:     
  Generate Metadata: True
  Http:              False
  Https:             True
  Relative URL:      /repos/pulp/pulp/dev/stable/6Server/i386/


== REST Call ==

In [7]: body
Out[7]: {'criteria': {'filters': {'unit': {'name': 'pulp'}}, 'type_ids': ['rpm']}}

In [8]: pic.POST('/v2/repositories/pulp-rhel6/actions/resolve_dependencies/', body=body)


== Exception ==


2012-08-08 10:17:10,004 6070:139662551262976: pulp.server.webservices.middleware.exception:ERROR: exception:46 Pulp exception occurred: PulpExecutionException
Traceback (most recent call last):
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/middleware/exception.py", line 44, in __call__
    return self.app(environ, start_response)
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/middleware/postponed.py", line 38, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 279, in wsgi
    result = self.handle_with_processors()
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 28, in _handle_with_processors
    return process(self.processors)
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 25, in process
    return p(lambda : process(processors))
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 558, in processor
    return handler()
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 25, in <lambda>
    return p(lambda : process(processors))
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 25, in process
    return p(lambda : process(processors))
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 573, in processor
    result = handler()
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 25, in <lambda>
    return p(lambda : process(processors))
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 27, in process
    return self.handle()
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 233, in handle
    return self._delegate(fn, self.fvars, args)
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 414, in _delegate
    return f()
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 422, in <lambda>
    f = lambda: self._delegate_sub_application(pat, what)
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 447, in _delegate_sub_application
    return app.handle_with_processors()
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 28, in _handle_with_processors
    return process(self.processors)
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 25, in process
    return p(lambda : process(processors))
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 558, in processor
    return handler()
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 25, in <lambda>
    return p(lambda : process(processors))
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 25, in process
    return p(lambda : process(processors))
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 573, in processor
    result = handler()
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 25, in <lambda>
    return p(lambda : process(processors))
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/application.py", line 27, in process
    return self.handle()
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 233, in handle
    return self._delegate(fn, self.fvars, args)
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 412, in _delegate
    return handle_class(cls)
  File "/usr/lib/python2.7/site-packages/web.py-0.32-py2.7.egg/web/application.py", line 387, in handle_class
    return tocall(*args)
  File "/home/jdob/code/pulp/platform/src/pulp/server/webservices/controllers/repositories.py", line 1033, in POST
    return execution.execute_sync_ok(self, call_request, timeout=timedelta(seconds=timeout))
  File "/home/jdob/code/pulp/platform/src/pulp/server/dispatch/task.py", line 130, in _run
    result = call(*args, **kwargs)
  File "/home/jdob/code/pulp/platform/src/pulp/server/managers/repo/dependency.py", line 53, in resolve_dependencies_by_criteria
    return self.resolve_dependencies_by_units(repo_id, units, options)
  File "/home/jdob/code/pulp/platform/src/pulp/server/managers/repo/dependency.py", line 114, in resolve_dependencies_by_units
    dep_report = importer_instance.resolve_dependencies(transfer_repo, transfer_units, conduit, call_config)
  File "/usr/lib/pulp/plugins/importers/yum_importer/importer.py", line 494, in resolve_dependencies
    dsolve = depsolver.DepSolver([repo], pkgs=pkglist)
  File "/home/jdob/code/pulp/rpm-support/src/pulp_rpm/yum_plugin/depsolver.py", line 35, in __init__
    self.loadPackages()
  File "/home/jdob/code/pulp/rpm-support/src/pulp_rpm/yum_plugin/depsolver.py", line 52, in loadPackages
    self._repostore.populateSack(which='all')
  File "/usr/lib/python2.7/site-packages/yum/repos.py", line 341, in populateSack
    sack.populate(repo, mdtype, callback, cacheonly)
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 186, in populate
    if self._check_db_version(repo, mydbtype):
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 294, in _check_db_version
    return repo._check_db_version(mdtype)
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1358, in _check_db_version
    repoXML = self.repoXML
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1536, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1532, in _getRepoXML
    self._loadRepoXML(text=self)
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1523, in _loadRepoXML
    return self._groupLoadRepoXML(text, self._mdpolicy2mdtypes())
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1497, in _groupLoadRepoXML
    if self._commonLoadRepoXML(text):
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1329, in _commonLoadRepoXML
    result = self._getFileRepoXML(local, text)
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1107, in _getFileRepoXML
    size=102400) # setting max size as 100K
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 933, in _getFile
    raise Errors.NoMoreMirrorsRepoError, errstr
PulpExecutionException: Pulp exception occurred: PulpExecutionException

Comment 1 Jay Dobies 2012-08-08 15:16:02 UTC
The criteria above produces no matching units (forgot we don't have a "pulp" package anymore). The following criteria will match units, however I see the same error.

{'criteria': {'filters': {'unit': {'name': 'pulp-server'}},
  'type_ids': ['rpm']}}

Comment 2 Pradeep Kilambi 2012-08-08 19:37:38 UTC
fixed! commit 4b37b94d6e6c7279c03848a46b9709faf672e31b

Comment 3 Jeff Ortel 2012-08-09 13:36:58 UTC
build: 0.322

Comment 4 Jay Dobies 2012-08-15 15:03:12 UTC
This isn't in the CLI, so to verify you'll have to use pic. Here are the steps:

$ ipython

Run the following, each on its own line, in ipython:

from pulp.common import pic

pic.connect()

body = {'criteria': {'filters': {'unit': {'name': <name>}}, 'type_ids': ['rpm']}}

pic.POST('/v2/repositories/<repo_id>/actions/resolve_dependencies/', body=body)



* For the body, substitute the value for a RPM you know to be in the repo where it says <name>. I downloaded the Pulp repository and used "pulp-server".

* For the last command, substitute in a valid repo ID for <repo_id>

The output is largely readable, but the important part of what you're looking for is that the server doesn't fall over itself with that giant traceback in pulp.log.

Comment 5 Preethi Thomas 2012-08-15 15:22:04 UTC
verified
[root@pulp-master ~]# rpm -q pulp-server
pulp-server-0.0.323-1.fc17.noarch

>>> pic.POST('/v2/repositories/pulp-rhel6/actions/resolve_dependencies/', body=body)
Request Body
{
  "criteria": {
    "type_ids": [
      "rpm"
    ], 
    "filters": {
      "unit": {
        "name": "pulp-server"
      }
    }
  }
}
Response Body
{
  "resolved": {
    "python-webpy": [
      {
        "name": "python-webpy", 
        "checksum": "69b776c6f96ad77a7279edfb39feba8a16918f973b6e72d834459fcc3a83858c", 
        "epoch": "0", 
        "version": "0.32", 
        "release": "9.fc17", 
        "arch": "noarch", 
        "checksumtype": "sha256"
      }
    ], 
    "python-oauth2 >= 1.5.170-2.pulp": [
      {
        "name": "python-oauth2", 
        "checksum": "f592e7152a3508fa611cac1c7e6dfc110fa4e9414b371ca8cd633ebf13d26408", 
        "epoch": "0", 
        "version": "1.5.170", 
        "release": "3.pulp.fc17", 
        "arch": "noarch", 
        "checksumtype": "sha256"
      }
    ], 
    "python-gofer >= 0.70": [
      {
        "name": "python-gofer", 
        "checksum": "d5fc33565494256e37f660ab9766355f73048331296b75bc3abaa4213d3f2f4a", 
        "epoch": "0", 
        "version": "0.71", 
        "release": "1.fc17", 
        "arch": "noarch", 
        "checksumtype": "sha256"
      }
    ], 
    "m2crypto >= 0.21.1.pulp-7": [
      {
        "name": "m2crypto", 
        "checksum": "6806b849546f1a51714a7a7f37d2bafae25263f2476cf3f380babedb0678bf67", 
        "epoch": "0", 
        "version": "0.21.1.pulp", 
        "release": "8.fc17", 
        "arch": "x86_64", 
        "checksumtype": "sha256"
      }
    ], 
    "python-pulp-common = 0.0.323": [
      {
        "name": "python-pulp-common", 
        "checksum": "69ea97beacfb50695d759ae095745bd11144a4f1a244fdedf8f85c705e37982e", 
        "epoch": "0", 
        "version": "0.0.323", 
        "release": "1.fc17", 
        "arch": "noarch", 
        "checksumtype": "sha256"
      }
    ], 
    "python-isodate >= 0.4.4-3.pulp": [
      {
        "name": "python-isodate", 
        "checksum": "79178d750cead582ed32be9282787222b5bd5094da9e40f1e7e21cd1833d6243", 
        "epoch": "0", 
        "version": "0.4.4", 
        "release": "5.pulp.fc17", 
        "arch": "noarch", 
        "checksumtype": "sha256"
      }
    ], 
    "mod_wsgi >= 3.3-3.pulp": [
      {
        "name": "mod_wsgi", 
        "checksum": "86407aabfec3885576bdc14da0613e6b5cc119b0de02fd75f2dabbbbb62ccd08", 
        "epoch": "0", 
        "version": "3.3", 
        "release": "4.pulp.fc17", 
        "arch": "x86_64", 
        "checksumtype": "sha256"
      }
    ], 
    "grinder >= 0.1.5-1": [
      {
        "name": "grinder", 
        "checksum": "7287122828123e4a18f96d98c1b8cf106ebd237424ab36946ff99c5fdfc1a362", 
        "epoch": "0", 
        "version": "0.1.5", 
        "release": "2.fc17", 
        "arch": "noarch", 
        "checksumtype": "sha256"
      }
    ]
  }, 
  "unresolved": {
    "crontabs": [], 
    "/usr/bin/python": [], 
    "python-setuptools": [], 
    "mod_ssl": [], 
    "python-BeautifulSoup": [], 
    "/bin/bash": [], 
    "python-simplejson >= 2.0.9": [], 
    "mongodb-server": [], 
    "mongodb": [], 
    "pymongo >= 1.9": [], 
    "qpid-cpp-server": [], 
    "httpd": [], 
    "openssl": [], 
    "python-httplib2": [], 
    "genisoimage": [], 
    "acl": [], 
    "python(abi) = 2.7": [], 
    "python-ldap": []
  }, 
  
>>>

Comment 6 Preethi Thomas 2013-01-09 17:09:23 UTC
Pulp v2.0 released