| Summary: | HTTP requests intermittently fail with non-sensical error: NoApplicableMethods | ||
|---|---|---|---|
| Product: | [Retired] Beaker | Reporter: | Dan Callaghan <dcallagh> |
| Component: | web UI | Assignee: | beaker-dev-list |
| Status: | CLOSED CANTFIX | QA Contact: | |
| Severity: | low | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 0.8 | CC: | bpeck, dcallagh, llim, mcsontos, rmancy, stl |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | Misc | ||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2013-07-29 01:25:38 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
Here is another one I saw today. It gave me a 500 instead of a 404 because it was an XML-RPC request (the new recipes.tasks.peer_roles call):
2013-04-03 18:31:36,101 cherrypy.msg INFO HTTP: Page handler: <bound method Root.RPC2 of <bkr.server.controllers.Root object at 0x7f47c2f00950>>
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/CherryPy-2.3.0-py2.6.egg/cherrypy/_cphttptools.py", line 121, in _run
self.main()
File "/usr/lib/python2.6/site-packages/CherryPy-2.3.0-py2.6.egg/cherrypy/_cphttptools.py", line 264, in main
body = page_handler(*virtual_path, **self.params)
File "<string>", line 3, in RPC2
File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 361, in expose
*args, **kw)
File "/usr/lib/python2.6/site-packages/turbogears/database.py", line 393, in run_with_transaction
@abstract()
File "/usr/lib/python2.6/site-packages/peak/rules/core.py", line 214, in __call__
raise self.__class__(*self.args+(args,kw)) # XXX
NoApplicableMethods: ((<function _expose at 0x7f47c36b59b0>, <function RPC2 at 0x7f47c2697938>, '', True, <bkr.server.controllers.Root object at 0x7f47c2f00950>), {})
(In reply to comment #2) The other interesting thing to note is that just yesterday I adjusted the mod_wsgi process settings on the box in question, from threads=1 processes=7 to threads=2 processes=2 (in an attempt at conserving memory). And now today I see this exception. So it fits with my theory that it is a race condition/thread safety issue in TurboGears or PEAK Rules. We should probably turn up our process life with any reduction in the number of processes, as it seems to be the destroying/creating in particular where the race condition is. This is how we've 'solved' the problem on other beaker installations. We are successfully working around this using threads=1: http://git.beaker-project.org/cgit/beaker/commit/?id=e89cfc2c and the ultimate fix is to simply stop using TurboGears, which we will be gradually working our way towards achieving (bug 988678). |
The get_distros AJAX call on the reserve workflow page (to /reserveworkflow/get_distro_options) intermittently returns a 404 for no good reason. Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/CherryPy-2.3.0-py2.6.egg/cherrypy/_cphttptools.py", line 121, in _run self.main() File "/usr/lib/python2.6/site-packages/CherryPy-2.3.0-py2.6.egg/cherrypy/_cphttptools.py", line 264, in main body = page_handler(*virtual_path, **self.params) File "<string>", line 3, in get_distro_options File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 374, in expose raise cherrypy.NotFound NotFound: (404, "The path '/reserveworkflow/get_distro_options' was not found.") Request Headers: COOKIE: tg-visit=4076ccd316cdd8b6765ab8b635043a91f4be33f0 Content-Length: USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.1) Gecko/20120209 Firefox/10.0.1 CONNECTION: keep-alive REFERER: http://localhost:9090/reserveworkflow/ X-REQUESTED-WITH: XMLHttpRequest HOST: localhost:9090 ACCEPT: application/json Remote-Addr: 127.0.0.1 ACCEPT-LANGUAGE: en-us,en;q=0.5 Content-Type: ACCEPT-ENCODING: gzip, deflate 127.0.0.1 - admin [22/Feb/2012:00:55:15 +0000] "GET /reserveworkflow/get_distro_options?tg_format=json&tg_random=1329872115108&arch=i386&arch=x86_64&distro_family=&tag= HTTP/1.1" 404 1475 "http://localhost:9090/reserveworkflow/" "Mozilla/5.0 (X11; Linux x86_64; rv:10.0.1) Gecko/20120209 Firefox/10.0.1" It seems to happen at random, with any combination of parameters. It's not reliably reproducible, though it happens very often for me when running the test suite. I can't find any evidence of it happening in production, which means the problem is probably connected to or triggered by the CherryPy server we use in tests. The underlying NoApplicableMethods exception comes from here: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 364, in expose *args, **kw) File "<generated code>", line 0, in run_with_transaction File "/usr/lib/python2.6/site-packages/peak/rules/core.py", line 153, in __call__ return self.body(*args, **kw) File "/usr/lib/python2.6/site-packages/turbogears/database.py", line 458, in sa_rwt retval = func(*args, **kw) File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 245, in _expose @abstract() File "/usr/lib/python2.6/site-packages/peak/rules/core.py", line 214, in __call__ raise self.__class__(*self.args+(args,kw)) # XXX NoApplicableMethods: ((<function get_distro_options at 0x3610758>, 'application/json', True, <bkr.server.reserve_workflow.ReserveWorkflow instance at 0x3876710>), {'distro_family': u'', 'tg_format': u'json', 'tag': u'', 'arch': u'i386', 'tg_random': u'1329873374091'}) though PEAK Rules is so convoluted and insane that I can't figure out why it might be happening.