Bug 1252874

Summary:	doesn't start on F23
Product:	[Fedora] Fedora	Reporter:	Jiri Popelka <jpopelka>
Component:	docker-registry	Assignee:	Marek Goldmann <mgoldman>
Status:	CLOSED EOL	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	23	CC:	admiller, arydekul, lsm5, mgoldman
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-02-15 20:15:51 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jiri Popelka 2015-08-12 12:31:20 UTC

After upgrade to F23 I'm not able to start docker-registry at all.

Looking at ExecStart= in docker-registry.service and running it by hand:
# gunicorn --access-logfile - --max-requests 100 --graceful-timeout 3600 -t 3600 -k gevent -b 0.0.0.0:5000 -w 1 docker_registry.wsgi:application 
[2015-08-12 14:25:59 +0000] [30863] [INFO] Starting gunicorn 19.3.0
[2015-08-12 14:25:59 +0000] [30863] [INFO] Listening at: http://0.0.0.0:5000 (30863)
[2015-08-12 14:25:59 +0000] [30863] [INFO] Using worker: gevent
[2015-08-12 14:25:59 +0000] [30868] [INFO] Booting worker with pid: 30868
[2015-08-12 14:25:59 +0000] [30868] [ERROR] Exception in worker process:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
    super(GeventWorker, self).init_process()
  File "/usr/lib/python2.7/site-packages/gunicorn/workers/base.py", line 118, in init_process
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/site-packages/gunicorn/util.py", line 355, in import_app
    __import__(module)
ImportError: No module named wsgi
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
    super(GeventWorker, self).init_process()
  File "/usr/lib/python2.7/site-packages/gunicorn/workers/base.py", line 118, in init_process
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/site-packages/gunicorn/util.py", line 355, in import_app
    __import__(module)
ImportError: No module named wsgi
[2015-08-12 14:25:59 +0000] [30868] [INFO] Worker exiting (pid: 30868)
Traceback (most recent call last):
  File "/bin/gunicorn", line 9, in <module>
    load_entry_point('gunicorn==19.3.0', 'console_scripts', 'gunicorn')()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/base.py", line 189, in run
    super(Application, self).run()
  File "/usr/lib/python2.7/site-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 174, in run
    self.manage_workers()
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 477, in manage_workers
    self.spawn_workers()
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 541, in spawn_workers
    time.sleep(0.1 * random.random())
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 214, in handle_chld
    self.reap_workers()
  File "/usr/lib/python2.7/site-packages/gunicorn/arbiter.py", line 459, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

gunicorn seems to fail on 'import docker_registry.wsgi'.


docker-registry-0.9.1-2.fc23.noarch
python-gunicorn-19.3.0-2.fc23.noarch
(Looking at bug #1207308, those are the same versions as I used on F22, so I have no idea why it used to work previously and not now)

Comment 1 Alexander Rydekull 2016-01-18 21:21:12 UTC

It is very simple. The python library path is searching for wsgi.py in the wrong dir. Well, and all the other required stuff.

Fixed it quickly for myself with simple symlinks:
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/wsgi.py /usr/lib/python2.7/site-packages/docker_registry/wsgi.py
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/extensions /usr/lib/python2.7/site-packages/docker_registry/extensions
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/extras /usr/lib/python2.7/site-packages/docker_registry/extras
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/server /usr/lib/python2.7/site-packages/docker_registry/server
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/app.py /usr/lib/python2.7/site-packages/docker_registry/app.py
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/toolkit.py /usr/lib/python2.7/site-packages/docker_registry/toolkit.py
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/storage /usr/lib/python2.7/site-packages/docker_registry/storage
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/lib /usr/lib/python2.7/site-packages/docker_registry/lib

But yeah, adjusting the libpath for python will solve it (or moving the files).

Comment 2 Alexander Rydekull 2016-01-18 22:01:51 UTC

Oh, forgot three links.

ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/tags.py /usr/lib/python2.7/site-packages/docker_registry/tags.py
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/images.py /usr/lib/python2.7/site-packages/docker_registry/images.py
ln -s /usr/lib/python2.7/site-packages/docker-registry/docker_registry/index.py /usr/lib/python2.7/site-packages/docker_registry/index.py

Comment 3 Lokesh Mandvekar 2016-02-15 20:15:51 UTC

docker-registry has been obsoleted by docker-distribution upstream, and we're now in the process of obsoleting it in fedora. Please use 'docker-distribution' instead.