Bug 1010166

Created attachment 804763 [details]
patch for building 0.7.49

This looks pretty ok to me from a quick glance... 

Are we at the point where we can do a scratch build and test it somewhere?

Or was there some remaining issues here with versions we needed to sort out?

There were some version issues with one of the deps. Will follow it up this week.

Anshu, Kevin , please test this, and let me know your comments: 

SRPM: http://echevemaster.fedorapeople.org/askbot/askbot-0.7.49-1.fc19.src.rpm
SPEC: http://echevemaster.fedorapeople.org/askbot/askbot.spec

@Echevemaster: its complaining about Lamson


[root@apps-fp-oi-00000340 /]# askbot-setup 
Traceback (most recent call last):
  File "/usr/bin/askbot-setup", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 3007, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: Lamson

and some more dependencies after manually installing lamson as well.

Did another spec and patch. This build does the initial askbot-setup run on epel-6 and fedora-20 mock, need to test it more though.

Following https://bugzilla.redhat.com/show_bug.cgi?id=972251 , I have not included lamson (and celery which needs lamson).

From comment 6 in above ticket

https://bugzilla.redhat.com/show_bug.cgi?id=972251#c6

c) drop the software entirely.  askbot has a dependency on it but I think it only is needed when running it using celery which we don't and don't plan to.

srpm and spec available at http://anshprat.fedorapeople.org/rhbz1010166/c7/

@anshu, sorry for delay, I was in an event in brazil and kept me very busy

[echevemaster@localhost SPECS]$ askbot-setup 

Deploying Askbot - Django Q&A forum application
Problems installing? -> please email admin

To CANCEL - hit Ctr-C at any time

Enter directory path (absolute or relative) to deploy
askbot. To choose current directory - enter "."
> 
Enter directory path (absolute or relative) to deploy
askbot. To choose current directory - enter "."
> .

Please select database engine:
1 - for postgresql, 2 - for sqlite, 3 - for mysql, 4 - oracle (type 1/2/3/4)
> 2
Please enter database file name
> askbot-test
Copying files: 
* __init__.py 
* manage.py 
* urls.py 
* django.wsgi 
Creating settings file
settings file created

copying directories:  * doc
* cron
* upfiles
Traceback (most recent call last):
  File "/usr/bin/askbot-setup", line 9, in <module>
    load_entry_point('askbot==0.7.49', 'console_scripts', 'askbot-setup')()
  File "/usr/lib/python2.6/site-packages/askbot/deployment/__init__.py", line 126, in askbot_setup
    deploy_askbot(options_dict)
  File "/usr/lib/python2.6/site-packages/askbot/deployment/__init__.py", line 186, in deploy_askbot
    context=options
  File "/usr/lib/python2.6/site-packages/askbot/deployment/path_utils.py", line 233, in deploy_into
    shutil.copytree(src, dst)
  File "/usr/lib64/python2.6/shutil.py", line 136, in copytree
    names = os.listdir(src)
OSError: [Errno 2] No such file or directory: '/usr/lib/python2.6/site-packages/askbot/upfiles'

Due that this dir have been deleted and moved to another location, needs to review that location, BTW, this is a oldest bug and was closed
https://bugzilla.redhat.com/show_bug.cgi?id=752586
I will to look more about that

alright, I fixed something similar for upfiles in a hacky way. I am also tied up this week, will get back to it after 10/26 onwards.

@Anshu I think that this is OK, let me know for build 
http://echevemaster.fedorapeople.org/askbot/1/askbot.spec
http://echevemaster.fedorapeople.org/askbot/1/askbot-0.7.49-1.el6.src.rpm

@echevemaster - It still has one issue that even my build has -

<mock-chroot>[root@apps-fp-oi-00000340 a]# python settings.py 
Traceback (most recent call last):
  File "settings.py", line 239, in <module>
    import djcelery
ImportError: No module named djcelery


We need to clean this up from the settings file. I did one patch for this but looks like even seeting.py.mustache might need cleanup.

The above was the root of the problem. The way the above problem manifests itself is as below:

<mock-chroot>[root@apps-fp-oi-00000340 a]# python manage.py runserver
Error: Can't find the file 'settings.py' in the directory containing 'manage.py'. It appears you've customized things.
You'll have to run django-admin.py, passing it your settings module.
(If the file settings.py does indeed exist, it's causing an ImportError somehow.)

http://echevemaster.fedorapeople.org/askbot/2/askbot-0.7.49-2.el6.src.rpm
http://echevemaster.fedorapeople.org/askbot/2/askbot.spec

First I've removed django-celery from my centos box

Solved!!
Added django-celery (and retrieved) and tested with
python manage.py collectstatic and the python console 

[echevemaster@localhost forum]$ python 
Python 2.6.6 (r266:84292, Jul 10 2013, 22:48:45) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import settings
>>> 

and 

python manage.py runserver

0 errors found
Django version 1.4.8, using settings 'forum.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

mm , I see that spec file has django-celery present. I think removing celery should be the way to go since celery depends upon lamson which is no longer available in fedora.

https://bugzilla.redhat.com/show_bug.cgi?id=972251#c6

If we want to keep celery, then we are good with this build. I would like to do a scratch build and run it in ask stg once after clarifying the above and before releasing the package.

Got this error:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_manager(settings)
  File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 459, in execute_manager
    utility.execute()
  File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 231, in execute
    self.validate()
  File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 266, in validate
    num_errors = get_validation_errors(s, app)
  File "/usr/lib/python2.6/site-packages/django/core/management/validation.py", line 30, in get_validation_errors
    for (app_name, error) in get_app_errors().items():
  File "/usr/lib/python2.6/site-packages/django/db/models/loading.py", line 158, in get_app_errors
    self._populate()
  File "/usr/lib/python2.6/site-packages/django/db/models/loading.py", line 67, in _populate
    self.load_app(app_name)
  File "/usr/lib/python2.6/site-packages/django/db/models/loading.py", line 88, in load_app
    models = import_module('.models', app_name)
  File "/usr/lib/python2.6/site-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/usr/lib/python2.6/site-packages/djcelery/models.py", line 13, in <module>
    from celery import schedules
  File "/usr/lib/python2.6/site-packages/celery/schedules.py", line 2, in <module>
    from dateutil.relativedelta import relativedelta
ImportError: No module named dateutil.relativedelta


I ve fixed most of these, let me put my spec and srpm ( it still has the celery issue)

missing python-dateutil :(

ok, I fixed the celery thing as well.

I had fixed the upfiles issue , will check how you have done it and copy from yours or improve mine as required.

http://anshprat.fedorapeople.org/rhbz1010166/c17/askbot-0.7.49-1.el6.src.rpm
http://anshprat.fedorapeople.org/rhbz1010166/c17/askbot.spec

One question, why remove celery, if since 6 months ago was put in production because improved the speed sending emails?, ok lamson isn't in fedora but, can't  be celery used in others tasks?

https://fedorahosted.org/fedora-infrastructure/ticket/3763

Two reasons -

1) I thought celery has dependency on lamson
2) django-celery and python-celery need to be of same version which is not true today for fedora. (Django-celery version looks for a matching python-celery version)

So if (1) above is not true, then we need to fix versions of django-celery / python-django-celery and django-celery to satisfy the (2) above before going for building askbot.

https://admin.fedoraproject.org/updates/search/celery

Other thing about this lines:
%if 0%{?rhel}
Requires:       python-dateutil15
%else
Requires:       python-dateutil
%endif

[root@localhost ~]# yum install python-dateutil15

[root@localhost ~]# python
Python 2.6.6 (r266:84292, Jul 10 2013, 22:48:45) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from dateutil.relativedelta import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named dateutil.relativedelta


yum install python-dateutil

[root@localhost ~]# python
Python 2.6.6 (r266:84292, Jul 10 2013, 22:48:45) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from dateutil.relativedelta import *
>>> 

So python-dateutil15 isn't right for epel or fedora (same version 1.5) . Still looking why this happens. it's weird regarding a higher version

Ah, I didnt notice the above.

I added another required for python-dateutil and hence ..

<mock-chroot>[root@apps-fp-oi-00000340 a]# rpm -qa|grep date
python-dateutil15-1.5-1.el6.noarch
python-dateutil-1.4.1-6.el6.noarch

Removing one of the above two requires should work.

Removing python-dateutil15 works it, but,  what we will doing with fedora-versions? have 1.5 by default.

Fedora has other problems as well -

Enter directory path (absolute or relative) to deploy
askbot. To choose current directory - enter "."
> .

Please select database engine:
1 - for postgresql, 2 - for sqlite, 3 - for mysql, 4 - oracle (type 1/2/3/4)
> 2
Please enter database file name
> b
Copying files: 
* __init__.py 
* manage.py 
* urls.py 
* django.wsgi 
Creating settings file
Traceback (most recent call last):
  File "/usr/bin/askbot-setup", line 9, in <module>
    load_entry_point('askbot==0.7.49', 'console_scripts', 'askbot-setup')()
  File "/usr/lib/python2.7/site-packages/askbot/deployment/__init__.py", line 126, in askbot_setup
    deploy_askbot(options_dict)
  File "/usr/lib/python2.7/site-packages/askbot/deployment/__init__.py", line 186, in deploy_askbot
    context=options
  File "/usr/lib/python2.7/site-packages/askbot/deployment/path_utils.py", line 192, in deploy_into
    settings_contents = SettingsTemplate(context).render()
  File "/usr/lib/python2.7/site-packages/askbot/deployment/template_loader.py", line 13, in __init__
    super(SettingsTemplate, self).__init__(context=context, **kwargs)
TypeError: object.__init__() takes no parameters


It requires psytache-0.3.1

http://askbot.org/en/question/11729/object__init__-takes-no-parameters/?answer=11785#post-id-11785

So what do we do about this?

<mock-chroot>[root@apps-fp-oi-00000340 a]# rpm -qa |grep pyst
pystache-0.5.3-2.fc20.noarch

Well, we can't  freeze us with deprecated packages in Fedora just because of maintain just one application, it's a problem of the developer  or is a problem of us? given that we deploy in infra with packages is mandatory maintain the package in epel but in fedora version this is a hell, I think that without the help of upstream,  we can not do much

Its upstream's issue. I ll see if I can patch it up and then push it upstream.

just remove parameters of 

import os
import pystache

SOURCE_DIR = os.path.dirname(os.path.dirname(__file__))

class SettingsTemplate(pystache.View):
    '''Class for settings'''

    template_path = os.path.join(SOURCE_DIR, 'setup_templates')
    template_name = "settings.py"

    def __init__(self, context, **kwargs):
        super(SettingsTemplate, self).__init__(context=context, **kwargs)

should be 

super(SettingsTemplate, self).__init__()

Doing a grep -ri in the base directory of askbot I've saw many super() passing arguments, (DeprecationWarnings since 2.6), someone has tested askbot and successfully run (me not, yet) in fedora versions?

So just looking at the celery deps, I would say askbot has been broken for fedora for a while since I dont see any history of a matching python-celery and django-celery.

I would suggest releasing 0.7.49 package for el6 and then work for fedora package as the latter looks like a bit more work which is not a blocker for el6.

thoughts?

I'm agree with you Anshu, (missing things are, change python-dateutil15 by python-dateutil) and then build. So we build?

I did a scratch build last night and nirik put it on ask-stg.
http://koji.fedoraproject.org/koji/taskinfo?taskID=6166276

There are some issues but I expect most of them to be related to the application setting rather than the build itself.

19:58 < anshprat> the bug thing being - ask.stg.fpo is redirecting to https://localhost:6081/questions/  https://ask.stg.fedoraproject.org/questions/ 
                  is working fine directly..

20:11 < anshprat> nirik: also, I don't see FAS login at https://ask.stg.fedoraproject.org/account/signin/?next=/

20:26 < anshprat> nirik: enabled debug..  csrf is failing coz of the redirect failure..     Referer checking failed - 
                  https://ask.stg.fedoraproject.org/account/signin/?next=/ does not match https://localhost:6081/.

We can go in for a build once we have verified that the above dont need a fix in the build but rather in the application instance.

@echevmaster  - all the above issues fixed - all were instance specific and nothing to do with package. The scratch build above is good to build the release package from.

PS: Am out for the next 10 days, so expect delayed response.

ok, I finally have this back on my radar. 

Could you post the final spec changes here so I can update the official packages?

mm, there were no changes after the scratch build in #comment 32

Scratch builds are only kept around for 2 weeks. If you look you can see all the packages are garbage collected there. ;) 

So, I need the spec changes to make a new official build...

askbot-0.7.49-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/askbot-0.7.49-1.el6

Package askbot-0.7.49-1.el6:
* should fix your issue,
* was pushed to the Fedora EPEL 6 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=epel-testing askbot-0.7.49-1.el6'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2014-0096/askbot-0.7.49-1.el6
then log in and leave karma (feedback).

On Celery:

"THIS LIBRARY IS NO LONGER REQUIRED

Since Celery 3.1 you can now use Celery with Django directly, so new users should not be using this library for integration.

Django-celery may still be used for the database result backend, and the Django periodic task admin, but these are optional features.

See the new Django tutorial at:

https://docs.celeryproject.org/en/latest/django/first-steps-with-django.html"

From https://pypi.python.org/pypi/django-celery .

So, if you don't load "djcelery" and django-celery installed, any attempt to ask a question with traceback: 

Exception Type:	ImportError
Exception Value:	
No module named celery.decorators
Exception Location:	/usr/lib/python2.6/site-packages/askbot/tasks.py in <module>, line 30

So, it's still needing/wanting/pulling in celery. 

loading that gets it past that. 

So, is there any way to actually disable celery? Or should we just not care and enable it?

I think we should enable it for now and can talk later to upstream and figure out if it is really needed

Yeah, I think so too. Ok, that just leaves getting the email sending working right, a final wipe and rebuild to confirm it's all good and then we can update production.

askbot-0.7.49-1.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.