Bug 1642145

Summary:	devel environment unit tests fail due to jinja2 version
Product:	[Retired] Beaker	Reporter:	Bill Peck <bpeck>
Component:	scheduler	Assignee:	Martin Styk <mastyk>
Status:	CLOSED NEXTRELEASE	QA Contact:	Christopher Beer <cbeer>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	develop	CC:	mastyk
Target Milestone:	27.0
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-11-14 10:35:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Bill Peck 2018-10-23 18:38:09 UTC

Description of problem:
The jinja2 packaging is not correct when following the development guide.

Version-Release number of selected component (if applicable):
26.0

How reproducible:
Everytime

Steps to Reproduce:
1. Setup devel environment by following https://beaker-project.org/dev/guide/
2. run just the kickstart tests
3. ./run-tests.sh -sv bkr.inttest.server.test_kickstart

Actual results:
Errors
see additional info below

Expected results:
should pass

Additional info:
======================================================================
ERROR: test_systemd (bkr.inttest.server.test_kickstart.KickstartTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/beaker/IntegrationTests/src/bkr/inttest/server/test_kickstart.py", line 3407, in test_systemd
    ''')
  File "/root/beaker/IntegrationTests/src/bkr/inttest/server/test_kickstart.py", line 462, in provision_recipe
    virt=virt, lab_controller=self.lab_controller)
  File "/root/beaker/Server/bkr/server/tests/data_setup.py", line 685, in mark_job_complete
    mark_recipe_running(recipe, **kwargs)
  File "/root/beaker/Server/bkr/server/tests/data_setup.py", line 808, in mark_recipe_running
    mark_recipe_installing(recipe, **kwargs)
  File "/root/beaker/Server/bkr/server/tests/data_setup.py", line 785, in mark_recipe_installing
    mark_recipe_waiting(recipe, **kwargs)
  File "/root/beaker/Server/bkr/server/tests/data_setup.py", line 750, in mark_recipe_waiting
    recipe.provision()
  File "/root/beaker/Server/bkr/server/model/scheduler.py", line 2645, in provision
    recipe=self, ks_appends=ks_appends)
  File "/root/beaker/Server/bkr/server/kickstart.py", line 305, in generate_kickstart
    result = template.render(context)
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 669, in render
    return self.environment.handle_exception(exc_info, True)
  File "/root/beaker/Server/bkr/server/kickstarts/default", line 83, in top-level template code
    {% snippet 'rhts_packages' %}
  File "/usr/lib64/python2.6/site-packages/jinja2/sandbox.py", line 251, in call
    return __context.call(__obj, *args, **kwargs)
  File "/root/beaker/Server/bkr/server/kickstart.py", line 289, in snippet
    retval = template.render(context)
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 669, in render
    return self.environment.handle_exception(exc_info, True)
  File "/root/beaker/Server/bkr/server/snippets/rhts_packages", line 18, in top-level template code
    {% for recipe_package in recipe.custom_packages|sort(attribute='package') %}
TypeError: do_sort() got an unexpected keyword argument 'attribute'
-------------------- >> begin captured logging << --------------------
bkr.server.tests.data_setup: DEBUG: Created user user248
bkr.server.tests.data_setup: DEBUG: Created distro Distro(name=u'CustomRHEL7.9-249')
bkr.server.tests.data_setup: DEBUG: Created distro tree DistroTree(distro=Distro(name=u'CustomRHEL7.9-249'), variant=u'Server', arch=Arch(u'x86_64'))
bkr.server.tests.data_setup: DEBUG: Created user user250
bkr.server.model.inventory: DEBUG: Idle system system251.KickstartTest.test.systemd changed lab controller, flagging it for scheduling
bkr.server.model.inventory: DEBUG: system251.KickstartTest.test.systemd status changed from symbol('NEVER_SET') to SystemStatus.automated
bkr.server.tests.data_setup: DEBUG: Created system system251.KickstartTest.test.systemd
bkr.server.model.scheduler: DEBUG: Reserving system system251.KickstartTest.test.systemd for recipe 109
bkr.server.model.activity: DEBUG: Tentative SystemActivity: object_id=46L, service=u'Scheduler', field=u'User', action=u'Reserved', old=u'', new=u'user248', user=user248
bkr.server.model.inventory: DEBUG: Created reservation for system system251.KickstartTest.test.systemd with type u'recipe', service u'Scheduler', user user248
bkr.server.tests.data_setup: DEBUG: Marked R:109 as scheduled with system system251.KickstartTest.test.systemd
--------------------- >> end captured logging << ---------------------

Comment 1 Dan Callaghan 2018-11-09 00:11:53 UTC

I think this issue arises when the tests end up importing Jinja 2.2 from the python-jinja2-2.2.1-2.el6 package, which is too old.

There is also python-jinja2-26-2.6-2.el6eng in the beaker-server repo and the code is expecting to import this instead but we might be lacking the necessary __requires__ trickery for the parallel-installable egg package.

However we probably never noticed the issue in our production or development environments because there we actually have python-jinja2-2.7.2-1.el6 from the CentOS OpenStack Juno repo which overrides the version shipped in RHEL6.

If you run into this issue in your development environment, the workaround is to enable OpenStack Juno repo as described here:

https://beaker-project.org/docs/admin-guide/openstack.html#package-prerequisites

and then yum upgrade python-jinja2. This will match what we have in production and dev right now.

The proper solution would be to add the necessary __requires__ stuff (similar to what we have for CherryPy and TurboGears already) so that this works without the OpenStack repo enabled which is supposed to be optional for Beaker.

Comment 3 Martin Styk 2018-11-13 12:41:17 UTC

https://gerrit.beaker-project.org/#/c/beaker/+/6331/

Comment 4 Martin Styk 2018-11-14 10:35:58 UTC

__requires__ was used to force using of Jinja2 >= 2.6.