Bug 1470322 - Jinja2 2.9+ breaks included template context [NEEDINFO]
Jinja2 2.9+ breaks included template context
Status: ASSIGNED
Product: Fedora
Classification: Fedora
Component: ansible (Show other bugs)
26
x86_64 Linux
unspecified Severity unspecified
: ---
: ---
Assigned To: Kevin Fenzi
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-12 14:06 EDT by Cyril Lopez
Modified: 2017-10-31 12:17 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
kevin: needinfo? (cylopez)


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Github 20063 None None None 2017-07-12 14:06 EDT

  None (edit)
Description Cyril Lopez 2017-07-12 14:06:58 EDT
Description of problem:

All detail here : https://github.com/ansible/ansible/issues/20063

Version-Release number of selected component (if applicable):
python2-jinja2-2.9.6-1.fc26.noarch
ansible-2.3.1.0-1.fc26.noarch
Comment 1 Cyril Lopez 2017-07-12 14:15:29 EDT
From jinja2 : https://github.com/pallets/jinja/issues/659
Comment 2 Kevin Fenzi 2017-07-12 14:29:08 EDT
Do you actually see the problem with 2.3.1.0 ? 

Both those issues are closed and I thought them fixed. 

Can you show a reproducer? also, might be good to file a new issue upstream, because ansible developers do not get notifications on closed issues.
Comment 3 Cyril Lopez 2017-07-12 15:40:17 EDT
I face the issue after the upgrade from fedora 25 to 26.
Last version working was :
ansible 2.3.1.0-1.fc25
python2-jinja2 2.8.1-1.fc25

This is this issue : https://github.com/debops/ansible-postfix/issues/84

The code :
 
- name: Generate Postfix configuration
  tags: bug
  template:
    src: 'etc/postfix/{{ item }}/ansible.j2'
    dest: '/etc/postfix/{{ item }}'
    owner: 'root'
    group: 'root'
    mode: '0644'
  with_items: [ 'main.cf', 'master.cf' ]
  notify: [ 'Check postfix and restart' ]

templates/etc/postfix/main.cf/ansible.j2   
# This file is managed by Ansible, all changes will be lost


# Current postfix capabilities: {{ ", ".join(postfix) }}

{% set postfix_tpl_maincf_parts = [
  'basic_options',
  'sasl_auth',
  'smtp_client',
  'archive',
  'tls',
  'postscreen_main',
  'smtpd_restrictions',
  'local_maincf' ] %}
{% for part in postfix_tpl_maincf_parts %}

# Start {{ part }}
{% include part + '.j2' %} {# here is a tactical comment
to make sure we have a whitespace following the inclusion
to avoid https://github.com/debops/ansible-postfix/issues/80 #}
{% endfor %}

Full log ansible : https://paste.fedoraproject.org/paste/CRMsYTnAJkGFmTIU2nHuyw
Comment 4 Kevin Fenzi 2017-07-20 17:59:19 EDT
Hey can you attach the full log to this bug (not a paste link, as they expire). 

Also, would you be willing to report this upstream or would you like me to do so?
Comment 5 Cyril Lopez 2017-08-16 05:33:42 EDT
Hello,

A other example easy to reproduce :
with github.com:centos-opstools/opstools-ansible.git and the file roles/sensu/server/templates/sensu-checks.json.j2 containing :

{% set comma = joiner(",\n") -%}
{
    "checks": {
{% for check in sensu_overcloud_checks %}
{{ comma() }}
{% include [
    "sensu-check-" + check.name + ".json.j2",
    "sensu-check-systemd.json.j2"
] %}
{% endfor %}
{% for check in sensu_overcloud_checks_pcs %}
{{ comma() }}
{% include [
    "sensu-check-" + check.name + ".json.j2",
    "sensu-check-pcs.json.j2"
] %}
{% endfor %}
{% for check in sensu_remote_checks %}
{{ comma() }}
{% include [
    "sensu-check-" + check.name + ".json.j2",
    "sensu-check-oschecks.json.j2"
] %}
{% endfor %}
}}


result is an issue :
failed: [showcasefr-monitoring] (item=sensu-checks) => {                                                                                                                                                                                                                       
    "failed": true,                                                                                                                                                                                                                                                            
    "item": "sensu-checks",                                                                                                                                                                                                                                                    
    "msg": "KeyError: 'undefined variable: 0'"                                                                                                                                                                                                                                 
}                                                                                                                                                                                                                                                                              

https://paste.fedoraproject.org/paste/w5SBhzz0DU1KiLsTDdZS1w
Comment 6 Thomas Moschny 2017-08-16 16:47:27 EDT
After skimming over the GH tickets, I think this is not a Jinja2 issue.
Comment 7 Kevin Fenzi 2017-08-16 19:59:15 EDT
An upstream ansible bug would be good here...
Comment 8 Kevin Fenzi 2017-10-30 16:33:59 EDT
Can you file an upstream bug here if you can still duplicate this?
Comment 9 Thomas Moschny 2017-10-31 09:44:03 EDT
According to the comments in https://github.com/ansible/ansible/issues/20494, this patch https://github.com/zmedico/ansible/commit/db6966969 should fix the issue.
Comment 10 Kevin Fenzi 2017-10-31 12:17:35 EDT
(In reply to Thomas Moschny from comment #9)
> According to the comments in
> https://github.com/ansible/ansible/issues/20494, this patch
> https://github.com/zmedico/ansible/commit/db6966969 should fix the issue.

yeah, that fix was in 2.4.0.0. So, please do try and see if you can duplicate it with 2.4.0 or 2.4.1 (just pushed to testing)

Note You need to log in before you can comment on or make changes to this bug.