jinja2 encoding error in python

Question

So I am trying to use jinja2 for a simple html template but I keep getting this error when I call render():

Warning: IronPythonEvaluator.EvaluateIronPythonScript operation failed. 
Traceback (most recent call last):
File "C:\Program Files (x86)\IronPython 2.7\Lib\jinja2\jinja2\loaders.py", line 125, in load
File "C:\Program Files (x86)\IronPython 2.7\Lib\jinja2\jinja2\environment.py", line 551, in compile
File "C:\Program Files (x86)\IronPython 2.7\Lib\jinja2\jinja2\environment.py", line 470, in _parse
File "C:\Program Files (x86)\IronPython 2.7\Lib\jinja2\jinja2\parser.py", line 31, in __init__
File "C:\Program Files (x86)\IronPython 2.7\Lib\jinja2\jinja2\environment.py", line 501, in _tokenize
File "C:\Program Files (x86)\IronPython 2.7\Lib\jinja2\jinja2\environment.py", line 494, in preprocess
File "<string>", line 21, in <module>
File "C:\Program Files (x86)\IronPython 2.7\Lib\jinja2\jinja2\environment.py", line 812, in get_template
File "C:\Program Files (x86)\IronPython 2.7\Lib\jinja2\jinja2\environment.py", line 786, in _load_template
UnicodeEncodeError: ('unknown', '\x00', 0, 1, '')

Now I understand that jinja2 only works with utf-8 so I am forcing my python code to that formatting:

#-*- coding: utf-8 -*-

import clr
import sys

pyt_path = r'C:\Program Files (x86)\IronPython 2.7\Lib'
sys.path.append(pyt_path)

pyt_path1 = r'C:\Program Files (x86)\IronPython 2.7\Lib\MarkupSafe'
sys.path.append(pyt_path1)

pyt_path2 = r'C:\Program Files (x86)\IronPython 2.7\Lib\jinja2'
sys.path.append(pyt_path2)

#The inputs to this node will be stored as a list in the IN variable.
dataEnteringNode = IN

from jinja2 import Environment, FileSystemLoader

j2_env = Environment(loader=FileSystemLoader(r'C:\Users\ksobon\Documents\Visual Studio 2015\Projects\WebSite2'), trim_blocks=True)
temp = j2_env.get_template('HTMLPage2.html')

OUT = temp.render(svgWidth = r'500')

Here's the template html file that I am trying to use:

<!DOCTYPE html>
<meta charset="utf-8">
<style>
    .chart rect {
      fill: steelblue;
      stroke: white;
    }
</style>
<svg class="chart"></svg>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script>

var data = [
  { name: "Locke", value: 42 },
  { name: "Reyes", value: 8 },
  { name: "Ford", value: 15 },
  { name: "Jarrah", value: 16 },
  { name: "Shephard", value: 23 },
  { name: "Kwon", value: 42 }
];

var w = {{svgWidth}},
    h = 400;

var x = d3.scale.linear()
    .domain([0, 1])
    .range([0, w]);

var y = d3.scale.linear()
    .domain([0, 100])
     .rangeRound([0, h]);

var chart = d3.select("body")
.append("svg:svg")
.attr("class", "chart")
.attr("width", w * data.length - 1)
.attr("height", h);

chart.selectAll("rect")
.data(data)
.enter().append("svg:rect")
.attr("x", function (d, i) { return x(i) - .5; })
.attr("y", function (d) { return h - y(d.value) - .5; })
.attr("width", w)
.attr("height", function (d) { return y(d.value); });

</script>

Any ideas what I am doing wrong? I am in IronPython2.7

score 1 · Answer 1 · edited May 23 '17 at 12:08

Edit:

UnicodeEncodeError: ('unknown', '\x00', 0, 1, '')

Hm... this seems the same error as in this question, some bug in the template loader?

Original answer:

Now I understand that jinja2 only works with utf-8 so I am forcing my python code to that formatting:

# -*- coding: utf-8 -*-

The line above just tells Python that the source code file is encoded using utf-8 so that it know how to interpret string literals.

However, this does not mean that the strings that are sent to Jinja are unicode instances, which is what Jinja expects. From the docs:

Jinja2 is using Unicode internally which means that you have to pass Unicode objects to the render function or bytestrings that only consist of ASCII characters.

What this means is that you need to convert your strings to unicode instances. For string literals you prefix them with u:

text = u'This string literal is a unicode instance'

For the string variables which are just plan sequences of bytes you need to decode them first, and that includes the HTML template you load:

>>> value = 'foö'
>>> type(value)
<type 'str'>
>>> unicode_value = value.decode('utf8')
>>> type(unicode_value)
<type 'unicode'>
>>> print unicode_value
foö

score 1 · Accepted Answer · answered Aug 25 '15 at 03:21

1

This is a known bug in IronPython. You can work around it by editing the Jinja2 source:

Using Jinja2 on IronPython with non-ASCII characters in a template will result in this error. Replace to_string = unicode with to_string = lambda x: unicode(x) near the top of runtime.py to work around this issue.

answered Aug 25 '15 at 03:21

Cameron

96,106
25
196
225

I dont see ```to_string = unicode``` instead i found ```to_string = text_type``` which is imported from ```jinja2._compat``` I tried setting ```to_string = lambda x: unicode(x)``` but that didn't work either. – konrad Aug 25 '15 at 03:32
I was using jinja2 2.8 at the moment and i realized that you were referring to version 2.5. It works on that version however so I guess i will stick with it. Thank you, – konrad Aug 25 '15 at 03:38
Oh, sorry, didn't realise it had changed. Did you try `to_string = lambda x: text_type(x)`? – Cameron Aug 25 '15 at 04:05
Ah, that won't work either, `text_type` is just assigned `unicode` anyway. I think everywhere that uses `text_type` would have to be changed :-( – Cameron Aug 25 '15 at 04:14
Yeah that was actually first thing that I tried but then realized what you realized that text_type was used in a lot of places so it wasn't working. I just went with 2.5 since that worked like a charm...I am up and running again so thanks for help! – konrad Aug 25 '15 at 13:10

score 1 · Answer 3 · answered Jun 03 '16 at 14:24

1

In jinja2 2.8 do not modify runtime.py as it says in accepted answer. Instead, replace in file _compat.py text_type = unicode at line 52 with

text_type = lambda x: unicode(x)

answered Jun 03 '16 at 14:24

AVKurov

156
1
2
6

jinja2 encoding error in python

3 Answers3