Python - 'ascii' codec can't encode character u'\xe9' in position 5: ordinal not in range(128)

Question

I'm trying to run a quick Django application that pulls data from Google AdWords and exposes the names of accounts that are managed by an agency. When doing so, I get the following error:

UnicodeEncodeError at /account-hierarchy/
'ascii' codec can't encode character u'\xe9' in position 5: ordinal not in range(128)

Here's the snippet:

<table class="pretty-table">
  <thead>
    <tr>
      <td>Customer ID</td>
      <td>Client Name</td>
      <td>Can Manage Clients</td>
      <td>Account Currency</td>
    </tr>
  </thead>
  {% for account in managed_accounts %}
    <tr>
      {% for field in account %}
        <td>{{ field }}</td>
      {% endfor %}
    </tr>
  {% endfor %}
</table>

where the call to {{ field }} is the problematic line.

I have already added

<meta http-equiv="content-type" content="text/html; charset=utf-8">

to the template I am rendering, but it still fails, so I believe the problem is not on the HTML template, but rather on the Python/Django engine.

Any ideas how I can fix it?

Here's the View code that renders the template:

def account_hierarchy(request):
manager_ids = settings.MANAGER_IDS
managed_accounts = []
for manager_id in manager_ids:
    managed_accounts.extend(adwords_utils.getManagedAccounts(manager_id))
return render_to_response('simple-table.html', {"managed_accounts": managed_accounts})

UPDATED Question

Python Version: 2.7.6
Models.py is currently empty

What's also curious is that if I remove this:

{% for field in account %}
    <td>{{ field }}</td>
  {% endfor %}

and I just print out the main array:

{{ managed_accounts }}

it works just fine. Not sure what's going on.

Curious fact #2: As I managed to output the full array, I checked for character 'é' and I didn't find it on the final output. Not sure where it was coming from.

Perhaps define the `__unicode__` method in your `models.py` would work — dazedconfused, Dec 12 '14 at 01:57
Could you also include some related codes in your `models.py`? — dazedconfused, Dec 12 '14 at 02:13
Then you would probably need to encode some of the data in `managed_accounts` — dazedconfused, Dec 12 '14 at 02:18

Jim DeLaHunt · Answer 1 · 2014-12-12T07:21:45.343

The problem is likely that, in some place in your code, you accidentally defined a data structure to be a Python byte string, when you should have made it a Python Unicode string. This leads Django and Python to convert from Django's Unicode string to your byte string, in a default way, using the ASCII codec.

The error message gives some clues:

The error is perhaps occurring when you call account_hierarchy()
The phrase character u'\xe9' in position 5: ordinal not in range(128) means that Python is trying to convert Unicode character 'é' (U+00E9) into an ASCII value from 0..127. Look in your data for an 'é'. Which field is it in?
The phrase 'ascii' codec means the conversion is likely inadvertent. If you intend to use UTF-8, you wouldn't have called the ASCII codec intentionally. But when you cause a conversion but don't specify a codec, Python uses ASCII.
The word encode means conversion from Unicode to byte-oriented encoding. You have read Python's Unicode HOWTO article a couple of times, haven't you?

Function render_to_response() uses settings DEFAULT_CHARSET and DEFAULT_CONTENT_TYPE](https://docs.djangoproject.com/en/dev/ref/settings/#std:setting-DEFAULT_CONTENT_TYPE). They should default to 'utf-8' and 'text/html', which should be appropriate for generating a UTF-8 encoded HTML page, but check.

I like the suggestion that you check your models.py to be sure that your models are defined in terms of Unicode data types, not byte string data types. Update: you say models.py is empty, so that won't be much help.

Character handling, and Unicode vs byte strings, are handled differently in Python 2 and Python 3. Which version of Python are you using? Update: Python 2.7.6, thanks. The Unicode HOWTO I linked to above is for Python 2.7.x.

If you make sure your code handles strings as Unicode throughout, unless you really want a byte string, that will likely fix this problem.

Update: Consider modifying your template to give you debugging information. Try something like these expressions, to see what's really in managed_accounts:

</thead>
<tr><td>managed_accounts is: {{ repr(managed_accounts) }}</tr></td>
{% for account in managed_accounts %}
  <tr>
  {% for field in account %}
    <td>{{ type(field) }}, {{ repr(field) }}</td>
  {% endfor %}
  </tr>
{% endfor %}

[Updated in response to multiple updates from original poster.]

Nice and detailed answer! – dazedconfused Dec 12 '14 at 02:29 — dazedconfused, Dec 12 '14 at 02:29

score 0 · Answer 2 · answered Jan 16 '18 at 02:42

your problem is same with mine,just encode question, you can add bellow code in your views.py of django project:

 1. #coding=utf-8  
 2. import sys
 3. reload(sys)
 4. sys.setdefaultencoding('utf-8')

my problem: Error during template rendering In template D:\PythonProjects\DjangoProject\guest\sign\templates\sign\guest_manage.html, error at line 72 enter image description here

score 0 · Answer 3 · answered Oct 09 '20 at 12:04

0

This is issue belongs to jinja2 templating and will be resolve by setting encoding as utf-8 as default encoding.

sys.setdefaultencoding('utf-8')

answered Oct 09 '20 at 12:04

Sakib Sakharkar

1
2

Python - 'ascii' codec can't encode character u'\xe9' in position 5: ordinal not in range(128)

3 Answers3