Django not matching unicode in url

Question

I have a problem with django 2.0, where a url that contains a unicode slug isn't matched, I searched for a solution but I didn't find one for my case, here's a simplified version of my code:

// models.py

class Level(models.Model):
    name = models.CharField(max_length=100)
    slug = models.SlugField(max_length=100, allow_unicode=True)

in my urls file I have those patterns:

// urls.py

urlpatterns = [
path('', views.index, name='index'),
path('level/<slug:level_slug>', views.level, name='level')]

Now if I go, say to http://127.0.0.1:8000/game/level/deuxième I get this error:

Request Method: GET
Request URL:    http://127.0.0.1:8000/game/level/deuxi%C3%A8me

Using the URLconf defined in mysite.urls, Django tried these URL patterns, in this order:

game/ [name='index']
game/level/<slug:level_slug> [name='level']
admin/
accounts/
The current path, game/level/deuxième, didn't match any of these.

but if I change the item's slug to deuxieme without the unicode character, it works fine, does anyone know the solution to this problem? thanks!

Slugs never contain non-ASCII characters. If you want to match non-ASCII characters then don't use the slug. — Ignacio Vazquez-Abrams, May 14 '18 at 00:15
@IgnacioVazquez-Abrams but I am using django 2.0 which should support non-ASCII characters, it does have the `allow_unicode` parameter. — nabil london, May 14 '18 at 00:19
["slug - Matches any slug string consisting of ASCII letters or numbers, plus the hyphen and underscore characters."](https://docs.djangoproject.com/en/2.0/topics/http/urls/#path-converters) — Ignacio Vazquez-Abrams, May 14 '18 at 00:25
@IgnacioVazquez-Abrams This is so confusing, what's `allow_unicode` used for? — nabil london, May 14 '18 at 00:28
But why isn't it allowed in the path url then? is it useless then? — nabil london, May 14 '18 at 00:32
Because you're using a path converter that explicitly excludes non-ASCII. — Ignacio Vazquez-Abrams, May 14 '18 at 00:33
Ok got it, I just changed `slug` to `str` and it worked, thank you very much for your help bro :-) — nabil london, May 14 '18 at 00:38
I have a similar problem, but using str instead of slug does not help. — keramat, Aug 10 '20 at 08:04

score 16 · Answer 1 · answered Jul 29 '18 at 15:13

16

In urls.py change path from using slug type to str.

From this:

    path('posts/<slug:slug>-<int:pk>/', views.PostDetailView.as_view()),

to this:

    path('posts/<str:slug>-<int:pk>/', views.PostDetailView.as_view()),

Explanation

As suggested in comments, the slug path converter

Matches any slug string consisting of ASCII letters or numbers, plus the hyphen and underscore characters. For example, building-your-1st-django-site.

but we want to keep those non-ascii characters, so we use str:

str - Matches any non-empty string, excluding the path separator, '/'. This is the default if a converter isn’t included in the expression.

answered Jul 29 '18 at 15:13

marcanuy

23,118
9
64
113

Does not work. The current path, api/v1/posts/, didn't match any of these. You can see the error removed non-English address after posts/. – keramat Aug 10 '20 at 07:41
1

I just want to thank you for this comment/solution. I never would've guessed it, since I have my slugfield's set for allowing UTF-8 & therefore "knew" it required me to use in the path. I was going bananas trying to figure out how to regex the path until you said that works for this. Thanks! – Zamphatta Aug 25 '20 at 20:27
@Zamphatta I'm glad it was helpful, I've spent some time figuring it out too, also wrote a post about slugs in Django at https://simpleit.rocks/python/django/generating-slugs-automatically-in-django-easy-solid-approaches/#2-ensure-uniqueness] which I think it may be useful. Cheers! – marcanuy Aug 25 '20 at 21:09

M.javid · Answer 2 · 2018-05-14T05:19:24.223

Use the unidecode library, and set the slug field by results of unidecode.unidecode function, this library support many of languages and detect language automatically and then replace original characters by english characters. For example if you want convert "Hello" word in china language to english characters, try below code:

$ pip install unidecode
$ python -c "import unidecode; print('---->', unidecode.unidecode('你好'))"
----> Ni Hao

Django not matching unicode in url

2 Answers2

Explanation

Linked