Smilies in form are saved as ???? in database

Question

In my Django app and MariaDB database, when users post form containing smilies like the smiley is saved as ???? which is clearly not intended.

The function in the views:

# -*- coding: utf-8 -*-

def send_letter(request):
    args = {}
    if request.method == 'POST':
        form = SendLetterForm(request.POST)

        if form.is_valid():           
            d = form.save(commit = False)
            try:
                author =  get_user(request)
            except:
                author = 'Guest'
            d.title = form.cleaned_data['title'] 
            d.body = form.cleaned_data['body'] 
            d.published = False
            d.save()

The model which is used to generates form is:

class SendLetter(models.Model):
    author = models.ForeignKey(User, blank=True, null=True)
    title = models.CharField(max_length=100, blank=False)
    body = models.TextField(blank=False)
    created = models.DateTimeField(default=datetime.datetime.now)
    published = models.BooleanField(default= False, blank=True)

The form:

class SendLetterForm(forms.ModelForm):    
           class Meta():
        model = SendLetter
        exclude = ('author','created', 'published')

The table collation is set to utf8mb4_bin and the text is in Persian. I have also tried other utf8 collations like utf8_persian_cli without success. I‌ have no issues saving Persian text without smilies.

When I insert text containing smilies directly into the table field, using PhpMyAdmin, the smilies are stored correctly.

I have also tried d.body=form['body'] (without cleaned_data) but got the same ???? after save.

So the question is how can I save smilies correctly without compromising the form security?

[Check if this answers your problem](https://stackoverflow.com/questions/35522102/django-mysql-arabic-encoding) — Bijoy, Jun 29 '18 at 06:06
@Bijoy, I'm already using `utf8` adn `utf8_general_ci` in the db. — Babr, Jun 29 '18 at 06:11
I changed `body` field collaction to `utf8mb4_bin` and also `utf8mb4_general_ci` but still got the same problem. — Babr, Jun 29 '18 at 06:21
Removed [tag:sql-injection] tag, because this question has nothing to do with SQL injection. — Bill Karwin, Jun 29 '18 at 14:57

score 2 · Accepted Answer · answered Jun 29 '18 at 08:05

2

Thanks to this answer adding

'OPTIONS': {'charset': 'utf8mb4'},

tto the settings.py DATABASE options and setting the fields collation to utf8mb4 solved the problem.

answered Jun 29 '18 at 08:05

Babr

1,971
14
33
47

Smilies in form are saved as ???? in database

1 Answers1