0

I am trying to insert a document string which I am reading through a csv file in to sqlite database which has characters like (children's), It throws the following error:

DatabaseError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

Before inserting the document I make sure I am converting the string to unistring .

unicode('Children's', 'unicode-escape')

I am using Django ORM to save the data. Is there something I can do to get rid of this?

Joel James
  • 3,870
  • 9
  • 32
  • 47

1 Answers1

1

The parameter to .execute() that causes the error is a Python bytestring instead of a unicode string like you said it was. Solution: create a unicode string instead: unicode(your_data, your_encoding). For most csv files, the encoding will eiter be "latin-1" or "uft-8".

Using "unicode-escape" is almost certainly wrong. Its's a Python-specific encoding meant to "Produce a string that is suitable as Unicode literal in Python source code" according to the docs (https://docs.python.org/2/library/codecs.html).

ghaering
  • 349
  • 2
  • 3