-1

I want to write some Japanese characters to a csv, but I can't get it to display properly. The csv ends up with characters like 学校 and I'm not sure what I'm doing wrong.

import requests
import csv

r = requests.get('http://jisho.org/api/v1/search/words?keyword=%23common')
data = r.json()

with open('common_words.csv', 'a', newline='', encoding='utf-8') as csvfile:
    writer = csv.DictWriter(csvfile, fieldnames=['word','reading'])
    for entry in data["data"]:
        word = entry["japanese"][0]["word"]
        reading = entry["japanese"][0]["reading"]
        writer.writerow({'word':word,'reading':reading,})
siowy
  • 238
  • 1
  • 2
  • 11
  • 1
    Some possibilities here: (1) you're using Python 2 rather than 3, so its `csv` module doesn't handle Unicode properly. (2) the input from the web service isn't in UTF-8, it's in something else like, say, Shift-JIS. (3) you're doing everything right in this code, but then whatever you're using to view the resulting output isn't using UTF-8. – abarnert Mar 12 '18 at 08:06
  • Working for me (in python 3 on linux) – Hirabayashi Taro Mar 12 '18 at 08:07
  • I can rule out (2)—the page is in UTF-8, and it has the proper headers, and `requests` interprets it properly. – abarnert Mar 12 '18 at 08:08
  • And from a quick test, Python 2 requires a couple of obvious changes to run at all, but with those changes, it creates an identical file to Python 3. So, assuming this is your real code, I'm betting it's (3), and you've written perfectly good code that generated a perfectly valid UTF-8 CSV file, but then viewed it in something (Windows command line, maybe?) that doesn't expect UTF-8. – abarnert Mar 12 '18 at 08:11
  • And if it's (3), you just need to use a different tool to display/import/whatever your CSV file, or learn how to use the tool you're using. You didn't tell us what tool that is, but really, whether it's Excel X.Y or TextEdit or some web interface doesn't matter; it's not a Python problem, or even a programming problem, so you probably need to ask somewhere else. – abarnert Mar 12 '18 at 08:13
  • Can be reproduced running your code and viewing in Ubuntu Libre Office. And by changing support font, it can show correctly. So as @abarnert pointed out, this may be a viewing problem. – lincr Mar 12 '18 at 08:19
  • Ah OK I'm using Windows and Excel 2017. I'll try again. Thanks for the help guys! – siowy Mar 12 '18 at 08:39
  • Ah! You guys are right. I opened it with a text editor and was able to view it properly. Something about excel reading csv that messes it up. Solved. – siowy Mar 13 '18 at 08:35

1 Answers1

0

The code is correct. Reading the csv file with excel was the problem. Opening the file with a text editor displayed the characters properly.

siowy
  • 238
  • 1
  • 2
  • 11