2

I'm still writing a program which uses data from Wikipedia pages. However, when I execute the code below, I get this exception:

UnicodeEncodeError: 'charmap' codec can't encode characters in position 30-31: character maps to <undefined>

Here is the code:

import wikipedia
print(wikipedia.summary("Barack Obama", 1))

I looked at the Wikipedia source code and I could only find UTF-8, but that is my standard decoding method anyways. So what I learn from this is that Wikipedia is using some strange symbols that somehow are contained in UTF-8 but somehow are not at the same time. How do I best deal with this if I still want to display the text?

gowner
  • 327
  • 1
  • 4
  • 11

1 Answers1

2

Your code runs perfectly on Mac OS X and Linux with Python3. I suspect you are using Windows with a non-UTF8 terminal. Change your terminal settings with :

chcp 65001

You might need to use Lucida Console font

Gerard Rozsavolgyi
  • 4,834
  • 4
  • 32
  • 39
  • I'm not using the terminal, I'm using Sublime Text 2. Sorry, forgot to mention that. – gowner Jan 04 '16 at 08:42
  • but the results is printing in a console ? – Gerard Rozsavolgyi Jan 04 '16 at 08:44
  • The results are printed in the Sublime Text internal console, yes. – gowner Jan 04 '16 at 08:46
  • so the console settings are not in utf-8. Try with command line in a terminal changed to utf-8 by the above command and you will see – Gerard Rozsavolgyi Jan 04 '16 at 08:51
  • or check if sublime doesn't put the -u option by default. In that case remove the -u option – Gerard Rozsavolgyi Jan 04 '16 at 09:01
  • I don't know if the problem is with Sublime since it uses the UTF-8 encoding for all other characters and all other Wikipedia articles just fine. It is just that one character in that one article that makes it give that exception. The problem doesn't seem to be that it isn't using the UTF-8 encoding, it seems to be that the character in question somehow refers to in the version UTF-8 it uses. – gowner Jan 04 '16 at 09:07
  • but it's working on Linux and Mac OS X for that article so could you check if the -u option is set on sublime or try in a terminal with the suggested settings ? – Gerard Rozsavolgyi Jan 04 '16 at 09:26
  • I did what you said and ran it in a command line and it did work, so the problem is actually with Sublime. I also did remove the "-u" from the build system but it didn't do anything. Now I need to find that mistake. – gowner Jan 04 '16 at 09:41
  • so probably an other setting for sublime text console if available or you can use a sublime plugin like https://packagecontrol.io/packages/Glue or https://github.com/joeyespo/sublimetext-console-exec – Gerard Rozsavolgyi Jan 04 '16 at 09:47