Cyrillic chars in Python 2.7

Question

In my script I pointed 1251 codepage. But Python 2.7.13 output incorrectly shows some cyrillic strings:

Программа 'Game Over' 2.0
('\xd2\xee \xe6\xe5', '\xf1\xe0\xec\xee\xe5', '\xf1\xee\xee\xe1\xf9\xe5\xed\xe8\xe5')
('\xd2\xee\xeb\xfc\xea\xee', '\xf7\xf3\xf2\xfc-\xf7\xf3\xf2\xfc', '\xef\xee\xe1\xee\xeb\xfc\xf8\xe5')
оно...

       GAME OVER




Нажмите Enter для выхода...

I read this and this topics before but it didn't help me. I tried such variants:

# -*- coding: utf-8 -*-
# -*- coding: cp1251 -*-

Why does it happen and how can I fix it?

At the same time Python 3.6.0 output writes all cyrillic chars correctly even without the codepage pointing:

Программа 'Game Over' 2.0
То же самое сообщение
Только чуть-чуть побольше
оно...

       GAME OVER




Нажмите Enter для выхода...

My code:

# coding: cp1251
# game_over.py
# © Andrey Bushman, 2017

print("Программа 'Game Over' " + "2.0")
print("То же", "самое", "сообщение")
print("Только", "чуть-чуть", "побольше")
#print("Вот", end=" ")
print("оно...")

print("""
       GAME OVER
      """)
print("\a")
input("\n\nНажмите Enter для выхода...")

Answer 1

print("То же", "самое", "сообщение")

Nothing to do with Cyrillic -- Python 2's print statement doesn't have parentheses.

So here you're printing the tuple ("То же", "самое", "сообщение"), not a string. This does the same thing:

tmp = ("То же", "самое", "сообщение")
print tmp

Either remove the parentheses, or add from __future__ import print_function at the top of your module.

Answer 2

For 2.7, you should make the strings unicode strings by using the u prefix. The following works both in IDLE and the console (when the console codepage is set to 1251 with chcp 1251).

# coding: utf_8
# game_over.py
# Andrey Bushman, 2017
from __future__ import print_function

print(u"Программа 'Game Over' 2.0"
      )
print (u"То же самое сообщение")
print(u"Только чуть-чуть побольше")
print(u"оно...")

print("""
       GAME OVER
      """)
print(u"\n\nНажмите Enter для выхода...", end='')
a = raw_input()

I separated the prompt and input because input(u'xxxx') was not working. raw_input is needed in 2.x to avoid evaluating the input.

Answer 3

I spend quite some time figuring out how to use Python 2.7 properly with non-latin1 code pages. The easiest solution I found, by far, is to switch to Python 3. Nothing else comes remotely close to it.

Answer 4

The print statement in python2 evaluates each comma-separated expression within the brackets and converts them to a string before it's printed. That's why each cyrillic character is converted to ASCII when you separate the values with commas.

What you can do is the following:

import codecs

text = ("То же", "самое", "сообщение")
for i in text:
    (codecs.decode(i, 'utf-8'))

Or:

text = ("То же", "самое", "сообщение")
print(' '.join(text))

Make sure you have the following line at the top of your python script if you're using python2.

# -*- coding: utf-8 -*-

Answer 5

Short answer: If you want to print chars other than ascii or those in your default codepage on Windows, use 3.6+. Explanation below.

To properly read a file, the encoding declaration must match the actual encoding of the bytes in the file. If you use a limited (non-utf) encoding and want to print strings to Command Prompt, the limited encoding and the console encoding should also match. Or rather, the subset of unicode that you try to print must be included the the subset that the console will accept.

In this case, if you declare the encoding as cp1251 and save it with IDLE, then IDLE appears to save it with that encoding. By definition, the only chars in the file must be in the cp1251 subset. When you print those characters, the console must accept at least the same subset. You can make Command Prompt accept Russian by running chcp 1251 as a command. (chcp == CHange CodePage.) Warning: this command only affects the current Command Prompt window. Anyway, by matching the encoding declaration and the console codepage, I got your code to run on 2.7, 3.5, and 3.6 in the console (but not in IDLE 2.7). But of course, non-ascii, non-cyrillic chars generated by your code will not print.

In 3.x, Python expects code to be utf_8 by default. For 3.6, Python's interface to Windows' consoles was re-written to put the console in utf_8 mode. So write code in an editor that saves it as utf_8 and, as you noticed, printing to the console in Windows works in 3.6. (In 3.x, printing to IDLE's shell has always worked for all the Basic Multilingual Plane (BMP subset) of unicode. Not working for higher codepoints is a current limitation of tk and hence tkinter, which IDLE uses.)