cant figure out how to read from utf-16 file and syslog it out to utf-8

Question

I am trying to take a file that is in UTF-16 format and then syslog it out to a syslog server in UTF-8 format. i am pretty new to python and programming. Everything is working except that it's not sending it out as UTF-8 in the syslog.

Code

import logging
import logging.handlers
import tailer
import os
import codecs

logger = logging.getLogger('myLogger')
logger.setLevel(logging.INFO)

#add handler to the logger
handler = logging.handlers.SysLogHandler(address=('x.x.x.x', 514))

#add formatter to the handler
formatter = logging.Formatter('')

handler.formatter = formatter
logger.addHandler(handler)

while True:
    for line in tailer.follow(open('z:\\ERRORLOG')):
        logger.info(str.decode('utf-8'), line)

i also tried the below at the end

while True:
for line in tailer.follow(open('z:\\ERRORLOG')):
    logger.info(str(line, 'utf-8'))

sorry Scott. just figured out how to add it properly afterwards — Rlourenco, Jun 02 '18 at 19:55
Probably you should use encoding parameter in open function instead of string's decode method https://docs.python.org/3/library/functions.html#open — NobbyNobbs, Jun 02 '18 at 21:14

NobbyNobbs · Accepted Answer · 2018-06-03T12:59:17.003

0

just use additional parameter encoding when you open file. simple example:

with open('utf-16.txt', 'r', encoding='utf-16') as fr:
    with open('utf-8.txt', 'w', encoding='utf-8') as fw:
        for line in fr:
            fw.write(line)

in your case:

while True:
    with open('z:\\ERRORLOG', encoding='utf-16') as fr:
        for line in tailer.follow(fr):
            logger.info(line)

edited Jun 03 '18 at 12:59

answered Jun 03 '18 at 08:59

NobbyNobbs

1,341
1
11
17

Hi nobby. thanks for the reply. But i dont want to write a second file. i want to syslog it directly off in a different encoding. if i create a new file with the new encoding then it could lead to other problems. is there a way i can use the encoding option in the logger.info section? – Rlourenco Jun 03 '18 at 12:14
@Rlourenco, I've updated the answer. The main point is python would decode file by itself if you pass proper encoding name in function `open`. – NobbyNobbs Jun 03 '18 at 12:54
thanks Nobby. your answer did put me on the right path. i however didnt get it to output in the right format but i found another solution to my problem so thank you. – Rlourenco Jun 08 '18 at 20:02

cant figure out how to read from utf-16 file and syslog it out to utf-8

1 Answers1