15

i am new to python and i was trying to write a simple code for converting a text file to avro. i am getting this error that module not found. I could clearly see in the schema.py file that the parse module exists. I will appreciate if someone could help me understand what i may be doing wrong.

import avro.schema, csv, codecs
from avro.datafile import DataFileReader, DataFileWriter
from avro.io import DatumReader, DatumWriter


def unicode_csv_reader(unicode_csv_data, dialect=csv.excel, **kwargs):
# csv.py doesn't do Unicode; encode temporarily as UTF-8:
csv_reader = csv.reader(utf_8_encoder(unicode_csv_data),
                        dialect=dialect, **kwargs)
for row in csv_reader:
    # decode UTF-8 back to Unicode, cell by cell:
    yield [unicode(cell, 'utf-8') for cell in row]

def utf_8_encoder(unicode_csv_data):
for line in unicode_csv_data:
    yield line.encode('utf-8')

schema = avro.schema.parse(open('C:/test/test.avsc', "rb").read())

I am using Python 3.5.2, avro-python3-1.8.1 on Windows 10.

Kevin K
  • 151
  • 1
  • 1
  • 4

1 Answers1

52

You got the avro code sample from their tutorial, but unfortunately it's not updated for avro-python3.

Instead of:

schema = avro.schema.parse(open('file.avsc', "rb").read())

You need to read the file in text mode, and use the Parse() method:

schema = avro.schema.Parse(open('file.avsc', "r").read())
jpdaigle
  • 1,265
  • 10
  • 12
  • 10
    2.5 years, still nobody updated that documentation error. Awesome. – Jeff Ellen May 30 '19 at 22:28
  • 3
    Awesome! I agree. How active it is. – Zhang Wei Jul 20 '19 at 02:56
  • 7
    And still at this point they haven't added language version constraints to their pypi packages. So if you accidentally install avro instead of avro-python3 you will get the python2 version installed without complaint. – jdi Aug 20 '19 at 21:50