0

I am trying to read with python a fairly large .qvd file into a DataFrame and then rewrite it in another file format, however when using this qvd.qvd_reader PyPI package and invoking:

df = qvd_reader.read(fpath)

I am getting "memory allocation of xxxxxxxxxxxx bytes failed", because obviously it cannot read/fetch all the data into the memory at once. My only option is to access N-rows of the .qvd (with a for loop) and write small files into chunks from that .qvd.

To do this I found this qvdfile repo and this thread.

The problem that I am facing is when I try to read any row with:

import qvdfile
qvd = qvdfile.QvdFile ("test.qvd")
print("First row of the file is {}".format(qvd.getRow(0))) # <- this lineis the issue

print("File has column '{}' with first value of '{}'".format(
        qvd.fields[0]["FieldName"], qvd.getFieldVal(qvd.fields[0]["FieldName"], 0)))
print("File has {} records".format(qvd.attribs["NoOfRecords"]))

I am getting an error like:

UNHANDLED YET TYPE: 111

File has column 'StringColumn' with first value of 'None'

File has 3 records

Or UNHANDLED YET TYPE For values, ranging from 0 up to ? (255 I guess) - when trying with other .qvd files.

or error like:

bitstring.ReadError: Reading off the end of the data. Tried to read 32 bits when only 0 available.

Obviously the problem is with the incomplete code in qvdfile.py at Line 130 (within the getFieldVal function)

My questions are -

• Can someone modify that sub, so it handles the rest of the byte types?

• Has anyone succeeded in reading such big .qvd files (without the QV api) and how?

• Is there any other library that can read a .qvd row by row that I am not aware of.

FYI -

• I tried to solve my problem with ChatGPT, without success.

• No access to Qlikview API on my machine (so please dont suggest win32com.Dispatch('QlikTech.QlikView')

• The .qvd files that I'm trying to read look like:

# File3.qvd - File has 67404124 records - memory allocation of x bytes failed

# File2.qvd - File has 48452228 records - memory allocation of x bytes failed

# File1.qvd - File has 25701860 records - was able to read with qvd.qvd_reader

1337Grrr
  • 23
  • 4

0 Answers0