0

I need a way to get the list containing specific column data into excel but getting memory error how can I use dask to complete this task, my system is having only 8 GB ram.

I'm creating a excel file out of a huge .dat file(containing text just like table with rows and columns) around(2 GB or more) taking a few columns from the .dat file I know the line number(from the structure file) of columns i need to extract and created a list out this data.

i will extract cell values from a structured file to get the line numbers to use as columns

df=pd.DataFrame()

with open(r"C:/new.dat" ,encoding ="utf-16") as f:
    content = f.readlines()
    f.close()
C = []

c=[]

for k in range(1,sheet.nrows):
    C = []
    C.append(sheet.cell_value(k,0))
    for line in content:
        C.append(line[int(sheet.cell_value(k,1))-1:int(sheet.cell_value(k, 2))]) 
    c.append(C)
    d=iter(c)


#getting memory error from the code below


row = 0
workbook = xlsxwriter.Workbook('table.xlsx')
worksheet = workbook.add_worksheet()
for col, data in enumerate(d):
    worksheet.write_column(row, col, data)
workbook.close()

I want to get the excel file without memory error.

rpanai
  • 12,515
  • 2
  • 42
  • 64
Ravi Teja
  • 1
  • 1
  • loc = (r"C:/Apr/Sample_Structure.xlsx") wb = xlrd.open_workbook(loc) sheet = wb.sheet_by_index(0) – Ravi Teja Jun 28 '19 at 12:14
  • 2
    Please don't put parts of the question into comments but use the [edit](https://stackoverflow.com/posts/56806743/edit) link to add to or correct your question. – NOhs Jun 28 '19 at 12:27
  • Dask recently dropped python 2.7 it's better to use the tag python. – rpanai Jun 28 '19 at 13:01

0 Answers0