1

I have multiple csv files (each file generated per day) with generic filename (say file_) and I append date-stamps to them.

For example: file_2015_10_19, file_2015_10_18and so on.

Now, I only want to read the 5 latest files and create a comparison plot.

For me plotting is no issue but sorting all the files and reading only the latest 5 is.

Remi Guan
  • 21,506
  • 17
  • 64
  • 87
Mac Pan
  • 11
  • 1
  • Once you have a list of file names, why not just do a regular old sort on the list? e.g., `last_five = sorted(list_of_files)[-5:]`. This is assuming that each file prefix this is the same. – clwainwright Oct 20 '15 at 05:13
  • Thanks clwainwright! I appreciate your help. – Mac Pan Oct 20 '15 at 14:44

2 Answers2

1

You need to read all the files, and then sort them. There isn't a shortcut I'm afraid.

You can sort them by the last modified time, or parse the date component and sort by the date

import glob
import os
import datetime

file_mask = 'file_*'
ts = 'file_%Y_%m_%d'
path_to_files = r'/foo/bar/zoo/'

def get_date_from_file(s):
    return datetime.datetime.strptime(s, ts)

all_files = glob.glob(os.path.join(path_to_files, file_mask))
sorted_files = sorted(all_files, key=lambda x: os.path.getmtime(x))[-5:]
sorted_by_date = sorted(all_files, key=get_date_from_file)[-5:]
Burhan Khalid
  • 169,990
  • 18
  • 245
  • 284
0
import os

# list all files in the directory - returns a list of files
files = os.listdir('.')

# sort the list in reverse order
files.sort(reverse=True)

# the top 5 items in the list are the files you need
sorted_files = files[:-5]

Hope this helps!

Adi Krishnan
  • 155
  • 8
  • @mac-pan mentions that he appends the date stamps to the file, so the above should work. Besides, I tried the same code on my local (more than once) before posting and noticed `files[:-5]` works. – Adi Krishnan Oct 20 '15 at 07:24
  • Thanks Adi! I'll try this method as well. Best! – Mac Pan Oct 20 '15 at 14:45