read many files and create animation-python

Question

I wrote script which create animation (movie) from fits files. One file has size 2.8 MB and the no. of files is 9000. Here is code

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import os
import pyfits
import glob
import re



Writer = animation.writers['ffmpeg']
writer = Writer(fps=15, metadata=dict(artist='Me'), bitrate=1800)

global numbers
numbers=re.compile(r'(\d+)')

def numericalSort(value):
    parts = numbers.split(value)
    parts[1::2] = map(int, parts[1::2])
    return parts


image_list=glob.glob('/kalib/*.fits')

image_list= sorted(image_list,key=numericalSort)
print image_list
fig = plt.figure("movie")
img = []

for i in range(0,len(image_list)):

    hdulist = pyfits.open(image_list[i])
    im = hdulist[0].data
    img.append([plt.imshow(im,cmap=plt.cm.Greys_r)])


ani = animation.ArtistAnimation(fig,img, interval=20, blit=True,repeat_delay=0)
ani.save('movie.mp4', writer=writer)

I think that my problem is when I create array img[]...I have 8 GB RAM and when the RAM is full my operating system terminate python script.

My question is: How I can read 9000 files and create animation? Is possible create some buffer or some parallel processing?

Any suggestion?

score 2 · Answer 1 · answered Nov 03 '15 at 09:26

I would recommend you to use ffmpeg. With the command image2pipe you don't have to load all the images into your RAM but rather one by one (i think) into a pipe.

In addition to that, ffmpeg allows you to manipulate the video (framerate, codec, format, etc...).

https://ffmpeg.org/ffmpeg.html

score 1 · Answer 2 · edited May 23 '17 at 11:59

1

You might be better off creating your animation with FuncAnimation instead of ArtistAnimation, as explained in ArtistAnimation vs FuncAnimation matplotlib animation matplotlib.animation FuncAnimation is more efficient in its memory usage. You might want to experiment with FuncAnimation's save_count parameter as well, check out the API documentation for examples.

edited May 23 '17 at 11:59

Community

1
1

answered Nov 03 '15 at 07:55

mori

72
10

Thanks for answer...but I think that this is not my problem ...I have problem with reading data array from files to img[] array...Animation is created when I have readed array img[] – Franta Konopnik Nov 03 '15 at 08:00
Right but the problem is you're not taking the right approach in the first place. You shouldn't be making a *list* of rendered images with imshow, because each one of those is taking up memory. FuncAnimation allows you to render one frame at a time on an as-needed basis without rendering all images at once. Perhaps you could be clearer about what you are hoping to accomplish, but making a large list of image artists isn't the way. – Iguananaut Nov 05 '15 at 20:05
As a followup, the reason what you're doing works for say, a line plot or something like that, is that lines and polygons and the like are trivial to represent in memory and animate. Whole images are a different story, and when you call imshow you're using memory for each pixel of each frame. A naïve approach won't work. – Iguananaut Nov 05 '15 at 20:07

read many files and create animation-python

2 Answers2