0

I am working on a recording application and I am able to get 10 - 15 screenshots per second. The issue is saving all of the images fast enough so that the queue of screenshots doesn't build up in RAM and drain the user's memory.

Currently I am able to do this by using two threads that save the images in order and then collect the garbage.

while (true)
{
    bmp = null;

    Dispatcher.Invoke(new Action(delegate()
    {
        if (images.Count > 0)
        {
            bmp = images[0];
            images.RemoveAt(0);
        }
    }));

    if (bmp != null)
    {
        bmp.Save("frames\\" + framesImagesIndex++ + ".png", ImageFormat.Png);
        bmp.Dispose();
        bmp = null;
        GC.Collect();
    }

    Thread.Sleep(10);
}

The problem with this, is that it is very CPU intensive so I do not think it will work on less capable systems. This method takes 30 CPU on my 3.31 GHz process.

I can alleviate the CPU to around 17 when I set it to write all the images to one file and write just the bytes by locking the bits of the bitmap.

byte[] rgbValues;

IntPtr ptr;

System.Drawing.Rectangle rect;

System.Drawing.Imaging.BitmapData bmpData;

System.Drawing.Imaging.PixelFormat pxf;


pxf = System.Drawing.Imaging.PixelFormat.Format24bppRgb;

//using(FileStream fs = new FileStream("frames.data", FileMode.OpenOrCreate, FileAccess.Write))
while (true)
{
    bmp = null;

    Dispatcher.Invoke(new Action(delegate()
    {
        if (images.Count > 0)
        {
            bmp = images[0];
            images.RemoveAt(0);
        }
    }));


    rect = new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height);
    bmpData = bmp.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadOnly, pxf);

    ptr = bmpData.Scan0;

    rgbValues = new byte[bmpData.Stride * bmp.Height];

    Marshal.Copy(ptr, rgbValues, 0, rgbValues.Length);

    fs.Write(rgbValues, 0, rgbValues.Length);

    bmp.UnlockBits(bmpData);

    bmp.Dispose();
    bmp = null;
    GC.Collect();
}

This method helps the CPU issue, but because its not fast enough. The queue of images builds up and overloads the ram so I don't know what else I can do to solve both issues.

Is there any other method available?

John
  • 5,942
  • 3
  • 42
  • 79
  • Keep in mind that you're forcing a Garbage collection at every single iteration of this loop.. and its a full collection of all generations. It may be better for you to just let the GC take care of cleaning up memory as it gets bigger (and you have nothing rooting these bitmaps). Remember, a GC suspends all threads whilst it goes to work. So everything effectively stops on that line while all generations are collected and the large object heap becomes fragmented. – Simon Whitehead Oct 01 '13 at 23:12
  • I tested it and GC takes 0 to 15 milliseconds and does not decrease the CPU usage when I remove it. Most of the time is taken up in file writing. – John Oct 01 '13 at 23:18
  • Even still.. it's a delay you don't really need. The CPU would no doubt be firing off because of `while (true)`.. have you thought about delegating the file-writing work off to some Threads? Or is this running in it's own Thread? – Simon Whitehead Oct 01 '13 at 23:20
  • 1
    Instead of focusing on the saving and cleaning up, what about implementing a way to throttle the collection process? – Scott Solmer Oct 01 '13 at 23:22
  • @Okuma.Scott, I am not asking for help to add a sleep call to my recording thread. I am asking for help to save 15 FPS to file more effectively. – John Oct 01 '13 at 23:28
  • @SimonWhitehead the recording is in a separate thread and this image saving thread is using 2 threads to be able to save the queue in time. The first method saves the queue in time, but it takes a lot of CPU to do it. I need make the file saving less CPU intensive. If I slow down the file saving at all, then the images array fills to the brink and crashes the application from lack of memory. – John Oct 01 '13 at 23:30
  • @John Have you compared the CPU usage with a commercial screen recording application? In my experience they usually do smash CPU as well.. – Simon Whitehead Oct 01 '13 at 23:35
  • @John Okay, what about saving in a different format than .png? If you save as raw data and convert later you may be able to save faster by not compressing on the fly. – Scott Solmer Oct 01 '13 at 23:35
  • You need to only add to the images array as fast as you can flush it. It looks like you're trying to fudge queueing and writing so that they happen to line up - but what happens if the user's computer is different (lower RPM hard drive or starts a proccess that IO starves your process) - you're going to crash if you don't add some sort of smart monitoring. – McAden Oct 01 '13 at 23:36
  • @Okuma.Scott jpeg is a bit faster, but the quality is pretty bad. Right now I am testing different qualities of jpeg to see if I can get a passable quality. – John Oct 01 '13 at 23:37
  • @McAden I did used to have it go only as fast as it could save, but I was not able to reach 15 FPS. Only 8-10 FPS. So I am looking to optimize the file saving so it can possibly revert back to that safe method and still achieve 15 FPS. – John Oct 01 '13 at 23:39
  • @SimonWhitehead camtasia takes just 5 CPU for 15 FPS – John Oct 01 '13 at 23:40
  • Have you profiled in Release as opposed to Debug? have you looked at some of the performance questions already existing around bitmaps like [this one](http://stackoverflow.com/questions/7535812/c-sharp-lockbits-perfomance-int-to-byte) – McAden Oct 01 '13 at 23:45
  • @McAden In release it gets around 12 CPU, but it is still a bit much. I don't want anyone's video playback getting lagged down by the CPU usage. – John Oct 01 '13 at 23:48
  • Also after more benchmarking. It looks like the time isn't used up in file writing, but rather when the bitmap saves to the stream. Not when it is going into the file. – John Oct 01 '13 at 23:51
  • While you are using CPU and ram you'll got problems. Try using GPU instead. as a sample GDI, GDI+ or opencl for c#. – redParrot Jul 14 '22 at 15:33
  • Can you use a video compression library like libx264? It's highly optimized and should handle 30 fps at 1920x1080 with a few CPU cores with `medium` preset. Or higher resolutions with low compression settings, like `superfast`. (A high quality setting like lossless may slow it down, too.) PNG shouldn't be that expensive to compress, if you tell your library not to spend a really long time on optimizing the lossless compression. – Peter Cordes Jul 15 '22 at 18:18

2 Answers2

0

First off, using a while loop with a Thread.Sleep(x); is very CPU intensive, as is GC.Collect() and calling up a new instance too often. You should rather look at using threads.

Action action;
public void Main()
{
action = new Action(delegate()
{
    if (images.Count > 0)
    {
        bmp = images[0];
        images.RemoveAt(0);
    }
});
Thread thread = new Thread(SaveBitmap());
thread.Start();
thread.Priority =  ThreadPriority.Highest
}
public void SaveBitmap()
{
    Dispatcher.Invoke(action);

if (bmp != null)
{
    bmp.Save("frames\\" + framesImagesIndex++ + ".png", ImageFormat.Png);
    bmp.Dispose();
    bmp = null;
}
}

Try using the above, or at least integrate some parts.

Kristof U.
  • 1,263
  • 10
  • 17
ismellike
  • 629
  • 1
  • 5
  • 14
0

Saving as PNG format is CPU intensive because of the compression involved. Save the images as .bmp and you will get a large speed increase, at the expense of using more disk space.

Andy
  • 7
  • 1
  • 2
    I'd start by checking if the PNG library has settings for the speed vs. compression ratio tradeoff, since the main compression in PNG is DEFLATE (like zip/gzip). I wonder what resolution this question was dealing with? x264 can encode 1920x1080p at 30fps with a few CPU cores on a machine that was new in 2013 (like Haswell), at least with a preset like `superfast` (higher bitrate for the same quality than if you let it spend more CPU time). e.g. 73 fps on my Skylake i7-6700k (from 2016) at `-preset faster -crf 23` on a live-action video; mostly-static screengrabs probably faster. – Peter Cordes Jul 15 '22 at 18:24