1

So my application allows for users to upload video, convert it using FFMPEG and then transfer it over to the Flash Media Server. Lately, I've run into an issue.

If ever there is an error when converting video, I automatically generate cfcatch report PDF. This time I came across a "Cannot allocate memory" error. This massively concerns me because I'm about to promote my system out and I can't afford for the scripts to stop running within the first few hours.

Is there a way to clean up the memory issues with ColdFusion? I mean, once the job has been done, can I essentially "reset" the memory that the server was using?

If you understand the potential disaster, I'm sure you'll understand why I've got to find out how to make sure my scripts are executing properly. The physical fix is to restart the server, but I obviously cannot be restarting the server every single time a user uploads a video...

James A Mohler
  • 11,060
  • 15
  • 46
  • 72
dcolumbus
  • 9,596
  • 26
  • 100
  • 165
  • Perhaps you should specify which version of ColdFusion you're on. – Gert Grenander Jul 27 '10 at 01:09
  • 1
    Run a batch/shell script from cfexecute, I personally don't think this sort of memory intensive operation should be done within the CF jvm. Trying to solve the problem though, you should run something like visualvm so you can get an idea of where the memory is being held first. Then there may be approaches that can fix it. – David Collie Jul 27 '10 at 07:42
  • Dave, how would you suggest such a task be completed? – dcolumbus Jul 29 '10 at 22:52

5 Answers5

3

Great answer but I couldn't get your script to work so re-touched and what a difference!!!

<cfloop collection="#REQUEST#" item="mydex">
    <cfset StructDelete(REQUEST, "#mydex#", "True")>
</cfloop>
<cfloop collection="#VARIABLES#" item="mydex">
    <cfset StructDelete(VARIABLES, "#mydex#", "True")>
</cfloop>
Will Ryan
  • 31
  • 2
  • 1
    Will, there is no need/benefit to specify the third argument to [StructDelete](http://cfdocs.org/StructDelete) here. Just `StructDelete(Request,mydex)` will work exactly the same. (WTF is "mydex" though!?) – Peter Boughton Dec 12 '12 at 17:01
2

I remember reading that some server versions don't properly dispose of COM objects and the like when a page request is finished. If any of this is being done via a CFC or Java class that's being set to a variable, you can put this in OnRequestEnd.cfm:

<cfset StructDelete(variables)>
<cfset StructDelete(request)>

This will get rid of any variables set on the page which are no longer needed. Shouldn't have any negative side effects, and should clear up any memory that might be still lurking in one of the variables set during the processing of that page.

You might also look into using something other than <cfexecute> to process the videos. Maybe have a background process that routinely checks a folder for videos and then converts them in the background? ColdFusion isn't necessarily efficient when it comes to batch processing.

Jordan Reiter
  • 20,467
  • 11
  • 95
  • 161
  • Jordan... I'm using cfexecute because I don't know how else to run the FFMPEG script. Upon upload, I create a record in the DB and then periodically run CRON jobs that run the various .cfm to take care of converting and moving the videos. – dcolumbus Aug 13 '10 at 16:55
  • As per [Will's answer](http://stackoverflow.com/questions/3340064/coldfusion-cfexecute-ffmpeg-memory-issue#13844258) it might be necessary to loop through and delete each key individually. – Peter Boughton Dec 12 '12 at 17:03
  • 1
    calling StructDelete or StructClear might remove the reference to the variable from ColdFusion, but will do nothing for freeing up memory for the JVM. The JVM can only reclaim memory from ColdFusion AFTER the request is done being processed. – rip747 Dec 12 '12 at 21:46
  • @rip747 assuming there's some kind of intelligent garbage collector running, if the problem is the objects weren't being disposed, I'm guessing once CF disposes the object, then the JVM will clear out the memory pointing to the referenced objects. Or am I missing something? – Jordan Reiter Dec 12 '12 at 22:19
0

I don't understand why in the world you're trying to reinvent the wheel when I wrote a DSL wrapper for FFMPEG with the fix for the memory leak included in it:

https://github.com/rip747/cfffmpeg

Fork and submit any enhancements or fixes that you'd like.

BTW, if you want to see how to really handle the memory issues that you are having, then read the article by CFSEARCHING:

http://cfsearching.blogspot.com/2007/12/using-ffmpeg-to-convert-video-files-to.html

Again, this approach is included in the DSL.

rip747
  • 9,375
  • 8
  • 36
  • 47
  • Probably because the question was asked 2 years ago, and your code was made available 11 months ago. :) – Jordan Reiter Dec 12 '12 at 22:22
  • Also, where in the code is the memory fix? I couldn't find it. – Jordan Reiter Dec 12 '12 at 22:25
  • go fig. wonder why then it showed up in my google reader. oh well my mistake. for the fix, look at this file: https://github.com/rip747/cfffmpeg/blob/master/src/convert/process.cfm – rip747 Dec 13 '12 at 20:22
0

I have come across some instances where it's helpful to actually manually run JVM garbage collection from within CF, usually when there is a long running thread doing long-term queue management and the request is very long running.

It might be worth a shot in your case.

To run the garbage collector from within CF you call the following:

<cfset runtime = CreateObject("java", "java.lang.Runtime").getRuntime()>
<cfset runtime.gc()>

Hope it helps!

Ciaran Archer
  • 12,316
  • 9
  • 38
  • 55
  • in my experience that will do absolutely nothing. calling the garbage collector directly tell the JVM to try to cleanup all discarded objects, however in most cases ColdFusion will not mark any objects as ready to be discarded until AFTER it is done processing the request. – rip747 Dec 12 '12 at 21:44
0

If you are on windows, I recommend calling a batch file to do the conversion and file transfer. You can execute the Batch file from CF. This will prevent CF from consuming the entire memory for the conversion and the task can continue running in the background. If you want to wait to get the status, add a "timer" using a CF Java object instantiation to check the status after X seconds.

or you can call a cmd window to run it - http://www.forta.com/blog/index.cfm/2006/7/31/Using-CFEXECUTE-To-Execute-Command-Line-Utilities

eapen
  • 579
  • 6
  • 15