Why do I leak memory in EOutOfResources?

Question

I try to implement a wrapper around this jpeg decoder library (original by Arnaud Bouchez). The library is DAMN fast but it does not support all jpegs!

For very large jpg files it fails (as expected) with a EOutOfResources exception.
So I try to silently skip those files. It works, but when I close the app, FastMM indicates a memory leak.

function FastJpgDecode(FileName: string; OUT ErrorType: string): TBitmap;
var Img: PJpegDecode;
    res: TJpegDecodeError;
    Stream: TMemoryStream;
begin
  Result:= NIL;
  Stream:= TMemoryStream.Create;
  TRY
    if Length(FileName) > MAX_PATH then { TMemoryStream does not support long paths }
     begin
      ErrorType:= 'File name too long!';
      Exit;
     end;

    Stream.LoadFromFile(FileName);
    Stream.Position:= 0;
    res:= JpegDecode(Stream.Memory, Stream.Size, Img);       
    case res of
     JPEG_SUCCESS:
      begin
       try
        Result:= Img.ToBitmap; // This will raise an EOutOfResources for large files!
       except
        on EOutOfResources do
          ErrorType:= 'JPEG_OUTOFMEM!';
       end;
      end;

     JPEG_EOF                : ErrorType:= 'JPEG_EOF!';
     JPEG_OUTOFMEM           : ErrorType:= 'JPEG_OUTOFMEM!';
     JPEG_CPUNOTSUPPORTED    : ErrorType:= 'JPEG_CPUNOTSUPPORTED!';
     JPEG_BADFILE            : ErrorType:= 'JPEG_BADFILE!';
     JPEG_FORMATNOTSUPPORTED : ErrorType:= 'JPEG_FORMATNOTSUPPORTED!';       // Not all jpegs are supported. In this case we fall back to WIC or the standard LoadGraph loader (WIC).   
    end;
  FINALLY
    Img.Free;
    Stream.Free;
  END;
end;



function TJpegDecode.ToBitmap: TBitmap;
begin
  if @self=nil
  then result := nil
  else
   begin
    result := TBitmap.Create;
    try
     if not ToBitmap(result)   // This will raise an EOutOfResources for large files!
     then FreeAndNil(result);
    except
      FreeAndNil(Result);
      raise;
    end;
   end;
end;

A memory block has been leaked. The size is: 36

This block was allocated by thread 0xD0C, and the stack trace (return addresses) at the time was: 407246 40830F 408ADE 43231B [Unknown function at __dbk_fcall_wrapper] 407246 40A532 53C353 [Unknown function at TMethodImplementationIntercept] 6E006F [Unknown function at TMethodImplementationIntercept] 7765648F [RtlNtStatusToDosError] 77656494 [RtlNtStatusToDosError] 767A7BEA [Unknown function at IsNLSDefinedString]

The block is currently used for an object of class: EOutOfResources The allocation number is: 4181

Current memory dump of 256 bytes starting at pointer address 7EEEA6C0: 74 7F............ t D . ü $ ú ......

A memory block has been leaked. The size is: 132
This block was allocated by thread 0xD0C, and the stack trace (return addresses) at the time was: 407246 40A2E7 40A518 53C341 [Unknown function at TMethodImplementationIntercept] 6E006F [Unknown function at TMethodImplementationIntercept] 7765648F [RtlNtStatusToDosError] 77656494 [RtlNtStatusToDosError] 767A7BEA [Unknown function at IsNLSDefinedString] 7677F0BA [VirtualQueryEx] 7677F177 [VirtualQuery] 898FD9 [GetFrameBasedStackTrace]

The block is currently used for an object of class: UnicodeString

The allocation number is: 4180

Current memory dump of 256 bytes starting at pointer address 7EFA24F0: B0 04 02 00 01 00 00 00......... . . . . . . . : . . . Not . . enough storage . . i . s . . a . v . a . i . l . a . b . l . e . . t . o . . p . r . o . c . e . s . s . . t . h . i . s . . c . o . m . m . a . n . d ............

This application has leaked memory. The small block leaks are (excluding expected leaks registered by pointer):

21 - 36 bytes: EOutOfResources x 1 117 - 132 bytes: UnicodeString x 1

Why does it leak memory there?

Answer 1

As @kami mentioned in comments, EHeapException has an internal AllowFree flag that is False by default, preventing instances of EHeapException from being freed by exception handlers.

EOutOfResources derives from EOutOfMemory, which in turn derives from EHeapException.

The SysUtils unit has 2 singleton objects of type EOutOfMemory and EInvalidPointer. Whenever the RTL raises those two specific exception types directly, it raises the same instance of those classes every time. So they have an AllowFree flag to prevent exception handlers from freeing the singletons. The singletons are freed when the SysUtils unit is finalized.

This is actually documented behavior:

http://docwiki.embarcadero.com/Libraries/en/System.SysUtils.EHeapException

Note: Memory for these exceptions is pre-allocated whenever an application starts and remains allocated as long as the application is running. Never raise EHeapException or its descendants directly.

http://docwiki.embarcadero.com/Libraries/en/System.SysUtils.EOutOfMemory

Memory for the EOutOfMemory exception is pre-allocated whenever an application starts and remains allocated as long as the application is running.

Note: Never raise EOutOfMemory directly. Instead, call the global OutOfMemoryError procedure.

However, although EOutOfResources derives from EHeapException, it is never used in a singleton manner, so its AllowFree flag really should never be False. So it seems to me that there are several bugs at play here:

• EOutOfResources is not really a heap error and should not have been derived from EHeapException to begin with. It is actually a common exception, for instance the Vcl.Graphics unit raises EOutOfResources for some of its GDI errors, which have nothing to do with the heap.

• EOutOfResources has its AllowFree flag set to False when it should be True instead. And the flag is private, so it can't be overwritten except by the SysUtils unit, which does so only for the 2 singletons during finalization. So basically, all EHeapException-derived exceptions get leaked at runtime.

• the singletons, as well as all other descendant instances, are not passed to the RTL's RegisterExpectedMemoryLeak() function when AllowFree is False so they can be omitted from leak reports.

This leak issue has existed since Delphi 5, and has already been reported to Embarcadero:

RSP-17193: EOutOfResources memory leak

RSP-19737: EOutOfResources exception causes memory leak