4

I'm using TIBDataSet to fetch records from Firebird database table using Delphi 2007. There is about 1 million records in that table and I'm getting the Out of memory error. The same query runs correctly with TClientDataset though.

Can you please tell me the difference between TClientDataset and TDataSet concerning the memory management ?

TLama
  • 75,147
  • 17
  • 214
  • 392
Aseem Choudhary
  • 63
  • 5
  • I do not know what exactly the difference is between them, but IIRC the out of memory problem in a TIBDataset could be solved by setting the `Unidirectional` property to true. Mind you, this changes the way the dataset behaves so it might not be appropiate in your case. – Guillem Vicens Dec 26 '12 at 08:37
  • 8
    Usually, when I read that somebody reads 1 Mio records, something is usually wrong in his design. There is usually a better way to solve your problem. – alzaimar Dec 26 '12 at 10:11
  • I do understand the comment by @alzaimar On the other hand: reading a million records should not crash a data access layer. – Jeroen Wiert Pluimers Dec 26 '12 at 14:12
  • Well, it depends on the machine, other running apps, in general: the amount of memory available, doesn't it? – alzaimar Dec 27 '12 at 08:58
  • Set Unidirectional to true as it was said above. And try to rethink your logic. If you need a million of records to do some calculations better do it server side using SQL query (SELECT with aggregates or EXECUTE BLOCK for complex evaluations) – Andrej Kirejeŭ Dec 27 '12 at 13:11
  • i think Tdataset provide temporary memory so when memory limit exceed it uses cache memory and this file is store with the extension .identcache and this will provide cache memory to execute operation and it limit is fixed to cache membory and TClientDataset load all data on client machine(harddisk) so its limit fixed to the size of client machine harddisk – Aseem Choudhary Jan 09 '13 at 09:28

1 Answers1

-1

@alzaimar has a point, millions of records loaded into memory on a 32bit platform is going to be a challenge whatever the dataset component you use.

So memory on the box is crucial.

We do similar things with applications written in Delphi 2006. What we do is run them on Windows7 64 bit or server 2008 64 bit.

Taking advantage of the additional memory for the OS and other applications.

Try The best memory manager for Delphi

Also put {$SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE} in the Source of yur project to get up to 4GB out of 32bit delphi on a 64bit box.

program Project15;

uses
  Forms,
  Unit15 in 'Unit15.pas' {Form15};

{$R *.res}

{$SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE}

begin
  Application.Initialize;
  Application.CreateForm(TForm15, Form15);
  Application.Run;
end.

Doing the above will let you see which is best for your application TDataSet or TClientDatset.

You might also consider using some third party data connection components that have improved performance and memory management over the basic Delphi data objects.

We use Devart components and SQL tools and they really are worth the money. Devart Data tools website

Jason Richards
  • 21
  • 1
  • 6