2

Have a 500.000 records database in xml form, try to work with in a xe5 Delphi 64bit application (ClientDataSet). If I read it completely it takes more than 4 GB memory. Tried to use the fetchondemand=False and PacketRecords=5000 properties (as the documentation says), but it does not work at all. The application is simply insensitive to these changes. Could anybody help me?

Laszlo
  • 51
  • 7
  • 1
    Can you edit this and post some code to reproduce the problem? – Mason Wheeler Feb 11 '14 at 17:20
  • Are you sure you don't mean FetchOnDemand = True (for the CDS to take any notice of the positive PacketRecords value)? Anyway, I could be wrong but I wouldn't expect either of these properties to affect the loading of the data from XML, because I thought the fetching of records is mediated by the Provider associated with the CDS and I didn't think the Provider is involved in loading from XML. – MartynA Feb 11 '14 at 17:36
  • @MasonWheeler Don't think any code is needed, since the application WORKS if I load all the records. MY problem is "only" the memory use, that's why I want to control how many records to load with one fetch – Laszlo Feb 11 '14 at 17:42
  • @MartynA Sure, FetchOnDemand = True loads automatically the records when the application needs them (by scrolling for example). False give the possibility to control when and how many to load. By the other hand, maybe You're right about the XML data, but nowhere is something precise about this. – Laszlo Feb 11 '14 at 17:51
  • @MasonWheeler Tried to set these properties in design mode, not in running mode (by program code). – Laszlo Feb 11 '14 at 17:55
  • 1
    @Laszlo: You can post that too; design mode is saved as DFM files, which are (usually, unless you mess with your project settings) a text format. – Mason Wheeler Feb 11 '14 at 18:17
  • @MasonWheeler This is the only reference to the ClientDataSet n the main (corresponding) dfm file: object ClientDataSetM: TClientDataSet Aggregates = <> FileName = 'N:\DP-DATA\Lee_County\LCPA\LCPAPerm_TC_Arh_Real.xml' FilterOptions = [foCaseInsensitive] Params = <> ReadOnly = True Left = 576 Top = 288 end – Laszlo Feb 11 '14 at 18:32
  • 1
    @Laszlo: You're missing the point of my request. Please *edit your question and post the actual code,* both on the DFM and the PAS side, enough for those of us reading to be able to reproduce it locally. Otherwise, it's very difficult for us to just magically intuit what the problem might be. – Mason Wheeler Feb 11 '14 at 19:17
  • @MasonWheeler I think you do not understand my point. The problem is the same for ANY similar program. I changed only two (default) options in design time as mentioned above. You could reproduce my problem with any short program with a ClientDataSet reading an XML database. If you want I could make a short demo for you, but I'm sure you could do it in 5 minutes better than me. You will see that changing the options nothing is happening. The questions are: these options really do not affect the XML database? If yes is there any other method to fetch only partial data in the memory in this case? – Laszlo Feb 11 '14 at 21:09
  • @MartynA Martyn you're right, FetchOnDemand is working only intermediated by a DataProvider component and a second DataSet. Thank you. – Laszlo Feb 13 '14 at 15:01

1 Answers1

1

Thank you for ALL who read my question and tried to help me. Finally, I found a demo where the mechanism is explained clearly. The solution is quite simple: there must be TWO datasets involved when want to work with fetchondemand, to limit the used memory. The first dataset with the default settings (True and -1)) is pointing to the XML database, the data provider is pointing to this, a second dataset (which has the fetchondemand property set to False and the DataPackets set to the desired record number to fetch in one packet) is pointing to the data provider, and the data aware grid is pointing to this second dataset only. The programmer must code the paging mechanism for parsing the whole XML database with the grid, with the GetNextPacket procedure.

Sorry for disturbing you with such a dumb question, hope the answer will help all the guys who were not clear with this mechanism as I was.

Thank you again,

Laszlo

Laszlo
  • 51
  • 7