0

I am trying to do an excel import of product data where the excel will have data which is already existing in the database. So it turns out to be updating the existing product data from the values coming from excel. Each product data has relations like Tags, Categories, etc. We assume Tags to be a master data which is unique. So each product can link to tags. I used the Doctrine ORM and using the persist and flush combination. But it takes a lot of time and memory to execute. Sometimes it exceeds memory limit when product entities are larger in number. Since Tags are shared, there are many number of SELECT statements executes during the persist. I tried to apply flush() and clear() during the operation.

Can someone suggest some solution on this. Can we use cache to eliminate frequent statements for data fetch if already selected ie shared among entities.

Kiran Krishnan
  • 30
  • 5
  • Not sure exactly what you are asking. You can certainly store a Tag in an array the first time you load it from the database and then check the array each time you need the same tag. When you flush then you have to empty the array as well and basically start over. Doing something like this can indeed save quite a bit of memory and processing time. – Cerad Dec 05 '20 at 13:35
  • Makes sense, but most of my logic is there in the Entities, that make sure Tags are not repeating. So i was thinking is there any option to add a cache/array cache in the find methods for each repository , so when we fetch a Tag from db, this can be stored in cache. – Kiran Krishnan Dec 07 '20 at 03:24
  • I'm still not understanding the actual question. Obviously you can tweak TagRepository::find($id) to work the way you want. Pretty sure you know that so you asking something different. Maybe add a bit of code to your question that illustrates the specific problem? Try to keep it simple. – Cerad Dec 07 '20 at 14:20

0 Answers0