It seems that the default doctrine listener used by FOSElasticaBundle does not support bulk index by default. I have an application where I want to add support for more complex search queries through ElasticSearch. The search engine only will perform queries through one unique entity Post
. When I create, edit or delete there is not any problem, the index in elasticsearch is updated automatically through the listener. My problem comes when I want to do bulk updates to hide or show more than one post at once, the listener is not receiving the signal to make the bulk index update in elasticsearch.
I am new to FOSElasticSearch so I do not know if I am missing something. I am using FOSElasticaBundle 6, Symfony 5.2 and ElasticSearch 7
Here you can find my fos_elastica.yaml
fos_elastica:
messenger: ~
clients:
default: { host: 127.0.0.1, port: 9200 }
indexes:
product:
settings:
index:
analysis:
analyzer:
my_analyzer:
type: snowball
language: English
persistence:
# the driver can be orm, mongodb or phpcr
driver: orm
model: App\Entity\Post
listener: { enabled: true }
provider: ~
finder: ~
elastica_to_model_transformer:
ignore_missing: true
properties:
title: { boost: 10, analyzer: my_analyzer }
tags: { boost: 8, analyzer: my_analyzer }
description: { boost: 6, analyzer: my_analyzer }
ispublished: { type: integer}
And here you can find the way I am updating more than once entity element at once in PostRepository
(the function is to update all post from one unique author, it is just an example):
public function bulkUpdate($ispublished, $authorid){
return $this->createQueryBuilder('p')
->update()
->set('p.ispublished', $ispublished)
->andWhere('p.authorid = :id')
->setParameter('id', $authorid)
->getQuery()
->execute();
}
Also I found that I could disable default listener, dispatch messages for each create, update or delete action through symfony/messenger
and consume them async in the background. I guess that I should create my own handler and dispatch specific messages (although I could not find an example about this in the doc) in each modifying action, although at the end I also have the same problem, as I do not know how to send a bulk index update/delete action
In the other hand I was thinking in executing all time a background script in python to check what rows were modified in mysql database and update those index with the script directly through ElasticSearch Api
I do not think that I will need to update more than 1k posts at once, so I would like to keep using the default listener to update posts automatically and avoid gaps between that an entity is modified and the index is updated in ElasticSearch. I just need to find the best way to update indexes in bulk as I have everything else already implemented and working
Sorry for all the text but I wanted to give all details about what I need to do