1

It seems that the default doctrine listener used by FOSElasticaBundle does not support bulk index by default. I have an application where I want to add support for more complex search queries through ElasticSearch. The search engine only will perform queries through one unique entity Post. When I create, edit or delete there is not any problem, the index in elasticsearch is updated automatically through the listener. My problem comes when I want to do bulk updates to hide or show more than one post at once, the listener is not receiving the signal to make the bulk index update in elasticsearch.

I am new to FOSElasticSearch so I do not know if I am missing something. I am using FOSElasticaBundle 6, Symfony 5.2 and ElasticSearch 7

Here you can find my fos_elastica.yaml

fos_elastica:
    messenger: ~
    clients:
        default: { host: 127.0.0.1, port: 9200 }
    indexes:
        product:
            settings:
                        index:
                            analysis:
                                analyzer:
                                    my_analyzer:
                                        type: snowball
                                        language: English
            persistence:
                        # the driver can be orm, mongodb or phpcr
                        driver: orm
                        model: App\Entity\Post
                        listener: { enabled: true }
                        provider: ~
                        finder: ~
                        elastica_to_model_transformer:
                            ignore_missing: true
            properties:
                        title: { boost: 10, analyzer: my_analyzer }
                        tags: { boost: 8, analyzer: my_analyzer }
                        description: { boost: 6, analyzer: my_analyzer }
                        ispublished: { type: integer}

And here you can find the way I am updating more than once entity element at once in PostRepository (the function is to update all post from one unique author, it is just an example):

   public function bulkUpdate($ispublished, $authorid){
        return $this->createQueryBuilder('p')
            ->update()
            ->set('p.ispublished', $ispublished)
            ->andWhere('p.authorid = :id')
            ->setParameter('id', $authorid)
            ->getQuery()
            ->execute();
    }

Also I found that I could disable default listener, dispatch messages for each create, update or delete action through symfony/messenger and consume them async in the background. I guess that I should create my own handler and dispatch specific messages (although I could not find an example about this in the doc) in each modifying action, although at the end I also have the same problem, as I do not know how to send a bulk index update/delete action

In the other hand I was thinking in executing all time a background script in python to check what rows were modified in mysql database and update those index with the script directly through ElasticSearch Api

I do not think that I will need to update more than 1k posts at once, so I would like to keep using the default listener to update posts automatically and avoid gaps between that an entity is modified and the index is updated in ElasticSearch. I just need to find the best way to update indexes in bulk as I have everything else already implemented and working

Sorry for all the text but I wanted to give all details about what I need to do

Kevin Gravell
  • 499
  • 2
  • 7
  • 20

0 Answers0