How to realize server-side caching for autocomplete in a ZF2 application?

Question

First the actual state: There is a ZF2 application with a form. The form contains some autocomplete fields (implemented on the frontend side with jQuery Autocomplete).

The SQL statements behind it look like this:

SELECT name FROM countries WHERE countries.name LIKE %en%
-> should find "Arg[en]tina", "Arm[en]ia", "B[en]in", "Turkm[en]istan" etc.
or
SELECT name FROM countries WHERE countries.continent_id = 2
-> should find "Afghanistan", "Armenia", "Azerbaijan" etc. (2 = Asia)
or
SELECT name FROM countries WHERE countries.continent_id = 2 AND countries.name LIKE %en%
-> should find "Arm[en]ia", "Turkm[en]istan" etc. (2 = Asia)

Of course, it leads to the problem, that the database gets terrorized by a lot of small autocomplete requests. Caching should help -- and I've already started implementing a Zend\Cache\Storage\Adapter\Apcu-based caching mechanism. But then I saw the next trouble: Using a common cache like APCu I cannot filter the results dynamically. So such a cache seems not to work for a case with autocomplete.

I'm pretty sure, that it's a common issue and there is already a solution for this.

How to realize a caching mechanism in a ZF2 application for the autocomplete functionality?

score 0 · Answer 1 · answered Oct 30 '16 at 23:54

There is nothing to do with the ZF2 here. This is all about a custom search service design and it's challenges.

Without a proper caching layer and/or full-text search engine, building an autocomplete implementation such this would be suicide for the application. You can easily achieve tens of thousands of unnecessarily repeated queries in a very short time.

In an "ideal" world, a good autocomplete implementation utilizes a full-text search engine under the hood such as Elasticsearch or Apache Solr. And uses their Completion Suggester and Suggester components respectively.

Anyway a simple autocompletion feature still achievable using only an object cache and database. Only you need is a helper method to create a proper "cache key" for the every letter combinations. For example:

   function createKeyByQuery($str)
   {
      return 'autocomplete-prefix-'.(string) $str;
   }

and in your suggest() method:

   public function suggest($keyword)
   {
       $key = $this->createKeyByQuery($keyword);
       if($this->cache->hasItem($key)) {
           return $this->cache->getItem($key);
       }

       // fetch form the database here
       $data = $this->db->query();
       $this->cache->setItem($key, $data);

       return $data;
   }

If the count of your filters are not too much, just make them part of the key too. In this scenario, signature of the suggest method would be:

  public function suggest($keyword, array $filters = []);

and key generator needs an update:

   function createKeyByQuery($str, array $filters = [])
   {
      return 'autocomplete-prefix-' . (string) $str . md5(serialize($filters));
   }

This solution may not appropriate for the complicated/domain related data because it has a pretty big invalidation challenge. Eg. how you would find the cache keys which holds the "Argentina" in the payload?

Since you're dealing only with the list of countries and continents as filter, it should solve the problem.

For the england keyword and two different filter with total of 10 filter options, there will be 10x2x7 = 140 distinct cache key(s). For a single filter with 5 option, 5x1x7 = 37 distinct keys.

APCu is a good choice for this implementation.

Hope it helps.

How to realize server-side caching for autocomplete in a ZF2 application?

1 Answers1