4

this will be a very baic question since im new to Spring-Redis

Im currently in the process of learning about Redis database and I'm working on a feature on priority, im compelled to Use Redis for this feature. Below in the challenge/Query im having.

Right now we have a DataModel as below:

@RedisHash("Org_Work")
public class OrgWork {

   private @Id @Indexed UUID id;
   private @Indexed String CorpDetails;
   private @Indexed String ContractType;
   private @Indexed String ContractAssigned;
   private @Indexed String State;
   private @Indexed String Country; 

}

public interface OrgWorkRepository extends CrudRepository<HoopCalendar, String> {

List<OrgWork> findByCorpDetailsAndContractTypeAndStateAndCountry(String CorpDetails, String ContractType, String ContractAssigned, String State, String Country);

}

we are developing an API to query on the above Datamodel where the front-end will send us CorpDetails ,ContractType, ContractAssigned, State and Country fields and we have to query these against the Redis Database and return back the DurationOfWork object.

In this case I will be having a load of approx. 100000 calls per minute.

Please let me know on if this is the right way and some suggestions on improving response time.

***Updated the query

Robert Niestroj
  • 15,299
  • 14
  • 76
  • 119
U C
  • 137
  • 2
  • 8
  • Is the query "CorpDetails ,ContractType, ContractAssigned, State and Country" expected to return just one result? I mean, can we treat it as a compound primary key? Or you mean to query against any combination of these fields? – LeoMurillo Jan 23 '20 at 20:45
  • we expect multiple records for the query. (the query is mentioned in OrdWorkRepository Interface) – U C Jan 23 '20 at 22:14

1 Answers1

7

See Spring Data Redis - 8.5. Secondary Indexes and:

  • 8.6. Query by Example
  • 8.10. Queries and Query Methods

The annotation @Indexed instructs Spring Data Redis (SDR) to create a secondary indexed as a set to index the field of the hash.

This means when you insert data, SDR will run seven commands to Redis:

HMSET "OrgWork:19315449-cda2-4f5c-b696-9cb8018fa1f9" "_class" "OrgWork" 
    "id" "19315449-cda2-4f5c-b696-9cb8018fa1f9" 
    "CorpDetails" "CorpDetailsValueHere" "ContractType" "ContractTypeValueHere" 
    ... "Country" "Costa Rica"
SADD  "OrgWork" "19315449-cda2-4f5c-b696-9cb8018fa1f9"                           
SADD  "OrgWork:CorpDetails:CorpDetailsValueHere" "19315449-cda2-4f5c-b696-9cb8018fa1f9"
SADD  "OrgWork:ContractType:ContractTypeValueHere" "19315449-cda2-4f5c-b696-9cb8018fa1f9"
...
SADD  "OrgWork:Country:Costa Rica" "19315449-cda2-4f5c-b696-9cb8018fa1f9"

Using Query by Example:

You want to create a repository:

interface OrgWorkRepository extends QueryByExampleExecutor<OrgWork> {
}

And then implement the query as in the example service below:

class OrgWorkService {

  @Autowired OrgWorkRepository orgWorkRepository;

  List<OrgWork> findOrgWorks(OrgWork probe) {
    return orgWorkRepository.findAll(Example.of(probe));
  }
}

And use as:

OrgWork orgWorkExample = new OrgWork();                          
orgWorkExample.setCorpDetails("CorpDetailsValueHere"); 
orgWorkExample.setContractType("ContractTypeValueHere");
...
List<OrgWork> results = orgWorkService.findOrgWorks(orgWorkExample);

Behind the scenes, SDR will take care of converting this to Redis commands to get your data, using a combination of SINTER and HGETALL:

SINTER   …:CorpDetails:CorpDetailsValueHere   …:ContractType:ContractTypeValueHere   ...
HGETALL "OrgWork:d70091b5-0b9a-4c0a-9551-519e61bc9ef3" 
HGETALL ...

This is a two-step process:

  1. Fetch keys contained in the intersection of secondary indexes, using SINTER
  2. Fetch each key returned by <1> individually, using HGETALL

A workload of 100,000 per minute should be manageable for Redis assuming you have a quality server, a reasonable dataset size, and the queries are somewhat specific in average.

SINTER has a time complexity of O(N*M) worst-case where N is the cardinality of the smallest set and M is the number of sets. You have one set for every dimension on your query.

HGETALL is O(N) where N is the size of the hash, 7 in your case.

As always, it is recommended you do some benchmarking to test if you're getting the desired performance and adjust if needed.

LeoMurillo
  • 6,048
  • 1
  • 19
  • 34
  • Thank you so much for your explanation @LeoMurillo . I have one more question.. what is the advantage of querying the way you explained over the below ? `List findByCorpDetailsAndContractTypeAndStateAndCountry(String CorpDetails, String ContractType, String ContractAssigned, String State, String Country) ` The way I mentioned in the second part of my question? – U C Jan 23 '20 at 22:10
  • 1
    Only the flavor of the code, on how you build the query. I figured you got one already, wanted to present the other one. I find QBE more object-oriented, the automatic derivation on CrudRepository smells a bit as long parameters; also, if you rename a field, you need to rename the function, I see a maintainability risk there, not sure if you get a compilation error or a warning or nothing in that case. A matter of taste. Behind the scenes, both should arrive to the same commands sent to Redis and exhibit the same performance – LeoMurillo Jan 23 '20 at 22:17
  • 1
    By the way, note the method name is missing AndContractAssigned, it is quite a long name :-) – LeoMurillo Jan 23 '20 at 22:20
  • Yeah Leo, even Im thinking of replacing the query with the way you just stated as it appears more OO . – U C Jan 23 '20 at 22:24
  • I have one more small question @Leo , Is there a way we can perform a range query ... say if there is a date attribute in the pojo class, can I construct a query to retrieve the results between certain range of dates ? like records with country US and State Washington and Date between 2019-07-19 to 2019-12-31 – U C Jan 23 '20 at 22:26
  • 1
    If it is supported is news to me, I think it isn't - it may be worth another SO question :-) – LeoMurillo Jan 23 '20 at 22:51
  • posted that question https://stackoverflow.com/questions/59888313/range-querying-in-redis-spring-data-jpa-redis Thank you so much for your answer again :) – U C Jan 23 '20 at 23:07