How to implement matching in between depositors and withdrawers using Node.js?

Question

I have a matching-engine microservice requirement for my application, i need to implement a matching between depositors and withdrawers . So, all the details of depositors and withdrawers are stored in redis cache. There will be tons of requests coming into this microservice and are put into redis cache. i need to match on the basis of same amounts.

1) How to do the matching, do i need to fetch details from cache into two separate arrays and perform matching on these two arrays. Iterating with both the arrays, is it the right way to do it?

2) As and when the data fetching into arrays goes bigger , the matching gets complicated and will take lot of time too. What is the better approach to deal with this scenario?

 Consider the following scenario:-

 w/d     |w/d amt|w/d-bonus|d/p    |  depositors   
 personA |10000  |   100   |person1|     70000
 personB |50000  |   750   |       |      
 personC |20000  |   100   |       |
 personD |1000   |   50    |       |
 personE |100000 |   1000  |       |

here person1 will be matched first with personB (50K -50K), personB is completely matched hence he will give person1 750 amt as bonus to depositor.

now still 20k of depositor has to be matched. so, that will be matched with personE , since personE got only 20K (personE needed 100K) he will give only 200 amt (for 100K we have 1K bonus then for 20K bonus will be 200/- amt)as bonus to depositor.

Point to be noted:

1) person1 is first matched with personB then he is matched with personE.

2)person1 could have matched totally with personE but person1 would get less bonus that is 700/- amt.

3)but now since person1 is matched with personB and personE , person1 will receive total bonus of 950/- amt 

Main aim is to provide depositor a maximum bonus.

above is an example , the perfect match would be D,B,A,E and D,B,E
we will consider D,B,A,E since we can satisfy 4 withdrawers and yet 
give the depositor maximum bonus. 

here: 
w/d => withdrawer
d/p => depositor
amt => amount

Answer 1

Keep a sorted set with your withdrawer elements, using as score bonus ÷ amount.

For

w/d     |w/d amt|w/d-bonus
personA |10000  |   100   

You store it using ZADD:

ZADD withdrawers 0.01 "personA|10000|100"

Then you consume using ZPOPMAX to get and remove the top element. Or you can use ZREVRANGEBYSCORE to get the same without removing, as follows:

ZREVRANGEBYSCORE withdrawers +inf -inf WITHSCORES LIMIT 0 1

When processing a withdrawer with ZPOPMAX, if the match is partial then you ZADD it again with the reduced amount and bonus.

If you are serving your depositors first-in first-out, use a list to queue them: LPUSH to queue, RPOP to process.

When processing a depositor, if the match is partial then you RPUSH it with the reduced amount.

Your match-maker client can use the blocking versions of the commands to consume: BRPOP for depositors and BZPOPMAX for withdrawers.

But if you have more than one engine doing the matches, you are better off doing the whole match using a Lua script for atomicity. The script should pop from both queues if none is empty, push back any updated record partially matched, and then return the match result if any (nil if a queue was empty) so you can process it: store it, execute it, whatever you're doing.

Initially, we said to use score = bonus ÷ amount for withdrawers, to maximize bonus for the depositors. But this leaves somewhat at random how to resolve ties in the score (you would get the lexicographically-sorted max record first).

There are many solutions to this.

You can use a more complex score to solve ties. For example, to get the bonus-maximizer with a minimal amount first:

score = round(bonus * 1000000 ÷ amount) + 1 ÷ amount

By rounding, you consider a technical tie a range of bonus ÷ amount. The size of this range is determined by the inverse of the multiplier. Here, 1000000 makes this range of size 0.000001.

Or you can simply do ZREVRANGEBYSCORE withdrawers score score to get all elements in a tie at the top, after calling ZREVRANGEBYSCORE withdrawers +inf -inf WITHSCORES LIMIT 0 1 to get the max score. Then you resolve the tie client-side.

Or you can try to use the lexicographical property of sorted sets, tweaking how you store the elements.