0

I am dealing with a recommendation problem, which involves 3 million users and 500,000 products. The purpose of the recommendation is to recommend 5-10 more products to the particular user when visiting a given product.

We finished the real-time recommendation engine part, but we found that it is not so easy to recommend in 1-2 seconds. We consider two strategies:

  1. Run the recommendation off-line in a batch mode, then store the results in a MySQL DB. However, it seems like that the number of entries in the tables is a huge number. Each user for each product should store at least 5-10 recommended products. If I create a table with user and product as table attributes, there will be 3 million x 500,000 entries.

  2. Predict the recommendation on the fly. We just tried it, but the response time is too long.

Can anyone give me some suggestions on this problem?

MWiesner
  • 8,868
  • 11
  • 36
  • 70
user1841342
  • 63
  • 1
  • 4
  • Why are you generating per user for every item? (just curious) – Julian Ortega Jun 07 '13 at 13:47
  • It depends on how the recommendation engine works - If the recommendation engine is just looking at the customers recent purchases (which should be relatively few), and there is some sort of (efficient) index between recent purchases and recommended products, then real-time recommendations should be fairly feasible. Can you tell us a little more about how the recommendation engine works? – Justin Jan 09 '14 at 14:41

1 Answers1

0

It looks strange to me to create a list of recommended products for each user-product-pair. Normally you only have a list of similar products for each product for use cases like yours.

fatih
  • 1,395
  • 10
  • 9