I am implementing a recommender system (python 3.9) for an online supermarket (hence users can re-order products). All the data is in the correct format which led to an interaction matrix, a weights matrix (to show the importance of some of the interactions), user features and item features. To make sure I don't get the warning 'Test interactions matrix and train interactions matrix share .. interactions', I have excluded all the train interactions from the test set. So if a user buys bananas every week, in the test set we don't have an interaction for bananas. But this does compute the precision, which, for the training precision, is 1 and validation precision is above 1.
precision_at_k(model, test, train, k=20, num_threads=NUM_THREADS, preserve_rows = True, **fit_features)
When I test the precision at k function from lightfm bit by bit, I see that they use predict_rank and this results into a lot of products getting the rank 0, which means (according to the source code: with 0 meaning the top of the list (most recommended) and n_items - 1 being the end of the list (least recommended). However then 90% of all products get rank 0. From these ranks, you continue in the source code with ranks.data, this is the part I don't understand. This array is longer than all my users but shorter than all my users * products. When it performs the computations "np.squeeze(np.array(ranks.sum(axis=1)))", it counts all the products that are in the top k however this leads to a number higher than k. This is the code:
ranks = model.predict_rank(
test_interactions,
train_interactions=train_interactions,
user_features=user_features,
item_features=item_features,
num_threads=num_threads,
check_intersections=check_intersections,
)
ranks.data = np.less(ranks.data, k, ranks.data)
precision = np.squeeze(np.array(ranks.sum(axis=1))) / k
