5

the benchmark for crab is (http://www.slideshare.net/marcelcaraciolo/crab-a-python-framework-for-building-recommender-systems page-37) 

Benchmarks Pure Python w/ Python w/ Scipy Dataset dicts and NumpyMovieLens 100k 15.32 s 9.56 s http://www.grouplens.org/node/73 Old Crab New Crab

however, in my case, I need to take more than 30 minutes to do it, I don't know the reason

my code is

model = MatrixPreferenceDataModel(recommend_data.data)
similarity = UserSimilarity(model, pearson_correlation)
recommender = UserBasedRecommender(model, similarity, with_preference=True)
recommender.recommend("6")

my data is NumpyMovieLens 100k, which contains 100,000 ratings from 1000 users on 1700 movies.

Tinyfool
  • 1,460
  • 2
  • 18
  • 40
user1687717
  • 3,375
  • 7
  • 26
  • 29

2 Answers2

1

When I last looked at Crab a little over a year ago, it was using dense arrays were it should be using sparse matrices (e.g. scipy.sparse). That made it horribly slow as well as a memory hog. I just had a quick glance and it seems Crab hasn't been updated for almost a year, so the situation is probably still the same.

Fred Foo
  • 355,277
  • 75
  • 744
  • 836
  • So, do you know any better python UserBasedRecommender lib? – Tinyfool Jan 16 '13 at 11:12
  • @Tinyfool: no, but it might be possible to hack sparse matrix support into Crab. Scipy has sparse SVD (`scipy.sparse.linalg.svds`); scikit-learn has k-NN with some support for sparse matrices. – Fred Foo Jan 16 '13 at 12:15
1

Crab is being re-designed. The data manipulation will be more efficient and less dependent of external libs. The API it self will be simplified. We want to release a stable version soon so the community can be involved. The objective is create a framework to fill this gap in Python. The project is hosted here https://github.com/python-recsys/crab.

Help and suggestions are very welcome.

Igor Medeiros
  • 4,026
  • 2
  • 26
  • 32