2

Getting extended processing times inside web2py when running clustering algorithms.

Tried running the algorithm on a standalone python instance and it finishes in 4-5s, in web2py, it takes over 10 minutes.

How do I pass the parameters from web2py user input to run the algorithm on a separate python instance, finish in 4-5s and return the results to web2py user view?

Aladdin Teng
  • 330
  • 1
  • 2
  • 13
  • What kind of data do you need to pass to and from the algorithm ? – hivert Jul 23 '13 at 11:32
  • Parameters like number of clusters, directory of data file (csv), which variable to cluster on, etc. The algorithm returns a python object, but I guess I can save that to an output file and return the file back to web2py. – Aladdin Teng Jul 23 '13 at 11:34
  • Did you try to profile you algorithm to understand why it's so slow in web2py ? I'd like to now if forking is sufficient or if you need to launch a brand new Python. – hivert Jul 23 '13 at 11:37
  • The algorithm is primarily a local search algorithm (Tabu) that generates several possible solutions and returns the optimal. Do you have any links on how to run a brand new Python from web2py? – Aladdin Teng Jul 23 '13 at 11:42

1 Answers1

3

You should use http://docs.python.org/2/library/subprocess.html#module-subprocess to a brand new Python instance and passing parameter through stdin / stdou using pickling (serialization).

hivert
  • 10,579
  • 3
  • 31
  • 56