0

I want to to take samples from a population and then doing some operation on the samples. I want to program this in Python. Do I have to use multiprocessing, multithreading or something like map-reduce? And how can I test if it is really faster? On my Macbook-Air it is not faster when I use threading for simple print out, then if I do a normal loop.

peterh
  • 11,875
  • 18
  • 85
  • 108
Hannah Schmidt
  • 13
  • 3
  • Welcome to stack overflow. All depend on number of samples you need to analyze and they size. Generally with anything bellow 1 billion or more, you don't need anything more than a normal loop maybe with some optimalization in the code.. but thats just a generalization. If you provide more specific information - someone could help eg. size of each sample, operation on samples that needs to be done; number of samples in the storage. – nakashu May 11 '16 at 13:50
  • 1
    To offer help, it would be great to have a look at what you are currently trying. Maybe you could post a simplified version of your code. – Dave May 11 '16 at 13:52
  • Thanks a lot. When I have more exactly information what I really want to do, I will be more specific. Could you tell me, why it is not faster at all, when I want to print 1000-times "Hello World" with python when I do a simple for-loop and wenn I do a for-loop and start threads. Is threading faster on a simple laptop (mac-air) or is it just faster when I start doing it on big machines? Thanks. – Hannah Schmidt May 11 '16 at 13:57
  • See this: https://wiki.python.org/moin/ParallelProcessing – John Coleman May 13 '16 at 15:17

1 Answers1

0

Assuming you are talking about cPython, the implementation has something known as GIL (global interpreter lock), which doesn't let the threads to execute on multiple CPU cores, so threading library can't be actually parallel which can explain why your print out test wasn't faster. Assuming you understand your problem and you have correctly divided your problem for parallelism, multiprocessing library can help you run your problem in parallel.

khajvah
  • 4,889
  • 9
  • 41
  • 63