0

I have a function and inside of it I'm using os.fork():

def translating(words):
    '''Translate words using API'''

    number_of_threads = 3
    t = []   #list to store translated words
    l = len(words_words)
    n = (l - (l % number_of_threads)) / number_of_threads   #number of words for each PID I will create


    for thread_number in range(number_of_threads):
        pid = os.fork()
        if pid != 0:
            '''print('Process {0} spawned' .format(pid))'''
        else:
            for i in words[thread_number*n:thread_number*n+n]:   #slice of words for each PID
                translation = translate.translate(i, 'en')['text']  #API returns list of translated words
                for a in translation:
                    t.append(a)
            os._exit(0)
    return t

My problem is 'return t' happens right after the second (from 3) PID is created. Before 'for-loops' finish iteration. Im stuck and have absolutely no idea why does this happen... What am I doing wrong?

Desprit
  • 707
  • 2
  • 11
  • 24
  • 1
    You have to use `os.wait()` to wait for a child to finish. Otherwise, the two processes run independently. – Barmar Jul 03 '14 at 21:25
  • 1
    See the _Multiple Forks_ example here: http://www.petercollingridge.co.uk/blog/running-multiple-processes-python – Barmar Jul 03 '14 at 21:30
  • Thank you Barmar, your advice helped me and after implementing os.waitpid() my function doesnt stop before all pids finish their job. But as user2040251 told below, the list I get is empty and he is absolutely right, I didnt mention it before. So I'll do my best to share data betweeb children and parent. Thank you again! – Desprit Jul 03 '14 at 21:49

1 Answers1

1

This code will always return an empty list anyway. fork creates a new process and it does not share memory with parent process. That is, each child process modifies its own copy of t list. You need to use interprocess communication to actually update t in parent process.

kraskevich
  • 18,368
  • 4
  • 33
  • 45
  • Thank you sir, that is true, my list is empty as I see now. Would you be so kind to advice me the best way to share data between children and parent please? Here http://stackoverflow.com/questions/6227039/python-fork-passing-data-from-child-to-parent Mr Gareth Rees told to use multiprocessing module. Is that the right way to go? – Desprit Jul 03 '14 at 21:52
  • 1
    You can use either a pipe or shared memory with mmap. It is difficult to say which is the best because it depends. – kraskevich Jul 03 '14 at 21:56