How to divide work between processes in MPI

Question

I have the following code to count the number of primes, I have divided the work in the loop across processors. The problem is that when the subroutine assigns sub-sections of the loop to processors it assigns according to rank and i cant seem to control the order in which they arrive

i.e i would like then to come as 0,1,2,3... and not like 2,1,0,3..

So that if there were 500 iterations in the loop and 5 processors.

• Rank 0 executes [1 - 100]

• Rank 1 executes [101-200] etc...

        program main
        implicit none
        include 'mpif.h'
        integer(4), parameter :: n = 36500
        integer(4) :: a(n)
        integer(4) :: i
        integer(4) :: j
        integer(4) :: ista
        integer(4) :: iend
        integer(4) :: sum
        integer(4) :: f=0
        integer(4) :: ssum
        integer(4) :: ierr
        integer(4) :: iproc
        integer(4) :: nproc
        call MPI_INIT(ierr)
        call MPI_COMM_SIZE(MPI_COMM_WORLD, nproc, ierr)
        call MPI_COMM_RANK(MPI_COMM_WORLD, iproc, ierr)
        call loop_range(2, n, nproc, iproc, ista, iend)
        sum = 0.0
        print *,ista,"-",iend,">",iproc
        do i = ista, iend
          f=0
          do j=2,INT(SQRT(REAL(i)))
         if(MOD(i,j)==0) then
         f=1
         end if
         end do
         if(f==0) then
         sum = sum + 1
        end if  
        end do
        call MPI_REDUCE(sum, ssum, 1, MPI_INTEGER,MPI_SUM, 0,MPI_COMM_WORLD, ierr)
        if ( iproc == 0 ) write(6,*)'Total No of primes=', ssum
        call MPI_FINALIZE(ierr)
      end program main
  
      subroutine para_range(n1, n2, nprocs, irank, ista, iend)
        integer(4) :: n1        ! Lowest value of iteration variable
        integer(4) :: n2        ! Highest value of iteration variable
        integer(4) :: nprocs    ! No of Cores/Processors you want to use
        integer(4) :: irank     ! Process rank
        integer(4) :: ista      ! Start of iterations for rank iproc
        integer(4) :: iend      ! End of iterations for rank iproc
        integer(4) :: iwork1, iwork2
  
        print *,irank
        iwork1 = ( n2 - n1 + 1 )  /  nprocs
        iwork2 = MOD(n2 - n1 + 1, nprocs)
        ista = irank * iwork1 + n1 + MIN(irank, iwork2)
        iend = ista + iwork1 - 1
        if ( iwork2 > irank ) then
             iend = iend + 1
        end if
      end subroutine para_range
  

I am using Open MPI.

Answer 1

This question is pretty much the same as your other question (Open MPI ranks are not in order) and the answer is the same. You're misunderstanding the problem.

The "ordering" of the ranks in your case can be seen as arbitrary and unimportant. The ranks that you assign the problem to will do the work you assign them. The problem you have is that you want them all to be printed out in sorted order. That is impossible in MPI. There is no guarantee that if you print messages from all of the ranks at exactly the same time, they will be printed in any particular order. The reason for this is that all of the output must first be sent to the process that launches the application mpiexec or mpirun and then printed to the screen. This transfer may be faster for some processes than others. If it's critical that you output all of the results in order, you must send them all to the same process first, then print them out from there. It is guaranteed that if you print text out all in the same rank, they will come out in the correct order.

So your code will look roughly like this:

...initialize variables...
...divide up work...
...each work does the work assigned to it...
...MPI_SEND result to rank 0...
if (rank == 0)
  MPI_RECV results from all processes
  print results in order