MPI Scatter / Send data to processes

Question

I am running my code using n = 2. I have been trying to research, but I do not get why I obtain different outputs when I try to run my code several times. Under you can see my output when I run with n = 2, and my txt.file contains 73 characters in each row, and consist of 4 rows.

This is my output the first time and run and is the expected and wanted output

This is the output the second time I run the code

This is the output the third time I run the code

I do not know what to do to prevent the errors happening in image 2 (where you have a \274 in the second printed line) and but specially with errors I get the third time running the code. Do i have to use MPI_Allocate? Am I freeing the matrix to early?

This is my code:

#define MAXCHAR 73
#define MAXLENGTH 100

int main(int argc, char** argv) {
FILE *fp;
char* filename = "/Users/test.txt";
char *line = malloc(MAXCHAR);
char (*matA)[MAXCHAR] = NULL;
char str[MAXCHAR];
int rowCount, num_rows, i, my_id, 
root_process,num_procs,rows_per_process;

MPI_Init(&argc, &argv);
root_process = 0;



/* find out MY process ID, and how many processes were started. */
MPI_Comm_rank(MPI_COMM_WORLD, &my_id);
MPI_Comm_size(MPI_COMM_WORLD, &num_procs);


// READ TXT FILE INTO DYNAMIC ARRAY
if(my_id == root_process){
    fp = fopen(filename, "r");
    if (fp == NULL){
        printf("Could not open file %s",filename);
        return 1;
    }
    //NUMBER OF lines
    size_t count=1000;
    rowCount=0;
    while(getline(&line, &count, fp)!=-1) {
        rowCount++;
    }
    //REWIND file
    rewind(fp);

    matA = malloc(rowCount*sizeof(matA));
    i = 0;
    while (fgets(str, MAXCHAR, fp) != NULL){
        for (size_t j = 0; j < MAXCHAR; j++) {
            if(str[j] == '\n'){
                continue;
            }
            matA[i][j] = str[j];
        }
        i++;
        num_rows = i;
    }
    fclose(fp);

}

// BCAST rowCount to Calculate rows each process will receive
MPI_Bcast(&rowCount, 1, MPI_INT, 0, MPI_COMM_WORLD);
rows_per_process = rowCount/num_procs;



char(*part_matrix)[MAXCHAR];
part_matrix = malloc(rows_per_process*sizeof(*part_matrix));


MPI_Scatter(&(matA[0][0]), rows_per_process*73, MPI_CHAR, &(part_matrix[0][0]), rows_per_process*73, MPI_CHAR, 0, MPI_COMM_WORLD);
printf("Process %i received %i rows:\n", my_id, rows_per_process);

// PRINTING
for (size_t i = 0; i < rows_per_process; i++) {
    printf("PROCESS %i PRINTS LINE NUMBER %zu:\n", my_id, i);
    for (size_t j = 0; j < MAXCHAR; j++) {
        printf("%c", part_matrix[i][j]);
    }
    printf("\n" );
}

free(part_matrix);
MPI_Finalize();
return 0;
}

Please format your code consistently, it is much harder to read than necessary. Also make sure to provide a [mcve] that actually compiles including input data and compilation / execution commands. — Zulan, Oct 25 '18 at 05:29
why do you `strcpy()` in a for loop ? Why is there a `MPI_Irecv()` without a `MPI_Send()` ? and why is it followed by a `MPI_Test()` ? It seems you are just throwing some code hoping a good samaritan will debug it for you, but unfortunately, this is **not** how SO works. — Gilles Gouaillardet, Oct 25 '18 at 05:32
you can observe such issues running in singleton mode under a memory debugger (for example `valgrind a.out`), and then fix how you populate `matA` and print `part_matrix`. — Gilles Gouaillardet, Oct 26 '18 at 02:04

score 0 · Accepted Answer · answered Oct 31 '18 at 12:21

I believe most of your problem comes from the fact that

char (*matA)[MAXCHAR] = NULL;
matA = malloc(rowCount*sizeof(matA));

Is a strange way of allocating your data.

char *matA = NULL;
matA = malloc(rowCount*MAXCHAR*sizeof(char));

seems more reasonable

same goes for

char *part_matrix=NULL;
part_matrix = (char*) malloc(rows_per_process*MAXCHAR*sizeof(char));

That way you are sure that matA and part_matrix is contiguous in memory and no harm will come (memory wise) when using MPI functions.

MPI_Scatter simply becomes MPI_Scatter(matA, rows_per_process*MAXCHAR, MPI_CHAR, part_matrix, rows_per_process*MAXCHAR, MPI_CHAR, 0, MPI_COMM_WORLD);

I changed the way your read your file in order to make in work...

matA = (char*)malloc(rowCount*MAXCHAR*sizeof(char));
i = 0;
while (getline(&line, &count, fp)!=-1){
    for (j = 0; j < MAXCHAR; j++) {
        matA[i*MAXCHAR+j] = line[j];
    }
    i++;
    num_rows = i;
}

And so you got your full working example with test.txt being

AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA  
BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB  
CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC  
DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD

main file

#include <stdlib.h>
#include <stdio.h>
#include "mpi.h"
#define MAXCHAR 73
#define MAXLENGTH 100

int main(int argc, char** argv) {
FILE *fp;
char* filename = "test.txt";
char *line = (char*) malloc(MAXLENGTH);
char * matA= NULL;
char str[MAXCHAR];
int rowCount, num_rows, i, my_id, 
root_process,num_procs,rows_per_process, j;

MPI_Init(&argc, &argv);
root_process = 0;



/* find out MY process ID, and how many processes were started. */
MPI_Comm_rank(MPI_COMM_WORLD, &my_id);
MPI_Comm_size(MPI_COMM_WORLD, &num_procs);

// READ TXT FILE INTO DYNAMIC ARRAY
if(my_id == root_process){
    fp = fopen(filename, "r");
    if (fp == NULL){
        printf("Could not open file %s",filename);
        return 1;
    }
    //NUMBER OF lines
    size_t count=1000;
    rowCount=0;
    while(getline(&line, &count, fp)!=-1) {
        rowCount++;
    }
    //REWIND file
    rewind(fp);

    matA = (char*)malloc(rowCount*MAXCHAR*sizeof(char));
    i = 0;
    while (getline(&line, &count, fp)!=-1){
        for (j = 0; j < MAXCHAR; j++) {
            matA[i*MAXCHAR+j] = line[j];
        }
        i++;
        num_rows = i;
    }
    fclose(fp);
}

// BCAST rowCount to Calculate rows each process will receive
MPI_Bcast(&rowCount, 1, MPI_INT, 0, MPI_COMM_WORLD);
rows_per_process = rowCount/num_procs;


char *part_matrix = NULL;
part_matrix = (char*) malloc(rows_per_process*MAXCHAR*sizeof(char));


MPI_Scatter(matA, rows_per_process*MAXCHAR, MPI_CHAR, part_matrix, rows_per_process*MAXCHAR, MPI_CHAR, 0, MPI_COMM_WORLD);

if(my_id == root_process){
free(matA);
}

printf("Process %i received %i rows:\n", my_id, rows_per_process);
// PRINTING
for (i = 0; i < rows_per_process; i++) {
    printf("PROCESS %i PRINTS LINE NUMBER %zu:\n", my_id, i);
    for (j = 0; j < MAXCHAR; j++) {
        printf("%c", part_matrix[i*MAXCHAR+j]);
    }
    printf("\n" );
}

free(part_matrix);
MPI_Finalize();
return 0;
}

mpirun -n 4 returns

PROCESS 0 PRINTS LINE NUMBER 0:
AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
Process 1 received 1 rows:
PROCESS 1 PRINTS LINE NUMBER 0:
BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB
Process 2 received 1 rows:
PROCESS 2 PRINTS LINE NUMBER 0:
CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC
Process 3 received 1 rows:
PROCESS 3 PRINTS LINE NUMBER 0:
DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD

mpirun -n 2 returns

Process 0 received 2 rows:
PROCESS 0 PRINTS LINE NUMBER 0:
AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
PROCESS 0 PRINTS LINE NUMBER 1:
BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB BBBBBBBB
Process 1 received 2 rows:
PROCESS 1 PRINTS LINE NUMBER 0:
CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC
PROCESS 1 PRINTS LINE NUMBER 1:
DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD DDDDDDDD

thank you! my first time using c, so still have a lot to learn. — MeryT, Nov 03 '18 at 11:24
No worries. However Upvoting and accepting the answer is always nice. — PilouPili, Nov 04 '18 at 01:05

MPI Scatter / Send data to processes

1 Answers1