Stack Overflow
Stack Overflow

Questions tagged [mpi-io]

MPI/IO provides high performance, portable, parallel I/O interface to high performance, portable, parallel MPI programs

The purpose of MPI/IO is to provide high performance, portable, parallel I/O interface to high performance, portable, parallel MPI programs. Parallel I/O is not a daily bread. Although some supercomputer systems in the past offered parallel disk subsystems, e.g., the Connection Machine CM5 had a Scalable Disk Array, SDA, the Connection Machine CM2 had the Data Vault and IBM SP had PIOFS and today it has GPFS, communication with those peripherals was architecture and operating system dependent.

See the MPI/IO site for more.

64 questions
0
votes
1 answer

Avoid repeated MPI_File_set_view in a datafile with a header

I am modernizing a simple MPI-IO subroutine from https://github.com/LadaF/PoisFFT/blob/master/src/testmpi.f90 (save_vtk() at the very end) somewhat in the spirit of MPI I/O, mix of single- and multiple-process output. My intention is to use this…
Vladimir F Героям слава
  • 57,977
  • 4
  • 76
  • 119
0
votes
1 answer

MPI_File_read_at_all giving invalid count argument when trying to read big files

I wrote a simple C program to try out MPI-IO; The program reads a text file and each process outputs the first character of the part it read. The program works fine with different sizes (100KB, 30MB, 500MB, 2.5GB) but when I tried a 7.5GB file, I…
Shazly
  • 95
  • 1
  • 1
  • 11
0
votes
1 answer

Am I misusing mpi_file_write_all?

I have a snippet of code writing data in a file with MPI-IO. It works well when I am using mpi_file_write, but if I switch to the collective mpi_file_write_all I get the wrong result. Do I have to change more than just the call to the write…
Ray
  • 787
  • 6
  • 9
0
votes
0 answers

How to write multiple variables to a file using MPI I/O using Fortran

The below code gives me 4 variables saving 4 different rows with 10 colums, whereas I need to save like 4 columns in 10 rows. I'm using hexdump syntax to extract from the file program main use mpi integer :: wsize,wrank,ierr,i,fh,offset integer ,…
Mallikarjun
  • 1
0
votes
1 answer

Incorrect results when reading binary file with MPI I/O

I am new to MPI and am struggling with reading a binary file. Specifically, I have a $198\times 50 \times 50$ array of integers (16 bit integers, to be specific) stored in a binary file. I want to use 2 compute nodes to process this file. So there…
rishu saxena
  • 83
  • 5
0
votes
0 answers

How to read binary data written using MPI-IO in Fortran?

I want to read out some output data from a Fortran code to data files for postprocessing. The code given to me uses MPI to write to those data files. For the most basic case, when the number of processors should be 1 and also the Nx, Ny, Nz are all…
han_eel
  • 1
0
votes
0 answers

Reading data from files using MPI in Fortran

I want to read data from some .dat files to a Fortran code for postprocessing. As a test case, I am just using one processor for MPI and trying to read single data file to my code. The content of the data file is as follows: qout0050.dat : 1 1…
han_eel
  • 1
0
votes
3 answers

Writing to multiple shared files with MPI-IO

I'm running a simulation with thousands of MPI processes and need to write output data to a small set of files. For example, even though I might have 10,000 processes I only want to write out 10 files, with 1,000 writing to each one (at some…
sevko
  • 1,402
  • 1
  • 18
  • 37
0
votes
2 answers

Reading text files using MPI-IO?

I have a text file which contains a header with the dimensions of a matrix, and then the matrix. Here's an example for a 3x3 matrix: 3 3 1 56 8 12 3 0 9 44 81 I kept getting garbage values with MPI-IO, and found out it only works with binary, not…
Moody
  • 1,297
  • 2
  • 12
  • 21
0
votes
1 answer

MPI_File_read_at line by line

I'm a new learner of MPI. I'd like to use MPI_File_read_at() to read the data from a txt file line by line. The length of each line is different, so when I am reading one line (set a length of buffer), sometimes it will also read the next line, send…
Ziwei
  • 57
  • 1
  • 3
  • 11
0
votes
2 answers

MPI_FILE_READ && little endian on Bluegene

I need to read (and write) some binary little endian file. I am writing my fortran code on a PC using Intel FC and Intel MPI. I/O works fine on PC, but final cause is running the program on Bluegene/P. The Bluegene/P(XL Fortran Compiler) has big…
Vasiliy
  • 61
  • 8
0
votes
2 answers

MPI collective output 5 noncontiguous 3D arrays in special form

During the realization of the course work I have to write MPI program to solve PDE continuum mechanics. (FORTRAN) In the sequence program file is written as follows: do i=1,XX do j=1,YY do k=1,ZZ write(ifile) R(i,j,k) …
Vasiliy
  • 61
  • 8
0
votes
2 answers

MPI read and write to HDFS

Does anyone know a good way to read/write files to my hdfs from within MPI? I've done a fair amount of digging trying to figure this out, and just need a general direction to pursue.
Kyle.
  • 156
  • 8
0
votes
2 answers

MPI IO Writing a file when offset is not known

I want to use MPI IO to write files. The processes is in a while loop and it calls a function which generates random amount of data. I want to write this data to a single file. How can I accomplish this?
vishesh
  • 113
  • 1
  • 7
0
votes
1 answer

MPI-IO File transfer between compute nodes

How to transfer files from localscrach of one compute to localscratch of another compute node in a cluster using MPI-IO?
saky
  • 1
1 2 3
4
5