Some content of the binary file missing after receiving from client

Question

[enter image description here][1]I have searched for the solution but none of those worked for me, can someone help me in understanding where I might have done wrong or something which I have misunderstood and not handled properly in my code.

For example the size of the file supposed to be received was 3656713 and the actual size received on the server is 3656464.

In my code BUFFER_SIZE is a macro and it is set to 4096, I think it does not matter because I have tried with different numbers like 256, 1024 etc,

I have attached both client and server code.

This is client which is sending media file:

    while(1){
        //char buff[BUFFER_SIZE]={0};
        memset(buff, 0, BUFFER_SIZE);
        while(!feof(fp)){
            int nread = fread(buff, 1, BUFFER_SIZE, fp);
            __android_log_print(ANDROID_LOG_INFO, "Native", "%d nread %d\n", ++i, nread);
            if(nread == 0){
                __android_log_print(ANDROID_LOG_INFO, "Native", "Something went wrong while reading the file");
                break;
            }
            if(nread > 0){
                int totalWritten = 0;
                do {
                    int actualWritten;
                    actualWritten = write (sockfd, buff + totalWritten, nread - totalWritten);
                    //actualWritten = send (sockfd, buff + totalWritten, nread - totalWritten, 0);
                    if( actualWritten == - 1 ) {
                        __android_log_print(ANDROID_LOG_INFO, "Native", "Unable to write to socket \n");
                        finalStatus = false;
                        break;
                    }
                    totalWritten += actualWritten;
                } while( totalWritten < nread );
            }
            memset(buff, 0, BUFFER_SIZE);
            //usleep(2);
        }
        if(feof(fp)){
            __android_log_print(ANDROID_LOG_INFO, "Native", "End of the file reached");
            EOFReached = true;
            fclose(fp);
        }
        if(ferror(fp)){
            fclose(fp);
            __android_log_print(ANDROID_LOG_INFO, "Native", "Something went wrong while reading the file content");
        }
        break;
    }

This is server which is receiving the media file:

 buffer = (char *)malloc(BUFFER_SIZE);
 while((bytesReceived = read(sockfd, buffer, BUFFER_SIZE)) > 0){
     receivedBytes = receivedBytes + bytesReceived;
     printf("%d : Actual size : %d Received size : %d\n", ++i, actualSizeOfTheFile, receivedBytes);
     if(strncmp(buffer, "MYNEWSTRANSFERCOMPLETE", 22) == 0){
         fclose(fp);
         printf("Closed the file as server received the command : %s\n", buffer);
         break;
     }else{
         //printf("%d Bytes recived %d and the data is %s\n", ++i, bytesReceived, buffer);
         fwrite(buffer, 1, bytesReceived, fp);
     }
     memset(buffer, '\0', BUFFER_SIZE);
 }
 printf("Final string received from client is %s\n", buffer);
 memset(buffer, '\0', BUFFER_SIZE);
 printf("Total bytes received from client is :::::%d", receivedBytes);
 if((bytesReceived = read(sockfd, buffer, BUFFER_SIZE)) > 0){
     printf("Final string received from client is %s\n", buffer);
     if((write(sockfd, "You can close the socket", 24)) > 0){
         printf("Acknowledging the client to confirming for closing the connection\n");
     }else{
         printf("Unable to acknowledge the client on close confirmation of the connection\n");
     }
 }else{
     printf("Socket might have been closed by the client\n\n");
 }

My problem is I am trying to transfer media file from Android client to Linux server over sockets using 'C'. Earlier when I started with simple file write and read the media file received and played without any issues.

But all of sudden I see some data being missed in the final received file on the Linux server.

As I am unable to add the screen shots pasting the log statements here`

Client Log

05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: 889 nread 4096 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: totalWritten : 4096 out of nread 4096 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: 890 nread 4096 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: totalWritten : 4096 out of nread 4096 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: 891 nread 4096 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: totalWritten : 4096 out of nread 4096 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: 892 nread 4096 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: totalWritten : 4096 out of nread 4096 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: 893 nread 3081 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: totalWritten : 3081 out of nread 3081 05-15 18:59:08.072 30084-30084/com.example.user.resumablesample I/Native: End of the file reached 05-15 19:04:22.387 30084-30094/com.example.user.resumablesample W/art: Suspending all threads took: 7.375ms

Server Log

2513 : Actual size : 3656713 Received size : 3644423 Received bytes : 1408 2514 : Actual size : 3656713 Received size : 3645831 Received bytes : 1408 2515 : Actual size : 3656713 Received size : 3647239 Received bytes : 1408 2516 : Actual size : 3656713 Received size : 3648647 Received bytes : 1408 2517 : Actual size : 3656713 Received size : 3650055 Received bytes : 1408 2518 : Actual size : 3656713 Received size : 3651463 Received bytes : 1408 2519 : Actual size : 3656713 Received size : 3652871 Received bytes : 1408 2520 : Actual size : 3656713 Received size : 3654279 Received bytes : 1408 2521 : Actual size : 3656713 Received size : 3655687 Received bytes : 1408 2522 : Actual size : 3656713 Received size : 3656464 Received bytes : 777

File listing on the sever -rw-r--r-- 1 magic_ramana magic_ramana 3653632 May 15 13:29 mynews_4zUIgxb5nh1494854941443.mp4 ` I observer that the last chunk 3081Bytes which client log says has been transferred, but the same amount of data is missing in the final file which I have listed. Final file size is 3653632, but that suppose to be 3656713 which is 3081 bytes more that suppose to be transferred in the last chunk

Answer 1

if(strncmp(buffer, "MYNEWSTRANSFERCOMPLETE", 22) == 0){

The problem is here. You cannot rely on receiving this as the first thing and the only thing in the buffer. TCP is a streaming protocol and can deliver the data to the receiving application any way it likes, including one byte at a time, or coalescing buffers, or anything in between, as long as it delivers all the data exactly once and in the right order.

I will also note that you aren't actually sending this string anywhere.

Similarly

if((write(sockfd, "You can close the socket", 24)) > 0){

and all such cannot be guaranteed to be received discretely. I will also note that you aren't actually receiving this data anywhere.

You will either have to transmit the file size ahead of the file, or close the socket after you've sent the file without using any kind of application protocol.

Answer 2

If you have not set the SO_LINGER socket option, closing the socket will discard queued data which has not been sent. I suspect that is what is happening here.

Answer 3

Notice the following code excerpts do not depend on some 'adhoc' text being transmitted by the client to the server to indicate the transfer is complete.

here are the needed corrections for the client code

bool finalStatus = true;
size_t nread;
while( 0 < (nread = fread(buff, 1, BUFFER_SIZE, fp) ) )
    __android_log_print(ANDROID_LOG_INFO, "Native", "%d nread %d\n", ++i, nread);        

    int totalWritten = 0;
    do
    {
        ssize_t actualWritten;
        actualWritten = write (sockfd, buff + totalWritten, (ssize_t)nread - totalWritten);
        //actualWritten = send (sockfd, buff + totalWritten, nread - totalWritten, 0);
        if( actualWritten == - 1 )
        {
            __android_log_print(ANDROID_LOG_INFO, "Native", "Unable to write to socket \n");
            finalStatus = false;
            break;
        }
        totalWritten += actualWritten;
    } while( totalWritten < nread && finalStatus);
}

if( !nread )
{
    if(feof(fp))
    {
        __android_log_print(ANDROID_LOG_INFO, "Native", "End of the file reached");
        EOFReached = true;;
    }

    else if(ferror(fp))
    {
        __android_log_print(ANDROID_LOG_INFO, "Native", "Something went wrong while reading the file content");
    }
}

fclose(fp);

here are the needed corrections for the server code

buffer = malloc(BUFFER_SIZE);
if( !buffer )
{
    perror( "malloc failed" );
    close( sockfd );
    exit( EXIT_FAILURE );
}

// implied else, malloc successful

int i = 0;
ssize_t receivedBytes = 0;
ssize_t bytesReceived;
while((bytesReceived = read(sockfd, buffer, BUFFER_SIZE)) > 0)
{
    receivedBytes = receivedBytes + bytesReceived;
    printf("%d : Actual size : %ld Received size : %ld\n", ++i, actualSizeOfTheFile, receivedBytes);
}

if( !bytesReceived )
{
    printf( "EOF reached\n" );
}

else if( 0 > bytesReceived )
{
    perror( "read failed" );
}

close( sockfd );

printf("Total bytes received from client is :::::%ld", receivedBytes);