Writing in a file make my program load really slowly

Question

I am trying to write csv files with fwrite/fputcsv in a loop but when I use those functions, they write in the first file and then the tab loads infinitely and doesn't do anything else.
I know those functions are the problem because when I remove them the rest of the code runs properly.

Here's my code :

<?php

$auth = '';
$date = getdate(); //Needed for the Trade lists files

$path = array(
    'Some_folder/file1.csv',
    'Some_folder/file2.csv',
    'Some_folder/file3.csv',
    'Some_folder/file4.csv'
);//Full path the files need to be in
$file = array(
    pathinfo($path[0])['basename'],
    pathinfo($path[1])['basename'],
    pathinfo($path[2])['basename'],
    pathinfo($path[3])['basename']
);//The names the files will have
$fp = array(
    fopen($file[0], 'w+'), 
    fopen($file[1], 'w+'),
    fopen($file[2], 'w+'),
    fopen($file[3], 'w+')
);// Files that are create by fopen

for($i = 0; $i < 4; $i++) {
    fputcsv($fp[$i], $file);
    echo 'test';
    $size = filesize($path[$i]);
    $cheaders = array('Authorization: Bearer ' .$auth.'[Private access token]',
    'Content-Type: application/octet-stream',
    'Dropbox-API-Arg: {"path":"/'.$file[$i].'", "mode":"add"}');

    $ch = curl_init('[Private information]');
    curl_setopt($ch, CURLOPT_HTTPHEADER, $cheaders);
    curl_setopt($ch, CURLOPT_PUT, true);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
    curl_setopt($ch, CURLOPT_INFILE, $fp[$i]);
    curl_setopt($ch, CURLOPT_INFILESIZE, $size);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $response = curl_exec($ch);

    curl_close($ch);
    fclose($fp[$i]);
}

?>

fputcsv and fwrite do write in the first file (file1.csv) but they make the program stuck in an infinite loading. The console doesn't say anything since I can't load the page and I don't know what else to try/do. I tried looking for a solution, unfortunately I didn't find it so here I am. I'll still look for an awnser but if someone has a solution, I'll take it too.

Thanks to all those who will try to help.

EDIT: It finally loaded but it took a really long time since it started loading approximately an hour ago. Is there something I did wrong for my program to be so slow to write so slowly in a file ?

oo just notice, your `$file` variable is not taking the absolute path of each file, you are taking only file's basename in the `$fp` handler — Al-Amin, Oct 26 '20 at 12:09
That's because my fopen creates the files in the current directory. The problem is, if I only write files, it takes less than 10s, if I only upload files same thing, but when I combine writing and uploading my program gets really slow. It takes approximately an hour to make it run. — R-son, Oct 26 '20 at 12:15
Try debugging the code to see what's consuming time. I think it's the API you are using is responding slowly while uploading file. Also try cURL to upload a single file manually to see whether the api is taking time to transfer file — Al-Amin, Oct 26 '20 at 12:36
The upload is consumming time but only if the writing part is here too. If I manually enter the data in the file, the upload is fast. — R-son, Oct 26 '20 at 12:57
is it the dropbox uploading api taking time or your server's upload in the first time? — Al-Amin, Oct 26 '20 at 13:32
I think this line is the problem : ```curl_setopt($ch, CURLOPT_INFILESIZE, $size);``` The program is faster when I remove it. But I know I need this line because the file will be empty if I upload it without this line — R-son, Oct 26 '20 at 14:20
split the task, first save the csv file in your server and then again use another loop for cURL to upload the files in the dropbox. echo something before using the second loop to get confirm the files are uploaded in your server. — Al-Amin, Oct 26 '20 at 14:21
`CURLOPT_INFILESIZE` is not associated with the problem , i think — Al-Amin, Oct 26 '20 at 14:24
Like you suggested, I spilted the script to split the task and it worked. The program is fast again now and the file contain the infos I wanted. Thank you. You should post it as an awnser so I can approve it and people who have the same problem could find a solution easily — R-son, Oct 26 '20 at 14:36

Al-Amin · Accepted Answer · 2020-10-26T14:48:09.860

I would suggest you to split the task in two sections. First write the files in your server using fopen in w mode. After writings, use another loop for reading the files using fopen in r mode for using in your cURL to upload in the server. Using the same handler for writing and uploading might cause the lagging problem.

You can use curl_multi_init for handling multiple cURL requests asynchronously. Here is also a post on it.

Writing in a file make my program load really slowly

1 Answers1