mass insert laravel for many records

Question

I'm trying to retrieve data from sql table A, modify some columns then insert the modified columns into sql table B.

However my issue is that when I use:

    $customer= new Customer;
    $fakecustomer= new Fakecustomer;

            $fake_customer_name_records = $fakecustomer->get()->toArray();

               foreach($fake_customer_name_records as $record){

            //process columns for each record

                 $fake_customer_name_records_arry[]=array(
                  'last_name'=>$last_name,
                  'first_name'=>$first_name,
                  'home_phone'=>$phonenumber,
                  );
}

            $customer->insert($fake_customer_name_records_arry);

It can only insert around 1000 records. Is there a way in Laravel for me to process about 60,000 records?

Thanks

Might be helpful if you tell us what happens when you try to process 60,000? — Jeemusu, Jul 04 '14 at 02:09

score 4 · Answer 1 · answered Jul 04 '14 at 07:20

I would suggest to use the "chunk" option here, and process records in "chunks". It's more native way, to my opinion. Here's what docs say :

Chunking Results

If you need to process a lot (thousands) of Eloquent records, using the chunk command will allow you to do without eating all of your RAM:
User::chunk(200, function($users)
{
    foreach ($users as $user)
    {
        //
    }
});
The first argument passed to the method is the number of records you wish to receive per "chunk". The Closure passed as the second argument will be called for each chunk that is pulled from the database.

Link to read more : click

Maybe you can help me. I want to implement chunk to my case. Look at this : https://stackoverflow.com/questions/51487769/how-to-insert-big-data-on-the-laravel — moses toh, Jul 24 '18 at 03:11

score 0 · Answer 2 · answered Jul 04 '14 at 04:17

0

Use an extra variable and sum 1 every iteration, when reaches 1000 (or lower) execute 'insert' and reset the counter.

answered Jul 04 '14 at 04:17

Ruke

141
5

score 0 · Answer 3 · answered Jul 04 '14 at 06:57

Have you tried disabling the query log with DB::disableQueryLog(); ? I had the same problem and this prety much solved it.

Also, when working with migrations or some process that's going to take a lot of time try to create a command instead of trying to do it with a controller.

mass insert laravel for many records

3 Answers3