Remove duplicates from PHP array (array_unique)

Question

I am getting a specific field from 4 different tables.

<?php 

//I connect to the database and the 4 tables

// Location of CSV  
$location = 'path.csv';

// List creation that will be updated with the fields and be put into my CSV file
$list = array();

// Read csv file to avoid adding duplicates
$file = fopen($location, 'r');
$data = array();

while($row = fgetcsv($file)) 
{
   $data[] = $row;
}

// Query 1
$sql = ('select distinct(field) as field from '.$table1.'');

// Run the query
$query = $Db->query($sql);

// Check for SQL errors
if ($Db->error) 
{
    return ($Db->error);
}

// Put data in the list
while ($row = $query->fetch_assoc()) 
{
    array_push($list,array($row['field'], ''));
}

// Query 2
$sql = ('select distinct(field) as field from '.$table2.'');

// Run the query
$query = $Db->query($sql);

// Check for SQL errors
if ($Db->error) 
{
    return ($Db->error);
}

// Put data in the list
while ($row = $query->fetch_assoc()) 
{
    array_push($list,array($row['field'], ''));
}

// Query 3
$sql = ('select distinct(field) as field from '.$table3.'');

// Run the query
$query = $Db->query($sql);

// Check for SQL errors
if ($Db->error) 
{
    return ($Db->error);
}

// Put data in the list
while ($row = $query->fetch_assoc()) 
{
    array_push($list,array($row['field'], ''));
}

// Query 4
$sql = ('select distinct(field) as field from '.$table4.'');

// Run the query
$query = $Db->query($sql);

// Check for SQL errors
if ($Db->error) 
{
    return ($Db->error);
}

// Put data in the list
while ($row = $query->fetch_assoc()) 
{
    array_push($list,array($row['field'], ''));
}


// Save list in the csv file without overwriting
$fp = fopen($location, 'a');

foreach (array_unique($list) as $fields) 
{
    if (in_array($fields, $data)) 
    {
        echo "Duplicate found";
    }
    else
    {
        echo "Save to file";
        fputcsv($fp, $fields);
    }           
}

fclose($fp);    

?>

In the end I check if the fields are already in the file. The only problem is that I still have duplicates because some tables might have exactly the same field. So, I want to remove the duplicates from the PHP array "list".

I am using :

$cleanlist = array_unique($list);

but I am getting an error:

PHP Notice: Array to string conversion

More specifically the change in my code is :

    $cleanlist = array_unique($list);

// Save list in the csv file without overwriting
$fp = fopen($location, 'a');

foreach ($cleanlist as $fields) 
{
    if (in_array($fields, $data)) 
    {
        echo "Duplicate found";
    }
    else
    {
        echo "Save to file";
        fputcsv($fp, $fields);
    }           
}

score 2 · Answer 1 · edited Jun 20 '20 at 09:12

2

As explain in the docs, array_unique compares elements as strings by default. You are getting this error because PHP is trying to convert an Array to string. You have a 2D array, an array of array.

You can use the flag SORT_REGULAR to compare the elements as they are. But be careful, only same key/value pairs are considered identicals.

edited Jun 20 '20 at 09:12

Community

1
1

answered Jan 06 '16 at 16:05

Jérémy Halin

553
4
29

score 1 · Accepted Answer · answered Jan 06 '16 at 16:35

1

You could reduce amount of code a lot by using UNION in SELECT statement.

SELECT field FROM table1
UNION
SELECT field FROM table2
UNION
SELECT field FROM table3
UNION
SELECT field FROM table4

UNION returns distinct results by default.

answered Jan 06 '16 at 16:35

Naktibalda

13,705
5
35
51

Great solution. It saves loads of lines! I posted an answer too but your is better. Mine is just a way to use array_unique for a n-dimension table. – Datacrawler Jan 06 '16 at 16:42

score 0 · Answer 3 · answered Jan 06 '16 at 16:11

0

That worked :

$list = array_map("unserialize", array_unique(array_map("serialize", $list)));

The $list is a 2d array.

answered Jan 06 '16 at 16:11

Datacrawler

2,780
8
46
100

Remove duplicates from PHP array (array_unique)

3 Answers3