remove duplicates from array (array unic by key)

Question

Array
(
    [0] => Array
        (
            [file] => /var/websites/example.com/assets/images/200px/1419050406e6648e1c766551a0ffc91380fd6ff3406002011-10-233750.jpg
            [md5] => 42479bee7a304d2318250de2ef1962a9
            [url] => http://example.com/assets/images/200px/1419050406e6648e1c766551a0ffc91380fd6ff3406002011-10-233750.jpg
        )

    [1] => Array
        (
            [file] => /var/websites/example.com/assets/images/200px/21277792606e6648e1c766551a0ffc91380fd6ff3406002011-10-233750.jpg
            [md5] => 42479bee7a304d2318250de2ef1962a9
            [url] => http://example.com/assets/images/200px/21277792606e6648e1c766551a0ffc91380fd6ff3406002011-10-233750.jpg
        )
)

How can I remove md5 key duplicates from above array?

Answer 1

<?php
$data = array(
  array(
    'md5' => 'alpha',
    'some' => 'value',
  ),
  array(
    'md5' => 'alpha',
    'some' => 'other value',
  ),
  array(
    'md5' => 'bravo',
    'some' => 'third value',
  ),
);
// walk input array
$_data = array();
foreach ($data as $v) {
  if (isset($_data[$v['md5']])) {
    // found duplicate
    continue;
  }
  // remember unique item
  $_data[$v['md5']] = $v;
}
// if you need a zero-based array, otheriwse work with $_data
$data = array_values($_data);

Answer 2

Here is my final function after your help guys.... Hope it helps somebody in the future...

    $data = array(
          array(
            'md5' => 'alpha',
            'some' => 'value',
          ),
          array(
            'md5' => 'alpha',
            'some' => 'other value',
          ),
          array(
            'md5' => 'bravo',
            'some' => 'third value',
          ),
        );
        // walk input array


        function remove_duplicateKeys($key,$data){

        $_data = array();

        foreach ($data as $v) {
          if (isset($_data[$v[$key]])) {
            // found duplicate
            continue;
          }
          // remember unique item
          $_data[$v[$key]] = $v;
        }
        // if you need a zero-based array
        // otherwise work with $_data
        $data = array_values($_data);
        return $data;
        }

$my_array = remove_duplicateKeys("md5",$data);

Answer 3

PHP does already have a function to remove duplicate elements from an array. But unfortunately, array_unique does only support string based comparisons:

Note: Two elements are considered equal if and only if (string) $elem1 === (string) $elem2. In words: when the string representation is the same. The first element will be used.

The problem is that any array turned into a string is equal to any other array:

Arrays are always converted to the string "Array"; […]

But you can use the uniqueness of array keys to solve this:

$index = array();
foreach ($arr as $key => $item) {
    if (isset($index[$item['md5']])) {
        unset($item[$key]);
    }
    $index[$item['md5']] = TRUE;
}

Answer 4

As array_unique operates on flat arrays, you can not use it directly. But you can first map all 'md5' values to a flat array, unique it and then get the elements with array_intersect_key:

$allMd5s = array_map(function($v) {return $v['md5'];}, $array);

$uniqueMd5s = array_unique($md5);

$result = array_intersect_key($arr, $uniqueMd5s);

Answer 5

Tldr.: The obligatory one-liner:

$out = array_values ( array_intersect_key( $in, array_unique( array_column( $in, 'md5' ) ) ) );

And step by step (with my own example):

$in = [
    [ 'k' => 'a' ],
    [ 'k' => 'a' ],
    [ 'k' => 'b' ]
];

print_r( array_column( $in, 'k' ) );

array_column returns a flat array, only with the values from all 'k'-keys:

Array
(
    [0] => a
    [1] => a
    [2] => b
)

We can now filter that with array_unique:

print_r( array_unique( array_column( $in, 'k' ) ) ) );

To get:

Array
(
    [0] => a
    [2] => b
)

See that we kept the keys of the original array (0 and 2)? We can use that with array_intersect_key. It computes the intersection of arrays using keys for comparison.

print_r( array_intersect_key( $in, array_unique( array_column( $in, 'k' ) ) ) );

We get the first [0] and third [2] entry of our original array.

Array
(
    [0] => Array
        (
            [k] => a
        )

    [2] => Array
        (
            [k] => b
        )

)

Now the keys are messed up (which might lead to problems if for loops), so we wrap the whole thing in array_values:

print_r(
  array_values( array_intersect_key( $in, array_unique( array_column( $in, 'k' ) ) ) ) 
);

Array
(
    [0] => Array
        (
            [k] => a
        )

    [1] => Array
        (
            [k] => b
        )

)

Answer 6

Use array_filter(). Quick code test (doesn't necessarily mirror your situation, but you should get the idea):

<?php
header('Content-Type: Text/Plain');
$array = array(
    0 => array('name' => 'samson'),
    1 => array('name' => 'delilah'),
    2 => array('name' => 'samson'),
    3 => array('name' => 'goliath'),
);

$md5Processed = array();

$newArray = array_filter($array, "uniqueMD5");

print_r($newArray);

exit;

function uniqueMD5( $data ){
    global $md5Processed;

    if( !in_array( $data['name'], $md5Processed ) )
    {
        $md5Processed[] = $data['name'];
        return true;
    }
}

Answer 7

// your array
$array = array(...);
// will be used to store md5 hashes
$md5 = array();
// walk through array
foreach ($array as $key => $arr) {
  // have we already seen this md5 hash?
  if (in_array($arr['md5'], $md5)){
    // remove element
    unset($array[$key]);
  }else {
    // keep element, but add it's md5
    $md5[] = $arr['md5'];
  }
}

Answer 8

Take a look at the function array_unique.