PHP unserialize fails with non-encoded characters?

Question

$ser = 'a:2:{i:0;s:5:"héllö";i:1;s:5:"wörld";}'; // fails
$ser2 = 'a:2:{i:0;s:5:"hello";i:1;s:5:"world";}'; // works
$out = unserialize($ser);
$out2 = unserialize($ser2);
print_r($out);
print_r($out2);
echo "<hr>";

But why?
Should I encode before serialzing than? How?

I am using Javascript to write the serialized string to a hidden field, than PHP's $_POST
In JS I have something like:

function writeImgData() {
    var caption_arr = new Array();
    $('.album img').each(function(index) {
         caption_arr.push($(this).attr('alt'));
    });
    $("#hidden-field").attr("value", serializeArray(caption_arr));
};

Answer 1

The reason why unserialize() fails with:

$ser = 'a:2:{i:0;s:5:"héllö";i:1;s:5:"wörld";}';

Is because the length for héllö and wörld are wrong, since PHP doesn't correctly handle multi-byte strings natively:

echo strlen('héllö'); // 7
echo strlen('wörld'); // 6

However if you try to unserialize() the following correct string:

$ser = 'a:2:{i:0;s:7:"héllö";i:1;s:6:"wörld";}';

echo '<pre>';
print_r(unserialize($ser));
echo '</pre>';

It works:

Array
(
    [0] => héllö
    [1] => wörld
)

If you use PHP serialize() it should correctly compute the lengths of multi-byte string indexes.

On the other hand, if you want to work with serialized data in multiple (programming) languages you should forget it and move to something like JSON, which is way more standardized.

Answer 2

I know this was posted like one year ago, but I just have this issue and come across this, and in fact I found a solution for it. This piece of code works like charm!

The idea behind is easy. It's just helping you by recalculating the length of the multibyte strings as posted by @Alix above.

A few modifications should suits your code:

/**
 * Mulit-byte Unserialize
 *
 * UTF-8 will screw up a serialized string
 *
 * @access private
 * @param string
 * @return string
 */
function mb_unserialize($string) {
    $string = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $string);
    return unserialize($string);
}

Source: http://snippets.dzone.com/posts/show/6592

Tested on my machine, and it works like charm!!

Answer 3

Lionel Chan answer modified to work with PHP >= 5.5 :

function mb_unserialize($string) {
    $string2 = preg_replace_callback(
        '!s:(\d+):"(.*?)";!s',
        function($m){
            $len = strlen($m[2]);
            $result = "s:$len:\"{$m[2]}\";";
            return $result;

        },
        $string);
    return unserialize($string2);
}    

This code uses preg_replace_callback as preg_replace with the /e modifier is obsolete since PHP 5.5.

Answer 4

The issue is - as pointed out by Alix - related to encoding.

Until PHP 5.4 the internal encoding for PHP was ISO-8859-1, this encoding uses a single byte for some characters that in unicode are multibyte. The result is that multibyte values serialized on UTF-8 system will not be readable on ISO-8859-1 systems.

The avoid problems like this make sure all systems use the same encoding:

mb_internal_encoding('utf-8');
$arr = array('foo' => 'bár');
$buf = serialize($arr);

You can use utf8_(encode|decode) to cleanup:

// Set system encoding to iso-8859-1
mb_internal_encoding('iso-8859-1');
$arr = unserialize(utf8_encode($serialized));
print_r($arr);

Answer 5

In reply to @Lionel above, in fact the function mb_unserialize() as you proposed won't work if the serialized string itself contains char sequence "; (quote followed by semicolon). Use with caution. For example:

$test = 'test";string'; 
// $test is now 's:12:"test";string";'
$string = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $test);
print $string; 
// output: s:4:"test";string";  (Wrong!!)

JSON is the ways to go, as mentioned by others, IMHO

Note: I post this as new answer as I don't know how to reply directly (new here).

Answer 6

Do not use PHP serialization/unserialization when the other end is not PHP. It is not meant to be a portable format - for example, it even includes ascii-1 characters for protected keys which is nothing you want to deal with in javascript (even though it would work perfectly fine, it's just extremely ugly).

Instead, use a portable format like JSON. XML would do the job, too, but JSON has less overhead and is more programmer-friendly as you can easily parse it into a simple data structure instead of having to deal with XPath, DOM trees etc.

Answer 7

This solution worked for me:

$unserialized = unserialize(utf8_encode($st));

Answer 8

One more slight variation here which will hopefully help someone ... I was serializing an array then writing it to a database. On retrieving the data the unserialize operation was failing.

It turns out that the database longtext field I was writing into was using latin1 not UTF8. When I switched it round everything worked as planned.

Thanks to all above who mentioned character encoding and got me on the right track!

Answer 9

/**
 * MULIT-BYTE UNSERIALIZE
 *
 * UTF-8 will screw up a serialized string
 *
 * @param string
 * @return string
 */
function mb_unserialize($string) {
    $string = preg_replace_callback('/!s:(\d+):"(.*?)";!se/', function($matches) { return 's:'.strlen($matches[1]).':"'.$matches[1].'";'; }, $string);
    return unserialize($string);
}

Answer 10

I would advise you to use javascript to encode as json and then use json_decode to unserialize.

Answer 11

we can break the string down to an array:

$finalArray = array();
$nodeArr = explode('&', $_POST['formData']);

foreach($nodeArr as $value){
    $childArr = explode('=', $value);
    $finalArray[$childArr[0]] = $childArr[1];
}

Answer 12

Serialize:

foreach ($income_data as $key => &$value)
{
    $value = urlencode($value);
}
$data_str = serialize($income_data);

Unserialize:

$data = unserialize($data_str);
foreach ($data as $key => &$value)
{
    $value = urldecode($value);
}

Answer 13

this one worked for me.

function mb_unserialize($string) {
    $string = mb_convert_encoding($string, "UTF-8", mb_detect_encoding($string, "UTF-8, ISO-8859-1, ISO-8859-15", true));
    $string = preg_replace_callback(
        '/s:([0-9]+):"(.*?)";/',
        function ($match) {
            return "s:".strlen($match[2]).":\"".$match[2]."\";"; 
        },
        $string
    );
    return unserialize($string);
}

Answer 14

In my case the problem was with line endings (likely some editor have changed my file from DOS to Unix).

I put together these apadtive wrappers:

function unserialize_fetchError($original, &$unserialized, &$errorMsg) {
    $unserialized = @unserialize($original);
    $errorMsg = error_get_last()['message'];
    return ( $unserialized !== false || $original == 'b:0;' );  // "$original == serialize(false)" is a good serialization even if deserialization actually returns false
}

function unserialize_checkAllLineEndings($original, &$unserialized, &$errorMsg, &$lineEndings) {
    if ( unserialize_fetchError($original, $unserialized, $errorMsg) ) {
        $lineEndings = 'unchanged';
        return true;
    } elseif ( unserialize_fetchError(str_replace("\n", "\n\r", $original), $unserialized, $errorMsg) ) {
        $lineEndings = '\n to \n\r';
        return true;
    } elseif ( unserialize_fetchError(str_replace("\n\r", "\n", $original), $unserialized, $errorMsg) ) {
        $lineEndings = '\n\r to \n';
        return true;
    } elseif ( unserialize_fetchError(str_replace("\r\n", "\n", $original), $unserialized, $errorMsg) ) {
        $lineEndings = '\r\n to \n';
        return true;
    } //else
    return false;
}