Laravel Eloquent object, longtext being truncated

Question

I've using Laravel, loading a db row into an Eloquent object. One of the columns is longtext, a JSON encoded array > 2 million characters long. The original error I was getting was json_decode failing on this column's value.

I tested in tinker. Simplified test code:

$item = Item::find(1);
echo $item->long_text_field;
var_dump(json_decode($item->long_text_field));
echo strlen($item->long_text_field);

On my local vagrant instance this shows the correct values.

...long json array in text, same as the value in the actual DB entry...
...var_dump of json array...
2334040

However on my remote dev server I'm getting

...long json array truncated in the middle...
NULL
1048576

Obviously the json_decode is failing because the string is truncated.

It truncates at a piece like this

"Eff Date":"1\”

Which should be

"Eff Date":"1\/30\/14 16:13”

There's a lot of escaped slashes like that in the longtext, and no strange characters at that point I can see. Does anyone have an idea why this text would truncate like that on one server and not another?

Ian Chadwick · Accepted Answer · 2016-02-26T09:02:18.667

9

The problem is the default PDO::MYSQL_ATTR_MAX_BUFFER_SIZE size is 1Mb.

To set this in Laravel you need to add an option to your database.php config file.

'connections' => [
    'mydb' => [
        'driver'    => 'mysql',
        'host'      => 'localhost',
        'database'  => 'mydb',
        'options'   => [
            PDO::MYSQL_ATTR_MAX_BUFFER_SIZE => 16777216
        ]
    ]
]

The above will set the max attribute size to 16Mb.

Note that if you're using mysqlnd driver you don't need this anymore and it'll actually break your code as the PDO::MYSQL_ATTR_MAX_BUFFER_SIZE constant doesn't exist.

edited Feb 26 '16 at 09:02

answered Oct 08 '15 at 07:24

Ian Chadwick

1,547
1
19
21

1

Got an error `Fatal error: Undefined class constant 'MYSQL_ATTR_MAX_BUFFER_SIZE' `, seems I am using mysqlnd, but the text is still truncated. Why? – leetom Aug 16 '16 at 08:01
Hmm strange. How long is the data you're trying to store and what length does it get truncated? Also what field type are you using in MySQL? – Ian Chadwick Aug 16 '16 at 08:50
1

@IanChadwick It turns out that I misused text as the field type. Problem solved after I changed that to longtext. Thanks! – leetom Aug 16 '16 at 09:36

score 0 · Answer 2 · answered Oct 03 '14 at 18:02

The problem may lie in the json_encoding stage, before you even retrieve the results.

Try disabling forward slash escaping by:

$str = "1/30/14 16:13";
echo json_encode($str, JSON_UNESCAPED_SLASHES);

Most probably, this is a symptom of having magic_quotes_gpc enabled (meaning you may be using PHP version < 5.2);

From the PHP manual:

; Magic quotes
;

; Magic quotes for incoming GET/POST/Cookie data.
magic_quotes_gpc = Off

; Magic quotes for runtime-generated data, e.g. data from SQL, from exec(), etc.
magic_quotes_runtime = Off

; Use Sybase-style magic quotes (escape ' with '' instead of \').
magic_quotes_sybase = Off

If access to the server configuration is unavailable, use of .htaccess is also an option. For example:

php_flag magic_quotes_gpc Off

Tried that. Text still truncates. Both my local and dev server are using PHP 5.5, so I don't think that's the issue. — wakeman, Oct 03 '14 at 21:02

Laravel Eloquent object, longtext being truncated

2 Answers2