Convert buffer to array

Question

I am setting memcached with

$memcached->set("item" , ["1" => "hello"]);

anything work in PHP ,

In Node.js with memcached plugin , I get a buffer instead of array in result

<Buffer 61 3a 25 61 34 3a>

I can not convert such buffer to array

In Node.js :

memcached.get("item" , function(err, data) {
  console.log(data);
}

Do you have any way ?

What happens if you just call `$memcached->set("item" , "hello");` in php? — SheetJS, Aug 09 '13 at 14:23
how about `$memcached->set("item" , ["1", "hello"]);` I wonder if PHP is not writing the array properly — SheetJS, Aug 09 '13 at 14:28
I can ge that as array well , may be the php serilize that , also I got array with this code phpjs.unserialize(data.toString()); — Ata, Aug 09 '13 at 14:31

score 70 · Answer 1 · edited Jun 20 '20 at 09:12

arr = [...buffer]

ES6 introduced a lot of other features, besides buffers.

You can even easily append like this:

arr.push(...buffer)

The ... operator expands enumerables such as arrays and buffers when used in array. It also expands them into separate function arguments.

Yes, it's also faster:

... : x100000: 835.850ms

Slice call from prototype : x100000: 2118.513ms

var array,
    buffer = new Buffer([1, 4, 4, 5, 6, 7, 5, 3, 5, 67, 7, 4, 3, 5, 76, 234, 24, 235, 24, 4, 234, 234, 234, 325, 32, 6246, 8, 89, 689, 7687, 56, 54, 643, 32, 213, 2134, 235, 346, 45756, 857, 987, 0790, 89, 57, 5, 32, 423, 54, 6, 765, 65, 745, 4, 34, 543, 43, 3, 3, 3, 34, 3, 63, 63, 35, 7, 537, 35, 75, 754, 7, 23, 234, 43, 6, 247, 35, 54, 745, 767, 5, 3, 2, 2, 6, 7, 32, 3, 56, 346, 4, 32, 32, 3, 4, 45, 5, 34, 45, 43, 43]),
    iter = 100000;

array = buffer;

console.time("... : x" + iter);
for (var i = iter; i--;) array = [...buffer]
console.timeEnd("... : x" + iter);

console.time("Apply/call/etc : x" + iter);
for (var i = iter; i--;) array = Array.prototype.slice.call(buffer, 0)
console.timeEnd("Apply/call/etc : x" + iter);

DarckBlezzer · Answer 2 · 2019-03-14T23:02:25.793

There is another way to convert to array of integers

Using toJSON()

Buffer.from('Text of example').toJSON()
{ type: 'Buffer',data: [ 84, 101, 120, 116, 32, 111, 102, 32, 101, 120, 97, 109, 112, 108, 101 ] }

// simple get data
Buffer.from('Text of example').toJSON().data
[ 84, 101, 120, 116, 32, 111, 102, 32, 101, 120, 97, 109, 112, 108, 101 ]

Example of benchmark

// I took this from @user4584267's answer
const buffer = new Buffer([1, 4, 4, 5, 6, 7, 5, 3, 5, 67, 7, 4, 3, 5, 76, 234, 24, 235, 24, 4, 234, 234, 234, 325, 32, 6246, 8, 89, 689, 7687, 56, 54, 643, 32, 213, 2134, 235, 346, 45756, 857, 987, 0790, 89, 57, 5, 32, 423, 54, 6, 765, 65, 745, 4, 34, 543, 43, 3, 3, 3, 34, 3, 63, 63, 35, 7, 537, 35, 75, 754, 7, 23, 234, 43, 6, 247, 35, 54, 745, 767, 5, 3, 2, 2, 6, 7, 32, 3, 56, 346, 4, 32, 32, 3, 4, 45, 5, 34, 45, 43, 43]);
let array = null;
const iterations = 100000;

console.time("...buffer");
for (let i = iterations; i=i-1;) array = [...buffer]
console.timeEnd("...buffer");

console.time("array.prototype.slice.call");
for (let i = iterations; i=i-1;) array = Array.prototype.slice.call(buffer, 0)
console.timeEnd("array.prototype.slice.call");

console.time("toJSON().data");
for (let i = iterations; i=i-1;) array = buffer.toJSON().data
console.timeEnd("toJSON().data");

OUTPUT

...buffer: 559.932ms
array.prototype.slice.call: 1176.535ms
toJSON().data: 30.571ms

or if you want more profesional and custom function in Buffer use this:

Buffer.prototype.toArrayInteger = function(){
    if (this.length > 0) {
        const data = new Array(this.length);
        for (let i = 0; i < this.length; i=i+1)
            data[i] = this[i];
        return data;
    }
    return [];
}

Example of benchmark:

const buffer = new Buffer([1, 4, 4, 5, 6, 7, 5, 3, 5, 67, 7, 4, 3, 5, 76, 234, 24, 235, 24, 4, 234, 234, 234, 325, 32, 6246, 8, 89, 689, 7687, 56, 54, 643, 32, 213, 2134, 235, 346, 45756, 857, 987, 0790, 89, 57, 5, 32, 423, 54, 6, 765, 65, 745, 4, 34, 543, 43, 3, 3, 3, 34, 3, 63, 63, 35, 7, 537, 35, 75, 754, 7, 23, 234, 43, 6, 247, 35, 54, 745, 767, 5, 3, 2, 2, 6, 7, 32, 3, 56, 346, 4, 32, 32, 3, 4, 45, 5, 34, 45, 43, 43]);
let array = null;
const iterations = 100000;

console.time("toArrayInteger");
for (let i = iterations; i=i-1;) buffer.toArrayInteger();
console.timeEnd("toArrayInteger");

Ouput:

toArrayInteger: 28.714ms

Note: In the last example I copied a function from Buffer.toJSON and custom it a lite

score 11 · Answer 3 · answered Mar 22 '17 at 13:47

11

Here you go:

var buffer = new Buffer([1,2,3])
var arr = Array.prototype.slice.call(buffer, 0)
console.log(arr)

answered Mar 22 '17 at 13:47

Chipintoza

295
4
16

8

Could you add a little more detail to your answer? Thanks! – maxshuty Mar 22 '17 at 14:09
2

What additional detail is needed?! – Dave Newton Aug 08 '18 at 17:40
Super concise answer that works! Thanks I was struggling to read the silly paragraphs. – murage kibicho Jul 04 '23 at 18:38

score 3 · Answer 4 · answered Aug 09 '13 at 15:13

3

I haven't used memcached so I am not sure just what this buffer represents or what you want to have instead. Sorry. Here is a function to split a buffer up into an array of bytes. More at node.js Buffer docs, hope it helps!

var hex = new Buffer("613a2561343a", "hex");
var l = hex.length; // in bytes
var output = [];
for(var i = 0; i < l; i++){
  var char = hex.toString('hex',i,i+1); // i is byte index of hex
  output.push(char);
};
console.log(output);
// output: [ '61', '3a', '25', '61', '34', '3a' ]

answered Aug 09 '13 at 15:13

Plato

10,812
2
41
61

1

it just split the buffer data , it do not convert each item to used character – Ata Aug 09 '13 at 15:20
Gotcha. Maybe there is a way to make PHP save a non-encoded string into memcached? I don't know either very well, sry - good luck! – Plato Aug 09 '13 at 15:21

score 2 · Answer 5 · answered Apr 28 '20 at 14:55

2

You can also use Array.from:

memcached.get("item" , function(err, data) {
  console.log(Array.from(data));
}

answered Apr 28 '20 at 14:55

hippietrail

15,848
18
99
158

JacobTDC · Answer 6 · 2017-08-23T17:54:59.463

1

I have a solution, although I am currently trying to find a better one:

function bufToArray(buffer) {
  let array = new Array();
  for (data of buffer.values()) array.push(data);
  return array;
}

EDIT : I found a simpler way:

var buffer = Buffer.from('NodeJS rocks!')
var array = new Function(`return [${Array.prototype.slice.call(buffer, 0)}]`)

But, like someone already said, [...buffer] is faster (and more code efficient).

You can also use new Uint8Array(buffer [, byteOffset [, length]]);

edited Aug 23 '17 at 17:54

answered Aug 12 '17 at 23:17

JacobTDC

487
1
4
17

'a simpler way' is eval which is totally uncalled-for here. – Estus Flask Mar 18 '18 at 15:53
Why in the world do you need a function constructor?? Is it more performant or in any way better than `var array = Array.prototype.slice.call(buffer, 0);`? – code Jan 31 '22 at 06:30

score -3 · Answer 7 · answered Aug 09 '13 at 14:32

-3

In interent , there was no information about that , but I have found the convert way

In nodejs , I have to use :

var arrayobject = phpjs.unserialize(data.toString());

but , it is very stupid way for getting array , it seem that php serilzie the data when setting memcache .

answered Aug 09 '13 at 14:32

Ata

12,126
19
63
97

You could try to store the array as JSON. `$memcached->set("item" , json_encode(["1" => "hello"]));`. Then decode the JSON in node.js. – gen_Eric Aug 09 '13 at 14:40
Yes , that is way , but think we get high rate on encode decode per sec , in such way , that is not efficent way – Ata Aug 09 '13 at 14:58

Convert buffer to array

7 Answers7

Linked