React/Node Trouble sending ImageData to server

Question

I have a problem where i have some issues to send canvas's imageData to my server to process it.

Here's my react/redux action to send data:

export const edit = (e) => dispatch => {
  let payload = store.getState().image.imageData
  payload = payload[payload.length-1]
  console.log(payload)
  const id = e.target.id ? e.target.id : e.target.parentElement.id
  fetch(`/api/editing/${id}`, {
    method: 'POST',
    // headers: {
    //   "Content-Type": "application/json"
    // },
    body: JSON.stringify({ imgData: payload })
  })
  .then(res => res.json())
  .then(json => {
    dispatch({ type: UPDATE_IMAGE, payload: json.imgData })
    // handle response display
  })
}

Here's my how I handle request:

const traitement = require('../../processing/traitement')

router.post('/:type', (req, res) => {
  const { imgData } = req.body
  const method = req.params.type
  const imgDataProcessed = traitement[method](imgData)
  return res.status(200).json({ imgData: imgDataProcessed })
})

Here's an example of what a treatment method looks like:

negatif: function(imgData) {
  console.log(imgData)
  var n = imgData.data.length;
  for(i=0; i<n; i+=4){
    imgData.data[i] = 255 - imgData.data[i];
    imgData.data[i+1] = 255 - imgData.data[i+1];
    imgData.data[i+2] = 255 - imgData.data[i+2];
    imgData.data[i+3] = 255;        
  }
  return imgData;
},

the console.log() just before I send (what I expect to send):

ImageData {data: Uint8ClampedArray(180000), width: 300, height: 150}
data: Uint8ClampedArray(180000) [0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 
  255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 
  255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 
  0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 
  255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 
  0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 255, 
  255, …]
height: 150
width: 300
__proto__: ImageData

I can't paste what I get from the server but I lose the width, height keys, the typeof is Object instead of ImageData and the data key is typeof Object instead of Uint8ClampedArray.

So my question is: How can I do to make my route access the same data I send so I can be able to process it?

As you can see I send them as stringified json and I have a json bodyparser middleware on my server, maybe it's from there. I also though about the content-type header

EDIT: Thanks to Kaiido I've modified my code that way which seems to work at 1 exception

How I modified my code, front:

  let payload = store.getState().image.imageData
  payload = payload[payload.length-1]
  const id = e.target.id ? e.target.id : e.target.parentElement.id;
  console.log(payload)
  const meta = {
    width: payload.width,
    height: payload.height
  }
  const formData = new FormData();
  formData.append("meta", JSON.stringify(meta));
  formData.append("data", new Blob([payload.data.buffer]))
  fetch(`/api/editing/${id}`, {
    method: "POST",
    body: formData
  })
  .then(res => res.arrayBuffer())
  .then(buffer => {
    console.log(buffer)
    const data = new Uint8ClampedArray(buffer)
    console.log(data.length)
    const newImg = new ImageData(data, payload.width, payload.height)
    return newImg
  })

back:

router.post('/:type', (req, res) => {
  let form = new formidable.IncomingForm()
  form.parse(req, (err, fields, files) => {
    fs.readFile(files.data.path, (err, data) => {
      const imgProcessed = traitement[req.params.type](data)
      console.log(imgProcessed.length)
      return res.status(200).json([imgProcessed])
    })
  })
})

1 issue left: assuming that for testing I'm using a 150300px image my data array should be 180000 long (300150*4) which it is until I send server's response. When the front receives response it calls res.arrayBuffer() then it creates a new Uint8ClampedArray but then my length is no more 180000 but 543810 in that case. As Kaiido said I might want to slice that array, which I tried, but doesn't works.

How should I slice it? 180000 firsts ones? 180000 lasts ones? some other way?

Answer 1

Do

const { imgData } = JSON.parse(req.body)

this is the reverse of JSON.stringify and needed as you are sending data via stringify but in route not parsing it back to Original Object.

Answer 2

JSON only has very few types it handles: Strings, Numbers, Arrays, Objects, Booleans and null.

An ImageData, which has no enumerable properties will get JSON.stringified to "{}", an empty object.

const img = new ImageData(6, 6);
img.data.fill(0xFF);
console.log('JSON', JSON.stringify(img)); // "{}"

const props = Object.getOwnPropertyDescriptors(img);
console.log('own properties', props); // {}

You'll loose all informations. So to circumvent this, you'd have to send all these manually in your request.
Sounds quite easy to simply move the needed values to an other object:

const img = new ImageData(6,6);
const dataholder = (({width, height, data}) => ({width, height, data}))(img);

console.log(JSON.stringify(dataholder));

However, you'll notice that data property is not stringified in a proper way.

Indeed, a TypedArray when JSON.stringified is converted to a simple object, with all its indexes set as keys. And what should have taken only 144 bytes to represent our 6px*6px image data now takes 1331 bytes.

const typedArray = new Uint8ClampedArray(6*6*4).fill(0xFF);

console.log(JSON.stringify(typedArray));

const as_string = new Blob([JSON.stringify(typedArray)]);
const as_arraybuffer = new Blob([typedArray]);

console.log('as string', as_string.size, 'bytes');
console.log('as arraybuffer', as_arraybuffer.size, 'bytes');

I'll let you do the Maths for bigger images, but you might consider an other approach...

---

Instead of sending a stringified version of this Uint8ClampedArray, you'd better send its underlying ArrayBuffer. That may sound like more code for you to write, but you'll save bandwidth, and maybe even trees.

So to send your ImageData to your server, you'll do

const img = new ImageData(6, 6);
img.data.fill(0xFF);

const meta = {
  width: img.width,
  height: img.height
  // add more metadata here if needed
};
const formdata = new FormData();
formdata.append("meta", JSON.stringify( meta ))
formdata.append("data", new Blob([img.data.buffer]));

console.log(...formdata); // [key, value], [key, value]
// here data is a File object and weights only 144 bytes

/* // send the formdata to your server
fetch('yoururl', {
  method: 'POST',
  body: formdata
})
.then(response => response.arrayBuffer())
... // we'll come to it later
*/

Now on your server, in order to retrieve all the parts of your request, you'll have to grab the form-data content, and access the image's data as a File upload.

You'll then have to read this uploaded file (e.g using fs.readFile(temp_path) and do the manips from the returned Buffer directly (since Buffers in node are Uint8Array).

Once your manips are done, you can send directly this modified Buffer as a response.

Now on front, you'll just have to ask for the Response to be consumed as an ArrayBuffer, and to generate a new ImageData from this ArrayBuffer:

//... see above
fetch('yoururl', {
  method: 'POST',
  body: formdata
})
// request as ArrayBuffer
.then(response => response.arrayBuffer())
.then(buffer => {
  // create a new View over our ArrayBuffer
  const data = new Uint8ClampedArray(buffer);
  const new_img = new ImageData(data, img.width, img.height);
  return new_img;
});

Here it is assuming that your front doesn't need metadata from server response (because you already have it).
If you are in a case where you need some metadata, then you can prepend it at the beginning of your ArrayBuffer response (assign a slot to let you know its size, then slice() your received Array in order to grab both the metadata and the actual image data.