Have image data given as base64-encoded JSON; I convert it into Nodejs Buffer type object. How do I show image again from saved Buffer data?

Question

My situation is the following.

I'm retrieving some image data from OpenAI's DALLE-2 API. The response gives me data as base64-encoded JSON. Once retrieved, I store it in a MongoDB database as Schema.Types.Buffer. I'm using Apollo-implementation of GraphQL for my API query language. I needed to create a custom scalar type so I could assign the Image model's binData field the custom scalar type ImgBinData. See below my type Image definition:

For the custom scalar type, I need to define the logic of how to handle it in the instance of the GraphQLScalarType class provided by Apollo Server. It's here where I create the Nodejs Buffer and save that to the image's binData field.

Where I'm not sure if what I'm doing is correct is once I want to retrieve the buffer data stored in an image's binData field. What do I do to convert this binary data back into the image it represents? Might I need to change what happens in the serialize method of my custom scalar? As of now, it's just returning the sequence of integers in the Buffer array:

I want to use this binary data as the source value for an image element in my view layer.

---

@Traynor, here is my React function component code:

import React from "react";
import { useQuery } from "@apollo/client";
import { GET_IMAGE_BY_ID } from "../utils/queries";

const ProfilePicture = () => {
  // run query here
  const { loading, error, data } = useQuery(GET_IMAGE_BY_ID, {
    variables: { imageId: "64715e7413b66dd4d30eea3b" },
  });

  if (loading) return null;
  if (error) return `Error! ${error}`;
  const imgBinData = data?.imageByImageId.binData.data;
  console.log(imgBinData);
  const imgType = "image/png";
  const blob = new Blob([new Uint8Array(imgBinData)], { type: imgType});
  const src = URL.createObjectURL(blob);
  return (
    <img
      src={src}
      className=""
      alt="A headshot of the user"
    />
  );
};

export default ProfilePicture;

Here is my logic for the custom scalar:

const { GraphQLScalarType, Kind } = require("graphql");

const imgBinDataScalar = new GraphQLScalarType({
  name: "ImgBinData",
  description:
    "Custom scalar for storing image bin data in form of Nodejs Buffer type",
  serialize: (bufferObj) => bufferObj,
  parseValue: (value) => Buffer.from(value, "base64"),
  parseLiteral: (ast) => Buffer.from(ast.value, "base64"),
});

module.exports = {
  imgBinDataScalar,
};

Below is a snapshot of the base64-encoded JSON I received from OpenAI's API. The value of the property b64_json is what I pass into as the value for my parseValue method in my custom scalar instance above. All that base64-encoded characters are fed into the Buffer.from method. I wanted to show you this so I don't leave any gaps from the moment I receive the data from OpenAI, through data transformation, up to wanting to display it in the img element.

Also, here is a snapshot of Chrome DevTools Network tab and the Elements tab:

Answer 1

So, the server sends buffer as JSON, which means you need to convert it blob, and then convert it into a URL, so that you can assing it to img.src, so that it can be displayed.

So, convert data array (I think it's data.imageByImageId.binData.data, this is why you should post code, instead of screenshots) into a Uint8Array (also, you'll need to store image type into a database, and retrieve it too), then create a Blob, and then convert blob to URL and assign it to src attribute (you'll probably need to adapt the selector)

Try this:

// make sure you pass the image data array property
const responseImage = data.imageByImageId.binData.data;

// you'll need to store and retrieve image type as well
const imgType = 'image/jpeg';

const blob = new Blob([new Uint8Array(responseImage)], {type:imgType});

// get the img element, and add image to src
document.querySelector('.profile__details__div--image img').src = window.URL.createObjectURL(blob);

Answer 2

It seems like your Scalar implementation is not doing quite what you want. The functions parseValue and parseLiteral take a String in base64 format and turn it into a Buffer. I assume that you want to use the Buffer type on the server and for the transport layer, you want to use base64 strings, similarly to what OpenAI do. These functions could be improved with validation and checking, that you are actually getting strings, but apart from that, they look fine.

Again, assuming that you want to use the Buffer type on the server and for the transport layer, your serialize function is wrong. It returns the plain buffer. This is why in GraphiQL, you are getting back a weird object:

{
  type: 'Buffer',
  data: [...],
}

Let's fix that and return a base64 string instead. You can use the toString method of Buffer.

  serialize: (value) => value.toString('base64')

Now, it looks like your idea on the react side is actually quite good! You did not add your code, but the browser screenshot seems like this is almost what you want (be careful, you have a typo in image/jpeg). You can create a source URL that uses base64 encoded data.

const url = `data:image/jpeg;base64,${imageByImgId.binData}`;

return <img src={url} />

This should be it.