Convert Texture2D to Byte[] using EncodeToJPG (in OpenVR)

Question

I have one question!

I just want, just...convert Texture2D to Byte[] using EncodeToJPG() function!

My workspace is Unity and C# Script, + OpenCvSharp.

Maybe you think that It is easy, but It have some problem.

this script use OpenVR(HTC VIVE).

Anyway, Here is my source.

var source = SteamVR_TrackedCamera.Source(undistorted);

//Receive texture data from VR.
texture = source.texture;

//Debug.Log(source.texture.width + "/" + source.texture.height);    
//size : 612 /460
if (texture == null)
{
    return;
}

//Input texture data into material, and it's in unity (quad GameObject).
//this G.O print display like camera
material.mainTexture = texture;


//here is my src, I want to save Image but texture.EncodeToJPG has some error.
Cv2.ImShow("_Texture ...", Mat.FromImageData(texture.EncodeToJPG()));
Cv2.ImWrite(SavePath+ "Image.jpg", Mat.FromImageData(texture.EncodeToJPG()));

and...there is problem. variable texture is abnormal Texture2D type.

if (_texture == null)
{
    _texture = Texture2D.CreateExternalTexture((int)header.nWidth, (int)header.nHeight, TextureFormat.RGBA32, false, false, nativeTex);
    //_texture = new Texture2D(612, 460, TextureFormat.RGBA32, false);

    uint width = 0, height = 0;
    var frameBounds = new VRTextureBounds_t();
    if (trackedCamera.GetVideoStreamTextureSize(deviceIndex, frameType, ref frameBounds, ref width, ref height) == EVRTrackedCameraError.None)
    {
        // Account for textures being upside-down in Unity.
        frameBounds.vMin = 1.0f - frameBounds.vMin;
        frameBounds.vMax = 1.0f - frameBounds.vMax;
        this.frameBounds = frameBounds;
    }
}
else
{
    _texture.UpdateExternalTexture(nativeTex);
    //_texture.Apply();
}

that _texture is created by function CreateExternalTexture and it has parameter of Intptr type named nativeTex.

I don't know how can I do?

+++ edit! Error display +++

You said there is a problem but you never mentioned it. Is there anything wrong with EncodeToJPG ? — Programmer, Jul 25 '18 at 09:05
What's your Unity version and is this problem happen in the Editor? Also call `byte []img = texture.EncodeToJPG()` then comment out the `Mat.FromImageData` code then tell me the result — Programmer, Jul 25 '18 at 09:26
and in script, there is no error but Cv2.ImShow(~) is not working and this source: **byte[] img = texture.EncodeToJPG();** **Cv2.ImShow("_Texture ...", Mat.FromImageData(img));** is also have same problem. (Unable to retrieve image reference. — bolero, Jul 25 '18 at 09:30
Do you get error with `byte[] img = texture.EncodeToJPG()`? If not then the issue is not `EncodeToJPG`. also, is this problem happening in the Editor? — Programmer, Jul 25 '18 at 09:31
I have confidence. Error is occured in **texture.EncodeToJPG()** here, because texture is not normal Texture2D type, this is maybe...like pointer? because this texture is made using **Intptr(nativeTex)** in parameter(CreateExternalTexture Function) — bolero, Jul 25 '18 at 09:35

score 0 · Answer 1 · answered Jul 25 '18 at 11:03

0

As Unity may not have direct access to an external texture I would suggest doing a Graphics.Blit() to transfer that texture to a RenderTexture via GPU first.

Once you have your texture in a RenderTexture you need to jump through one more hoop, and readpixels from RenderTexture.active into another standard, RAM based Texture2D, and you should be able to encode it then.

Remember to Apply() the texture after doing ReadPixels()

answered Jul 25 '18 at 11:03

zambari

4,797
1
12
22

Thank you! I used Graphics.Blit() Function and this worked! – bolero Jul 30 '18 at 01:52
@ipsuck at least thank zambari for his help. You did not even upvote his answer or acknowledge him in your self-proclaimed answer – mr5 Jul 30 '18 at 08:59

score 0 · Accepted Answer · answered Jul 30 '18 at 01:56

declared this function:

private Texture2D ReadExternalTexture(Texture externalTexture)
{
    Texture2D myTexture2D = new Texture2D(texture.width, texture.height);
    //myTexture2D => empty Texture2D type variable
    if (myTexture2D == null)
    {
        Debug.Log("var: myTexture2D is null state.");
        myTexture2D = new Texture2D(texture.width, texture.height);
    }

    //Make RenderTexture type variable
    RenderTexture tmp = RenderTexture.GetTemporary(
    externalTexture.width,
    externalTexture.height,
    0,
    RenderTextureFormat.ARGB32,
    RenderTextureReadWrite.sRGB);

    Graphics.Blit(externalTexture, tmp);
    RenderTexture previous = RenderTexture.active;
    RenderTexture.active = tmp;

    myTexture2D.ReadPixels(new UnityEngine.Rect(0, 0, tmp.width, tmp.height), 0,     0);
    myTexture2D.Apply();

    RenderTexture.active = previous;
    RenderTexture.ReleaseTemporary(tmp);

    return myTexture2D;
}

and use ReadExternalTexture Function in void Update():

_texture = ReadExternalTexture(material.mainTexture);

Mat mat = Mat.FromImageData(_texture.EncodeToPNG());

Cv2.Flip(mat, mat, 0);
//Flip(src, dest, 0->updown flip / 1->leftright flip)

Cv2.Resize(mat, mat, new Size(224, 224));

Cv2.ImShow("Image", mat);
//Cv2.ImWrite(SavePath + "Image.jpg", mat);

and this worked totally!

Convert Texture2D to Byte[] using EncodeToJPG (in OpenVR)

2 Answers2