How can I decode rosbridge data in the browser?
So far ive been able to decode the following types:
- Uncompressed raw RGB
- Uncompressed raw Depth
- JPEG compressed RGB
My Problem now is decoding compressed depth and PointCloud2 data. As far as my understanding goes, the data is encoded as base64. The depth image has been compressed to a mono16 PNG. I have tried many different approaches, but none seem to work. The depth image is supposed to contain 307200 depth values of 16 bits each.
I do not want to display this data in something like ros3djs or webviz (cloud not figure out how they do the decoding). I want to decode the data and use it in my own analysis.
Steps to reproduce:
Here is a sample file. It contains the data field of the JSON message: https://drive.google.com/file/d/18ZPpWrH9TKtPBbevfGdceZVpmmkiP4bh/view?usp=sharing
OR
- Make sure you have a device publising or a rosbag playing
- roslaunch rosbridge_server rosbridge_websocket.launch
- Launch your web page and make sure you added roslibjs in your script tag
The JS of my web page is simplified to this:
var ros = new ROSLIB.Ros({
url: 'ws://127.0.0.1:9090'
});
var depthListener = new ROSLIB.Topic({
ros: ros,
name: '/camera/color/image_raw/compressedDepth',
messageType: 'sensor_msgs/CompressedImage'
});
var pointCloudListener = new ROSLIB.Topic({
ros: ros,
name: '/camera/depth/color/points',
messageType: 'sensor_msgs/PointCloud2'
});
depthListener.subscribe(function (message) {
console.log(message);
depthListener.unsubscribe();
});
pointCloudListener.subscribe(function (message) {
console.log(message);
pointCloudListener.unsubscribe();
});
I have set the two topics to unsubscribe after the first message so that my console does net get flooded.
Provided a screenshot of the console logs for depth image
and for pointcloud
This is what I have so far, but the onload function is never triggered.
image = new Image();
image.src = "data:image/png;base64, " + message.data
image.onload = function(){
image.decode().then(() =>{
if(image.width != 0 && image.height != 0){
canvas.width = image.width;
canvas.height = image.height;
ctx = canvas.getContext('2d');
ctx.drawImage(image, 0, 0);
image_data = canvas.getContext('2d').getImageData(0,0, 640,480).data;
}
});
}
I think this opencv code was used to compress the image. The message can essentially be thought of as a 16 bit gray scale image.
Please comment if I can update the question with specific information
Thanks in adance