I have 2 set data acquired from Kinect 1- depth image with size 480640 (uint16) from a scene 2- color image with same size (480640*3 single) from same scene.
How can I merge these data together to generate a colored 3D point clouds with PLY format in Matlab?
Unfortunately I don't have an access to Kinect anymore and I should use only these data.
I've never tried to do that in matlab, but i think that this is what you are looking for:
http://es.mathworks.com/help/vision/ref/pcfromkinect.html
The tool is inside the Computer Vision System Toolbox™.
Nice question. You should use this tutorial from Burrus - basically you need to use your depth information to convert the color/depth centroids into the 3rd dimension. Be aware that Kinect v1 depth and color streams are slightly mismatched, so take that into account as well.
The tutorial can be found here: http://nicolas.burrus.name/index.php/Research/KinectCalibration You can also use the work of this author: Khoshelham, K., & Elberink, S. O. (2012) - Accuracy and resolution of Kinect depth data for indoor mapping applications
The matlab code should be something like:
% All formulas and values from:
% Khoshelham, K., & Elberink, S. O. (2012).
% Accuracy and resolution of Kinect depth data for indoor mapping applications.
% Sensors (Basel, Switzerland), 12(2), 1437–54. doi:10.3390/s120201437
load('janFrameThousand.mat')
pc=zeros([size(D) 3]);
W=size(D,2);
H=size(D,1);
f=5.453;
for indWidth = 1:W
for indHeight= 1:H
% copy z value
pc(indHeight,indWidth,3)=D(indHeight,indWidth);
% calc x value
pc(indHeight,indWidth,1)=-(pc(indHeight,indWidth,3)/f)*...
((indWidth-W/2)*0.0093+0.063);
% calc y value
pc(indHeight,indWidth,2)=-(pc(indHeight,indWidth,3)/f)*...
((indHeight-H/2)*0.0093+0.039);
end
end
X=pc(:,:,1);
% X=X(:);
Y=pc(:,:,2);
% Y=Y(:);
Z=-pc(:,:,3);
Z(Z==0)=NaN;
Surface=surf(X,Y,Z,'edgecolor','none','facecolor','interp');
lighting gouraud
camlight
% colormap(repmat(winter,20,1))
axis image
axis vis3d
xlabel('X axis')
ylabel('Y axis')
zlabel('Z axis')
Here is the MATLAB OOP code to convert RGB-D images to point clouds, provided the camera's intrinsic parameters:
% Class definition
classdef rgbd2pointcloud < handle
% Public properties
properties
color
depth
json_filename
cam_intrinsics
end
% Public methods
methods
% Constructor
function obj = rgbd2pointcloud(color, depth, cam_intrinsics, json_filename)
if isstring(color)
obj.color = imread(color);
else
obj.color = color;
end
if isstring(depth)
obj.depth = double(imread(depth));
else
obj.depth = double(depth);
end
obj.json_filename = json_filename;
obj.cam_intrinsics = cam_intrinsics;
end
% Get the XYZ and RGB data
function [xyz, rgb] = xyz_rgb(obj)
cam_params = get_camera_intrinsics(obj);
xx = 1 : cam_params.width;
yy = 1 : cam_params.height;
[xv, yv] = meshgrid(xx, yy);
u = reshape(xv', 1, []);
v = reshape(yv', 1, []);
z = reshape(obj.depth', 1, []) / cam_params.depth_scale;
x = (u - cam_params.cx) .* z / cam_params.fx;
y = (v - cam_params.cy) .* z / cam_params.fy;
xyz = [x; y; z]';
rgb = reshape(pagetranspose(obj.color), [], 3);
end
% Write to the point cloud
function write2file(obj, xyz, rgb, file_name)
ptCloud = pointCloud(xyz, Color = rgb);
pcwrite(ptCloud,file_name, 'Encoding', 'compressed')
end
end
% Private methods
methods (Access = private)
% JSON reader
function cam_params = readJSON(obj)
fid = fopen(obj.json_filename);
raw = fread(fid,inf);
str = char(raw');
fclose(fid);
cam_params = jsondecode(str);
end
% Get the camera intrinsics
function cam_params = get_camera_intrinsics(obj)
if ~isempty(obj.json_filename)
cam_params = readJSON;
else
cam_params = obj.cam_intrinsics;
end
end
end
end
Below is the code usage:
%% Inputs
% Read the RGB and D images
color = imread("redwood_847.png");
depth = imread("redwood_847d.png");
% Json filename
json_filename = [];
% Output point cloud filename
file_name = 'output.pcd';
% Camera intrinsics
camera_intrinsic.cx = 319.5;
camera_intrinsic.cy = 239.5;
camera_intrinsic.fx = 525;
camera_intrinsic.fy = 525;
camera_intrinsic.depth_scale = 1000; % Depth scale (constant) to convert mm to m vice-versa
camera_intrinsic.width = 640;
camera_intrinsic.height = 480;
%% Object callback
obj = rgbd2pointcloud(color, depth, camera_intrinsic, json_filename);
[xyz, rgb] = obj.xyz_rgb();
obj.write2file(xyz, rgb, file_name)
%% Display the point cloud
figure;
pcshow('output.pcd', 'VerticalAxis', 'x', 'VerticalAxisDir', 'down')
xlabel("X")
ylabel("Y")
zlabel("Z")
I have also uploaded this code to the MATLAB File Exchange
