I would say it's the large number of points you are drawing that's causing the slowdown. One option is to downsample.. Also you could use lower-level functions to draw (check this related post for a comparison of plot3/scatter3/line performance).
Consider the animation below optimized for speed:
[X Y Z] = sphere(64);
X = X(:); Y = Y(:); Z = Z(:);
%# set-up figure
hFig = figure('Backingstore','off', 'renderer','zbuffer');
%# use lower-level function LINE
line(0.50*[X,X], 0.50*[Y,Y], 0.50*[Z,Z], 'LineStyle','none', 'Marker','.', 'MarkerSize',1, 'Color','r')
line(0.75*[X,X], 0.75*[Y,Y], 0.75*[Z,Z], 'LineStyle','none', 'Marker','.', 'MarkerSize',1, 'Color','g')
line(1.00*[X,X], 1.00*[Y,Y], 1.00*[Z,Z], 'LineStyle','none', 'Marker','.', 'MarkerSize',1, 'Color','b')
view(3)
%# freeze the aspect ratio to override stretch-to-fill behaviour
axis vis3d
%# fix the axes limits manually
%#set(gca, 'xlim',[-1 1], 'ylim',[-1 1], 'zlim',[-1 1])
axis manual
%# maybe even remove the tick labels
%set(gca, 'xticklabel',[], 'yticklabel',[], 'zticklabel',[])
%# animate (until figure is closed)
while ishandle(hFig); camorbit(0.9,-0.1); drawnow; end

Note how we are using the Z-buffer renderer, and turned off the Backingstore property.
EDIT:
If I understood correctly, what you are trying to do is to record a screencast (using a 3rd-party app), while you manually rotate the figure, but in your case these manual rotations are "jumpy". On the other animating your figure with CAMORBIT/VIEW in a while-loop is running smooth...
I propose an alternative solution: start by rotating the figure using the mouse and write these view configurations at each step (azimuth,elevation). Then you can replay them using the VIEW function while recording the video, something like:
v = [...]; %# matrix where each row specify Az/El of view
for i=1:size(v,1)
view( v(i,:) )
drawnow
end
The downside is that you will have to press/rotate/release using the mouse in small steps (the ROTATE3D object does not expose a mouse-motion event)
I wrote a simple function to help you in this process. It loads the saved figure, enable 3d-rotation, and keeps track of the intermediate position at each step. Once finished, press the "Done" button to return the list of views...
function v = rotationDemo(figFileName)
views = []; %# list of views (Az,El)
hFig = hgload(figFileName); %# load the saved figure
views(1,:) = get(gca,'View'); %# store initial view
%# add a button, used to terminate the process
hButton = uicontrol('Style','pushbutton', 'Position',[400 1 80 20], ...
'String','Done?', 'Callback',@buttonCallback);
set(hFig, 'Toolbar','figure') %# restore toolbar
%# start 3d rotation, and handle post-callback to record intermediate views
h = rotate3d(hFig); %# get rotation object
set(h, 'ActionPostCallback',@rotateCallback)
set(h, 'Enable','on') %# enable rotation
msgbox('Rotate the view step-by-step', 'rotate3d', 'warn', 'modal')
uiwait(hFig) %# wait for user to click button
delete(hButton) %# delete button on finish
set(h, 'Enable','off') %# disable rotation
v = round(views); %# return the list of views
%# callback functions
function rotateCallback(o,e)
views(end+1,:) = get(e.Axes,'View'); %# add current view to list
end
function buttonCallback(o,e)
uiresume(gcbf) %# uiresume(hFig)
end
end

You can call the above function, then replay the animation:
v = rotationDemo('smooth_rotation.fig');
for i=1:size(v,1)
view(v(i,:))
drawnow
end
We can smooth the transitions by simple interpolation:
v = rotationDemo('smooth_rotation.fig');
n = size(v,1);
nn = linspace(1,n,100)'; %'# use 100 steps
vv = round( [interp1(v(:,1),nn) interp1(v(:,2),nn)] );
for i=1:size(vv,1)
view(vv(i,:))
DRAWNOW %# or PAUSE(..) to slow it down
end
As a side note, I should mention that ROTATE3D and CAMORBIT have different effects. ROTATE3D changes the View
property of the current axis, while CAMORBIT controls the camera properties CameraTarget
/CameraPosition
/CameraUpVector
of the current axis.