Writing a paint program à la MS Paint - how to interpolate between mouse move events?

Question

I want to write a paint program in the style of MS Paint.

For painting things on screen when the user moves the mouse, I have to wait for mouse move events and draw on the screen whenever I receive one. Apparently, mose move events are not sent very often, so I have to interpolate the mouse movement by drawing a line between the current mouse position and the previous one. In pseudocode, this looks something like this:

var positionOld = null

def handleMouseMove(positionNew):
    if mouse.button.down:
        if positionOld == null:
            positionOld = positionNew
        screen.draw.line(positionOld,positionNew)
        positionOld = positionNew

Now my question: interpolating with straight line segments looks too jagged for my taste, can you recommend a better interpolation method? What method do GIMP or Adobe Photoshop implement?

Alternatively, is there a way to increase the frequency of the mouse move events that I receive? The GUI framework I'm using is wxWidgets.

GUI framework: wxWidgets.
(Programming language: Haskell, but that's irrelevant here)

EDIT: Clarification: I want something that looks smoother than straight line segments, see the picture (original size):

EDIT2: The code I'm using looks like this:

-- create bitmap and derive drawing context
im      <- imageCreateSized (sy 800 600)
bitmap  <- bitmapCreateFromImage im (-1)    -- wxBitmap
dc      <- memoryDCCreate                   -- wxMemoryDC
memoryDCSelectObject dc bitmap

...
-- handle mouse move
onMouse ... sw (MouseLeftDrag posNew _) = do
    ...
    line dc posOld posNew [color     := white
                          , penJoin  := JoinRound
                          , penWidth := 2]
    repaint sw                              -- a wxScrolledWindow

-- handle paint event
onPaint ... = do
    ...
    -- draw bitmap on the wxScrolledWindow
    drawBitmap dc_sw bitmap pointZero False []

which might make a difference. Maybe my choices of wx-classes is why I'm getting a rather low frequency of mouse move events.

Answer 1

Live demos

• version 1 - more smooth, but more changing while you draw: http://jsfiddle.net/Ub7RV/1/
• version 2 - less smooth but more stable: http://jsfiddle.net/Ub7RV/2/

The way to go is 

Spline interpolation of the points

The solution is to store coordinates of the points and then perform spline interpolation.

I took the solution demonstrated here and modified it. They computed the spline after you stop drawing. I modified the code so that it draws immediately. You might see though that the spline is changing during the drawing. For real application, you probably will need two canvases - one with the old drawings and the other with just the current drawing, that will change constantly until your mouse stops.

Version 1 uses spline simplification - deletes points that are close to the line - which results in smoother splines but produce less "stable" result. Version 2 uses all points on the line and produces much more stable solution though (and computationally less expensive).

Answer 2

You can make them really smooth using splines: http://freespace.virgin.net/hugo.elias/graphics/x_bezier.htm

But you'll have to delay the drawing of each line segment until one frame later, so that you have the start and end points, plus the next and previous points available for the calculation.

Answer 3

so, as I see the problem of jagged edge of freehand made curve, when the mouse are moved very fast, is not solved !!! In my opinion there are need to work around with the polling frequency of mousemove event in the system i.e. using different mouse driver or smf.. And the second way is the math.. using some kind of algorithm, to accuratly bend the straight line between two points when the mouse event is polled out.. For clear view you can compare how is drawed free hand line in photoshop and how in mspaint.. thanks folks.. ;)

Answer 4

I agree with harviz - the problem isn't solved. It should be solved on the operating system level by recording mouse movements in a priority thread, but no operating system I know of does that. However, the app developer can also work around this operating system limitation by interpolating better than linear.

Since mouse movement events don't always come fast enough, linear interpolation isn't always enough.

I experimented a little bit with the spline idea brought up by Rocketmagnet.

Instead of putting a line between two points A and D, look at the point P preceding A and use a cubic spline with the following control points B = A + v' and C = D - w', where

v = A - P,
w = D - A,
w' = w / 4 and
v' = v * |w| / |v| / 4.

This means we fall into the second point with the same angle as the line interpolation would, but go out a starting point in the same angle the previous segment came in, making the edge smooth. We use the length of the segment for both control point distances to make the size of the bend fit its proportion.

The following picture shows the result with very few data points (indicated in grey).

The sequence starts at the top left and ends in the middle.

There is still some level of uneasiness here which may be alleviated if one uses both the previous and the next point to adjust for both angles, but that would also mean to draw one point less than what one has got. I find this result already satisfactory, so I didn't try.

Answer 5

I think you need to look into the Device Context documentation for wxWidgets.

I have some code that draws like this:

//screenArea is a wxStaticBitmap
int startx, starty;
void OnMouseDown(wxMouseEvent& event)
{
    screenArea->CaptureMouse();
    xstart = event.GetX();
    ystart = event.GetY();
    event.Skip();
}
void OnMouseMove(wxMouseEvent& event)
{
    if(event.Dragging() && event.LeftIsDown())
    {
        wxClientDC dc(screenArea);
        dc.SetPen(*wxBLACK_PEN);
        dc.DrawLine(startx, starty, event.GetX(), event.GetY());
    }
    startx = event.GetX();
    starty = event.GetY();
    event.Skip();
}

I know it's C++ but you said the language was irrelevant, so I hope it helps anyway.

This lets me do this:

which seems significantly smoother than your example.

Answer 6

Interpolating mouse movements with line segments is fine, GIMP does it that way, too, as the following screenshot from a very fast mouse movement shows:

So, smoothness comes from a high frequency of mouse move events. WxWidgets can do that, as the example code for a related question demonstrates.

The problem is in your code, Heinrich. Namely, drawing into a large bitmap first and then copying the whole bitmap to the screen is not cheap! To estimate how efficient you need to be, compare your problem to video games: a smooth rate of 30 mouse move events per second correspond to 30fps. Copying a double buffer is no problem for modern machines, but WxHaskell is likely not optimized for video games, so it's not surprising that you experience some jitter.

The solution is to draw only as much as necessary, i.e. just the lines, directly on the screen, for example as shown in the link above.