Straighten B-Spline

Question

I've interpolated a spline to fit pixel data from an image with a curve that I would like to straighten. I'm not sure what tools are appropriate to solve this problem. Can someone recommend an approach?

Here's how I'm getting my spline:

import numpy as np
from skimage import io
from scipy import interpolate
import matplotlib.pyplot as plt
from sklearn.neighbors import NearestNeighbors
import networkx as nx

# Read a skeletonized image, return an array of points on the skeleton, and divide them into x and y coordinates
skeleton = io.imread('skeleton.png')
curvepoints = np.where(skeleton==False)
xpoints = curvepoints[1]
ypoints = -curvepoints[0]

# reformats x and y coordinates into a 2-dimensional array
inputarray = np.c_[xpoints, ypoints]

# runs a nearest neighbors algorithm on the coordinate array
clf = NearestNeighbors(2).fit(inputarray)
G = clf.kneighbors_graph()
T = nx.from_scipy_sparse_matrix(G)

# sorts coordinates according to their nearest neighbors order
order = list(nx.dfs_preorder_nodes(T, 0))
xx = xpoints[order]
yy = ypoints[order]

# Loops over all points in the coordinate array as origin, determining which results in the shortest path
paths = [list(nx.dfs_preorder_nodes(T, i)) for i in range(len(inputarray))]

mindist = np.inf
minidx = 0

for i in range(len(inputarray)):
    p = paths[i]           # order of nodes
    ordered = inputarray[p]    # ordered nodes
    # find cost of that order by the sum of euclidean distances between points (i) and (i+1)
    cost = (((ordered[:-1] - ordered[1:])**2).sum(1)).sum()
    if cost < mindist:
        mindist = cost
        minidx = i

opt_order = paths[minidx]

xxx = xpoints[opt_order]
yyy = ypoints[opt_order]

# fits a spline to the ordered coordinates
tckp, u = interpolate.splprep([xxx, yyy], s=3, k=2, nest=-1)
xpointsnew, ypointsnew = interpolate.splev(np.linspace(0,1,270), tckp)

# prints spline variables
print(tckp)

# plots the spline
plt.plot(xpointsnew, ypointsnew, 'r-')
plt.show()

My broader project is to follow the approach outlined in A novel method for straightening curved text-lines in stylistic documents. That article is reasonably detailed in finding the line that describes curved text, but much less so where straightening the curve is concerned. I have trouble visualizing the only reference to straightening that I see is in the abstract:

find the angle between the normal at a point on the curve and the vertical line, and finally visit each point on the text and rotate by their corresponding angles.

I also found Geometric warp of image in python, which seems promising. If I could rectify the spline, I think that would allow me to set a range of target points for the affine transform to map to. Unfortunately, I haven't found an approach to rectify my spline and test it.

Finally, this program implements an algorithm to straighten splines, but the paper on the algorithm is behind a pay wall and I can't make sense of the javascript.

Basically, I'm lost and in need of pointers.

Update

The affine transformation was the only approach I had any idea how to start exploring, so I've been working on that since I posted. I generated a set of destination coordinates by performing an approximate rectification of the curve based on the euclidean distance between points on my b-spline.

From where the last code block left off:

# calculate euclidian distances between adjacent points on the curve
newcoordinates = np.c_[xpointsnew, ypointsnew]
l = len(newcoordinates) - 1
pointsteps = []
for index, obj in enumerate(newcoordinates):
    if index < l:
        ord1 = np.c_[newcoordinates[index][0], newcoordinates[index][1]]
        ord2 = np.c_[newcoordinates[index + 1][0], newcoordinates[index + 1][1]]
        length = spatial.distance.cdist(ord1, ord2)
        pointsteps.append(length)

# calculate euclidian distance between first point and each consecutive point
xpositions = np.asarray(pointsteps).cumsum()

# compose target coordinates for the line after the transform
targetcoordinates = [(0,0),]
for element in xpositions:
    targetcoordinates.append((element, 0))

# perform affine transformation with newcoordinates as control points and   targetcoordinates as target coordinates

tform = PiecewiseAffineTransform()
tform.estimate(newcoordinates, targetcoordinates)

I'm presently hung up on errors with the affine transform (scipy.spatial.qhull.QhullError: QH6154 Qhull precision error: Initial simplex is flat (facet 1 is coplanar with the interior point) ), but I'm not sure whether it's because of a problem with how I'm feeding the data in, or because I'm abusing the transform to do my projection.

Answer 1

I got the same error with you when using scipy.spatial.ConvexHull. First, let me explain my project: what i wanted to do is to segment the people from its background(image matting). In my code, first I read an image and a trimap, then according to the trimap, I segment the original image to foreground, bakground and unknown pixels. Here is part of the coed:

img = scipy.misc.imread('sweater_black.png') #color_image

trimap = scipy.misc.imread('sw_trimap.png', flatten='True') #trimap

bg = trimap == 0 #background

fg = trimap == 255 #foreground

unknown = True ^ np.logical_or(fg,bg) #unknown pixels

fg_px = img[fg] #here i got the rgb value of the foreground pixels,then send them to the ConvexHull

fg_hull = scipy.spatial.ConvexHull(fg_px)

But i got an error here.So I check the Array of fg_px and then I found this array is n*4. which means every scalar i send to ConvexHull has four values. Howerver, the input of ConvexHUll should be 3 dimension. I source my error and found that the input color image is 32bits(rgb channel and alpha channel) which means it has an alpha channel. After transferring the image to 24 bit (which means only rgb channels), the code works.

In one sentence, the input of ConvexHull should be b*4, so check your input data! Hope this works for you~