2

My question is the same as this one:

Can I plot the SVM decision boundary for 3 features(in 3D space) using Scikit learn?

One of the answers stated:

Take this example scikit-learn.org/stable/auto_examples/svm/plot_iris.html. If you have 3 features, all you have to do is add a z coordinate to your meshgrid defined similarly as x and y and pass it to your predict method and contourf method.

So I attempted to do just that, convert the 2D example to 3D:

print(__doc__)

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm, datasets

# import some data to play with
iris = datasets.load_iris()
X = iris.data[:, :3]  # we only take the first two features. We could
                      # avoid this ugly slicing by using a two-dim dataset
y = iris.target

h = .02  # step size in the mesh

# we create an instance of SVM and fit out data. We do not scale our
# data since we want to plot the support vectors
C = 1.0  # SVM regularization parameter
svc = svm.SVC(kernel='linear', C=C).fit(X, y)
rbf_svc = svm.SVC(kernel='rbf', gamma=0.7, C=C).fit(X, y)
poly_svc = svm.SVC(kernel='poly', degree=3, C=C).fit(X, y)
lin_svc = svm.LinearSVC(C=C).fit(X, y)

# create a mesh to plot in
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
z_min, z_max = X[:, 2].min() - 1, X[:, 2].max() + 1
xx, yy, zz = np.meshgrid(np.arange(x_min, x_max, h),
                     np.arange(y_min, y_max, h),
                     np.arange(z_min, z_max, h))

# title for the plots
titles = ['SVC with linear kernel',
          'LinearSVC (linear kernel)',
          'SVC with RBF kernel',
          'SVC with polynomial (degree 3) kernel']


for i, clf in enumerate((svc, lin_svc, rbf_svc, poly_svc)):
    # Plot the decision boundary. For that, we will assign a color to each
    # point in the mesh [x_min, x_max]x[y_min, y_max].
    plt.subplot(2, 2, i + 1, projection='3d')
    plt.subplots_adjust(wspace=0.4, hspace=0.4)

    pred = clf.predict(np.c_[xx.ravel(), yy.ravel(), zz.ravel()])

    # Put the result into a color plot
    pred = pred.reshape(xx.shape)
    plt.contourf(xx, yy, zz, pred, cmap=plt.cm.coolwarm, alpha=0.8)

    # Plot also the training points
    plt.scatter(X[:, 0], X[:, 1], X[:, 2], c=y, cmap=plt.cm.coolwarm)
    plt.xlabel('Sepal length')
    plt.ylabel('Sepal width')
    plt.xlim(xx.min(), xx.max())
    plt.ylim(yy.min(), yy.max())
    plt.zlim(zz.min(), zz.max())
    plt.xticks(())
    plt.yticks(())
    plt.zticks(())
    plt.title(titles[i])

plt.show()

But I get the same error every time no matter how I tweak it around:

=========================
3D surface (checkerboard)
=========================

Demonstrates plotting a 3D surface colored in a checkerboard pattern.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-54-b750fe83f0a6> in <module>()
     46     # Put the result into a color plot
     47     pred = pred.reshape(xx.shape)
---> 48     plt.contourf(xx, yy, zz, pred, cmap=plt.cm.coolwarm, alpha=0.8)
     49 
     50     # Plot also the training points

C:\Users\BNielson\Anaconda3\envs\PythonMachineLearning\lib\site-packages\matplotlib\pyplot.py in contourf(*args, **kwargs)
   2873                       mplDeprecation)
   2874     try:
-> 2875         ret = ax.contourf(*args, **kwargs)
   2876     finally:
   2877         ax._hold = washold

C:\Users\BNielson\Anaconda3\envs\PythonMachineLearning\lib\site-packages\mpl_toolkits\mplot3d\axes3d.py in contourf(self, X, Y, Z, *args, **kwargs)
   2194 
   2195         jX, jY, jZ = art3d.rotate_axes(X, Y, Z, zdir)
-> 2196         cset = Axes.contourf(self, jX, jY, jZ, *args, **kwargs)
   2197         self.add_contourf_set(cset, zdir, offset)
   2198 

C:\Users\BNielson\Anaconda3\envs\PythonMachineLearning\lib\site-packages\matplotlib\__init__.py in inner(ax, *args, **kwargs)
   1889                     warnings.warn(msg % (label_namer, func.__name__),
   1890                                   RuntimeWarning, stacklevel=2)
-> 1891             return func(ax, *args, **kwargs)
   1892         pre_doc = inner.__doc__
   1893         if pre_doc is None:

C:\Users\BNielson\Anaconda3\envs\PythonMachineLearning\lib\site-packages\matplotlib\axes\_axes.py in contourf(self, *args, **kwargs)
   5827             self.cla()
   5828         kwargs['filled'] = True
-> 5829         contours = mcontour.QuadContourSet(self, *args, **kwargs)
   5830         self.autoscale_view()
   5831         return contours

C:\Users\BNielson\Anaconda3\envs\PythonMachineLearning\lib\site-packages\matplotlib\contour.py in __init__(self, ax, *args, **kwargs)
    862         self._transform = kwargs.get('transform', None)
    863 
--> 864         self._process_args(*args, **kwargs)
    865         self._process_levels()
    866 

C:\Users\BNielson\Anaconda3\envs\PythonMachineLearning\lib\site-packages\matplotlib\contour.py in _process_args(self, *args, **kwargs)
   1427                 self._corner_mask = mpl.rcParams['contour.corner_mask']
   1428 
-> 1429             x, y, z = self._contour_args(args, kwargs)
   1430 
   1431             _mask = ma.getmask(z)

C:\Users\BNielson\Anaconda3\envs\PythonMachineLearning\lib\site-packages\matplotlib\contour.py in _contour_args(self, args, kwargs)
   1506             args = args[1:]
   1507         elif Nargs <= 4:
-> 1508             x, y, z = self._check_xyz(args[:3], kwargs)
   1509             args = args[3:]
   1510         else:

C:\Users\BNielson\Anaconda3\envs\PythonMachineLearning\lib\site-packages\matplotlib\contour.py in _check_xyz(self, args, kwargs)
   1540 
   1541         if z.ndim != 2:
-> 1542             raise TypeError("Input z must be a 2D array.")
   1543         else:
   1544             Ny, Nx = z.shape

TypeError: Input z must be a 2D array.
Bruce Nielson
  • 753
  • 8
  • 23
  • 1
    You can't use contourf in this case. By adding zz you now have three independent variables + your prediction, so actually four dimensions, which means you have to use color as 4th dimension one way or another. Since you generate a 3d mesh from three variables, you can probably plot your mesh as a scatter with prediction as a colour - I tried your code and scatter worked, although the result is probably not as easy to look at as you expect it to be. – Oleg Medvedyev Jul 16 '17 at 07:28
  • 1
    This comment you reference in the question is not particularly useful, I'd say. Looking at [how contour plots look like in 3D](http://matplotlib.org/mpl_toolkits/mplot3d/tutorial.html#contour-plots) they are not well suited to show decision boundaries. Maybe you first decide on how you want to visualize the data and then ask for help on obtaining the desired result. – ImportanceOfBeingErnest Jul 16 '17 at 08:52
  • @omdv, post your code so I can try it out. – Bruce Nielson Jul 16 '17 at 13:05
  • @ImportanceOfBeingErnest, yes, I think that is the problem, that I only have a vague idea of what I want. But that make sense since really I'm searching for a best approach. I actually want to see the decision boundary across an ensemble, so I need a flexible approach. Let's start with the obvious, though. how do I make it work well for say just Logistic Regression? – Bruce Nielson Jul 16 '17 at 13:06
  • I should add, I'm pretty new with Matplotlib, so I might just be struggling with the concepts there and just need a good example to get me on the right path. – Bruce Nielson Jul 16 '17 at 13:07

0 Answers0