4

Hope there are some neural network heads out here who can help with this! (:

Background: I'm using an accelerometer to draw shapes in the air, and I'm wondering if it's possible to use a neural network to detect what shape I've drawn.

Let's assume I have a predefined 'vocabulary' of 3 possible shapes (say, circle, oval, figure of eight). I draw many circles, collect the resultant accelerometer data, and preprocess it to produce a set S of N (x,y,z) vectors. (The x, y, z are acceleration values.)

Finally, I have a neural network with 3 boolean outputs (circle, oval, fig-8).

My question: What should my inputs be?

Only solution I can think of: Have 3N inputs (x1, y1, z1, x2, y2, z2, ..., xN, yN, zN).

My concern: Such a solution doesn't tell the neural network that x1, y1, z1 are related to each other, that they have the same timestamp. From what I understand, the neural network should be given information about this 'relationship' in some way. Is that necessary? If yes, is it possible to do this in this specific case, and how? Is a neural network completely the wrong solution for this problem of shape-detection-from-accelerometer-data?

awreccan
  • 671
  • 1
  • 7
  • 11
  • Do you have any experience with Recurrent Neural Networks? They belong to a class of networks which were designed to handle *time series* which is what you're aiming for. – jorgenkg Oct 30 '13 at 18:25
  • I've read up on RNNs. It sounds like RNNs would be useful for predicting/forecasting time series. But my problem is about _identifying/categorising_ time series: it's similar to the NN being trained with three stock price charts (A, B, C), and given another stock price chart, being asked to categorise it as _most similar to_ stock A/B/C. To further complicate matters, my data is 3-dimensional. Any ideas on how this can be accomplished? – awreccan Nov 02 '13 at 03:05
  • Neural networks, including RNNs, are great classifiers. What I believe you intend to classify different input sequences (detected motions) as different patterns. Thus a network would be able to tell you if the current stream of data is similar to your dataset (defined motion patterns). Regarding the 3D data: take a look at the figure below. Input(x,y,z) are fed at the same time. – jorgenkg Nov 03 '13 at 16:28

1 Answers1

1

How it could be implemented

Recurrent network

A few pointers

This will hopefully help you along the path of solving this problem by using Neural Networks! If you decide to use these networks, I would recommend you to look into eg: Long Short Time Memory (the LSTM Neural Network ) and specifically to read this thesis.

Time series

The network will be able to pay respect to the previous vector inputs from the sensory data, but as you might guess: we cannot add indefinitely many layers. In fact, recurrent networks will start to struggle if we add too many layers. This problem is also discussed in the thesis I linked to.

LSTM

The LSTM Neural Network is specifically designed to be able to "recognize" different input patterns. Which would comply to your request of recognizing a smaller set of gestures.

Community
  • 1
  • 1
jorgenkg
  • 4,140
  • 1
  • 34
  • 48
  • Thanks for the links! But I don't get the rationale behind the your suggested implementation. Does it mean that I should train each layer individually (with inputs at t=T and outputs at t=T-1)? If yes, why should I let the network provide me possibly erroneous data for t=T-1 when I have the real data ready? – awreccan Nov 01 '13 at 20:26
  • No, this is not how a RNN works. You could look at it as a ordinary NN, but with the input data fed as a stream instead of increasing the number of inputs. – jorgenkg Nov 03 '13 at 16:29
  • I've done a little more research on RNNs. (I had put this project on the backburner for a while, but I'm back on it now.) I'm still trying to understand what exactly you are recommending. (It's probably because of my lack of knowledge, nothing to do with your answer.) Am I understanding correctly that you're recommending an RNN such as in this [**image**](http://www.cse.unsw.edu.au/~waleed/phd/html/img142.png), but with multiple context layers, such as in this [**image**](http://i39.tinypic.com/25f0n51.png)? – awreccan Nov 30 '13 at 09:39
  • If that's the case, isn't that effectively a sliding window on the hidden layer instead of the inputs? If so, wouldn't the past inputs (known with certainty) be preferable that past hidden layer outputs? – awreccan Nov 30 '13 at 09:43