3

I've been doing some research on the feasibility of building a mobile/web app that allows users to say a phrase and detects the accent of the user (Boston, New York, Canadian, etc.). There will be about 5 to 10 predefined phrases that a user can say. I'm familiar with some of the Speech to Text API's that are available (Nuance, Bing, Google, etc.) but none seem to offer this additional functionality. The closest examples that I've found are Google Now or Microsoft's Speaker Recognition API:

http://www.androidauthority.com/google-now-accents-515684/

https://www.microsoft.com/cognitive-services/en-us/speaker-recognition-api

Because there are going to be 5-10 predefined phrases I'm thinking of using a machine learning software like Tensorflow or Wekinator. I'd have initial audio created in each accent to use as the initial data. Before I dig deeper into this path I just wanted to get some feedback on this approach or if there are better approaches out there. Let me know if I need to clarify anything.

user1034642
  • 67
  • 7

2 Answers2

5

There is no public API for such a rare task.

Accent detection as language detection is commonly implemented with i-vectors. Tutorial is here. Implementation is available in Kaldi.

You need significant amount of data to train the system even if your sentences are fixed. It might be easier to collect accented speech without focusing on the specific sentences you have.

End-to-end tensorflow implementation is also possible but would probably require too much data since you need to separate speaker-instrinic things from accent-instrinic things (basically perform the factorization like i-vector is doing). You can find descriptions of similar works like this and this one.

Nikolay Shmyrev
  • 24,897
  • 5
  • 43
  • 87
3

You could use(this is just an idea, you will need to experiment a lot) a neural network with as many outputs as possible accents you have with a softmax output layer and cross entropy cost function

Luis Leal
  • 3,388
  • 5
  • 26
  • 49