8

Is there any package that I can use to create an app that can process speech to text?

It should include the following features:

  • offline speech to text
  • continuous listening(10-30 minutes)
  • recognizer would work for all languages that Android/IOS supports

So far I found this https://pub.dev/packages/speech_recognition but it says:

the iOS API sends intermediate results, On my Android device, only the final transcription is received

Other limitations: On iOS, by default the plugin is configured for French, English, Russian, Spanish, Italian. On Android, without additional installations, it will probably works only with the default device locale.

Somebody tested this package and had good results? Or do you have any other suggestions?

Chris
  • 6,105
  • 6
  • 38
  • 55

4 Answers4

7

A more straightforward solution using flutter speech_to_text library on version 5.6.1 and without using bloc library as in previous answers.

Basically, whenever the statusListener method is called with the done status we call the Listen method again.

main.dart
import 'package:flutter/material.dart';
import 'package:speech_to_text/speech_recognition_error.dart';
import 'package:speech_to_text/speech_recognition_result.dart';
import 'package:speech_to_text/speech_to_text.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      title: 'Flutter Demo',
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({Key? key}) : super(key: key);

  @override
  MyHomePageState createState() => MyHomePageState();
}

class MyHomePageState extends State<MyHomePage> {
  final SpeechToText _speechToText = SpeechToText();
  bool _speechEnabled = false;
  bool _speechAvailable = false;
  String _lastWords = '';
  String _currentWords = '';
  final String _selectedLocaleId = 'es_MX';

  printLocales() async {
    var locales = await _speechToText.locales();
    for (var local in locales) {
      debugPrint(local.name);
      debugPrint(local.localeId);
    }
  }

  @override
  void initState() {
    super.initState();
    _initSpeech();
  }

  void errorListener(SpeechRecognitionError error) {
    debugPrint(error.errorMsg.toString());
  }

  void statusListener(String status) async {
    debugPrint("status $status");
    if (status == "done" && _speechEnabled) {
      setState(() {
        _lastWords += " $_currentWords";
        _currentWords = "";
        _speechEnabled = false;
      });
      await _startListening();
    }
  }

  /// This has to happen only once per app
  void _initSpeech() async {
    _speechAvailable = await _speechToText.initialize(
        onError: errorListener,
        onStatus: statusListener
    );
    setState(() {});
  }

  /// Each time to start a speech recognition session
  Future _startListening() async {
    debugPrint("=================================================");
    await _stopListening();
    await Future.delayed(const Duration(milliseconds: 50));
    await _speechToText.listen(
        onResult: _onSpeechResult,
        localeId: _selectedLocaleId,
        cancelOnError: false,
        partialResults: true,
        listenMode: ListenMode.dictation
    );
    setState(() {
      _speechEnabled = true;
    });
  }

  /// Manually stop the active speech recognition session
  /// Note that there are also timeouts that each platform enforces
  /// and the SpeechToText plugin supports setting timeouts on the
  /// listen method.
  Future _stopListening() async {
    setState(() {
      _speechEnabled = false;
    });
    await _speechToText.stop();
  }

  /// This is the callback that the SpeechToText plugin calls when
  /// the platform returns recognized words.
  void _onSpeechResult(SpeechRecognitionResult result) {
    setState(() {
      _currentWords = result.recognizedWords;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Speech Demo'),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Container(
              padding: const EdgeInsets.all(16),
              child: const Text(
                'Recognized words:',
                style: TextStyle(fontSize: 20.0),
              ),
            ),
            Expanded(
              child: Container(
                padding: const EdgeInsets.all(16),
                child: Text(
                  _lastWords.isNotEmpty
                      ? '$_lastWords $_currentWords'
                      : _speechAvailable
                      ? 'Tap the microphone to start listening...'
                      : 'Speech not available',
                ),
              ),
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed:
        _speechToText.isNotListening ? _startListening : _stopListening,
        tooltip: 'Listen',
        child: Icon(_speechToText.isNotListening ? Icons.mic_off : Icons.mic),
      ),
    );
  }
}
Sergio G
  • 101
  • 1
  • 6
  • It's really cool! In general, it works fine. But do you also have an idea of fallback strategy? The code you provided doesn't listen anymore after an error occurs. I modified errorListener() as ``` void errorListener(SpeechRecognitionError error) async { debugPrint(error.errorMsg.toString()); if (_speechEnabled) { await _startListening(); } } ``` But, somehow the callstack doesn't come into the errorListener :( Do you have any idea on fallback strategy? – 임정섭 Sep 20 '22 at 09:18
  • Could you explain how to reproduce your error? – Sergio G Sep 22 '22 at 02:09
  • With your code, start the api and don't say anything for a while ( like 10 seconds)! The mic button would still be listening state but the prompt is no more get updated. The app seems to stuck if an error occurs. – 임정섭 Sep 22 '22 at 03:23
  • I/flutter ( 4955): status listening I/flutter ( 4955): status notListening I/flutter ( 4955): status done I/flutter ( 4955): ================================================= I/flutter ( 4955): status listening the app says it's still listening but nothing happens more than 10 mins. – 임정섭 Sep 22 '22 at 08:35
  • I think that's because cancelOnError is set to true, try it with false value. – Faizan Ahmad Sep 22 '22 at 21:51
  • @임정섭 did it work with [Faizan Ahmad suggestion](https://stackoverflow.com/questions/58060889/flutter-dart-speech-to-text-offline-and-continuous-for-any-language#comment130350746_73597058)? – Sergio G Sep 26 '22 at 13:47
  • @SergioG Unfortunately not :( – 임정섭 Sep 28 '22 at 07:47
  • Oh, I rebuilded the app and @FaizanAhmad 's solution seems to work! I add my code. https://gist.github.com/jsrimr/09af9516ee1c5907453635b03ce885ae – 임정섭 Sep 28 '22 at 07:54
  • 2
    Amazing!! I will update the code posted here then – Sergio G Sep 29 '22 at 12:50
  • this is amazing, however when I use this in the main page of my app everything works fine, but when I use it in a "secondary page" it doesn't work. Do you know how I can fix this? – CastoldiG Mar 03 '23 at 17:29
4

I'm using https://pub.dev/packages/speech_to_text now. It is actively maintained and it works pretty good. I think some custom code can be written to make it listen continuously.

Edit:

As requested, please see the continuous listening logic below. I only used it as a proof of concept, so I wouldn't recommend it for production apps. As far as I know the Android API does not support continuous listening out of the box.

SpeechRecognitionBloc

import 'package:bloc/bloc.dart';
import 'package:meta/meta.dart';
import 'package:template_mobile/core/sevices/speech_recognition_service.dart';
import 'package:template_mobile/core/state/event/speech_recognition_event.dart';
import 'package:template_mobile/core/state/state/speech_recognition_state.dart';

class SpeechRecognitionBloc
    extends Bloc<SpeechRecognitionEvent, SpeechRecognitionState> {
  final SpeechRecognitionService speechRecognitionService;

  SpeechRecognitionBloc({
    @required this.speechRecognitionService,
  }) : assert(speechRecognitionService != null) {
    speechRecognitionService.errors.stream.listen((errorResult) {
      add(SpeechRecognitionErrorEvent(
        error: "${errorResult.errorMsg} - ${errorResult.permanent}",
      ));
    });

    speechRecognitionService.statuses.stream.listen((status) {
      if (state is SpeechRecognitionRecognizedState) {
        var currentState = state as SpeechRecognitionRecognizedState;
        if (currentState.finalResult) {
          add(SpeechRecognitionStatusChangedEvent());
        }
      }
    });

    speechRecognitionService.words.stream.listen((speechResult) {
      add(SpeechRecognitionRecognizedEvent(
        words: speechResult.recognizedWords,
        finalResult: speechResult.finalResult,
      ));
    });
  }

  @override
  SpeechRecognitionState get initialState =>
      SpeechRecognitionUninitializedState();

  @override
  Stream<SpeechRecognitionState> mapEventToState(
      SpeechRecognitionEvent event) async* {
    if (event is SpeechRecognitionInitEvent) {
      var hasSpeech = await speechRecognitionService.initSpeech();
      if (hasSpeech) {
        yield SpeechRecognitionAvailableState();
      } else {
        yield SpeechRecognitionUnavailableState();
      }
    }

    if (event is SpeechRecognitionStartPressEvent) {
      yield SpeechRecognitionStartPressedState();
      add(SpeechRecognitionStartEvent());
    }

    if (event is SpeechRecognitionStartEvent) {
      speechRecognitionService.startListening();
      yield SpeechRecognitionStartedState();
    }

    if (event is SpeechRecognitionStopPressEvent) {
      yield SpeechRecognitionStopPressedState();
      add(SpeechRecognitionStopEvent());
    }

    if (event is SpeechRecognitionStopEvent) {
      speechRecognitionService.stopListening();
      yield SpeechRecognitionStopedState();
    }

    if (event is SpeechRecognitionCancelEvent) {
      speechRecognitionService.cancelListening();
      yield SpeechRecognitionCanceledState();
    }

    if (event is SpeechRecognitionRecognizedEvent) {
      yield SpeechRecognitionRecognizedState(
          words: event.words, finalResult: event.finalResult);
      if (event.finalResult == true &&
          speechRecognitionService.statuses.value == 'notListening') {
        await Future.delayed(Duration(milliseconds: 50));
        add(SpeechRecognitionStatusChangedEvent());
      }
    }

    if (event is SpeechRecognitionErrorEvent) {
      yield SpeechRecognitionErrorState(error: event.error);
      // Just for UI updates for the state to propagates
      await Future.delayed(Duration(milliseconds: 50));
      add(SpeechRecognitionInitEvent());
      await Future.delayed(Duration(milliseconds: 50));
      add(SpeechRecognitionStartPressEvent());
    }

    if (event is SpeechRecognitionStatusChangedEvent) {
      yield SpeechRecognitionStatusState();
      add(SpeechRecognitionStartPressEvent());
    }
  }
}

SpeechRecognitionService

import 'dart:async';

import 'package:rxdart/rxdart.dart';
import 'package:speech_to_text/speech_recognition_error.dart';
import 'package:speech_to_text/speech_recognition_result.dart';
import 'package:speech_to_text/speech_to_text.dart';

class SpeechRecognitionService {
  final SpeechToText speech = SpeechToText();

  var errors = StreamController<SpeechRecognitionError>();
  var statuses = BehaviorSubject<String>();
  var words = StreamController<SpeechRecognitionResult>();

  var _localeId = '';

  Future<bool> initSpeech() async {
    bool hasSpeech = await speech.initialize(
      onError: errorListener,
      onStatus: statusListener,
    );

    if (hasSpeech) {
      var systemLocale = await speech.systemLocale();
      _localeId = systemLocale.localeId;
    }

    return hasSpeech;
  }

  void startListening() {
    speech.stop();
    speech.listen(
        onResult: resultListener,
        listenFor: Duration(minutes: 1),
        localeId: _localeId,
        onSoundLevelChange: null,
        cancelOnError: true,
        partialResults: true);
  }

  void errorListener(SpeechRecognitionError error) {
    errors.add(error);
  }

  void statusListener(String status) {
    statuses.add(status);
  }

  void resultListener(SpeechRecognitionResult result) {
    words.add(result);
  }

  void stopListening() {
    speech.stop();
  }

  void cancelListening() {
    speech.cancel();
  }
}
Chris
  • 6,105
  • 6
  • 38
  • 55
  • 1
    Were you able to make it listen continuously? – SIMMORSAL Jul 23 '20 at 20:50
  • Yes, sort of. I was able to smoothly restart the speech after each result/error. Of course it is not ideal, but as far as I understand the Android API does not support continuous recognition. – Chris Jul 24 '20 at 07:44
  • could you please upload your code as a gist or something so I could use it? – SIMMORSAL Jul 24 '20 at 20:51
  • @SIMMORSAL I updated the answer with the requested code. – Chris Jul 25 '20 at 15:08
  • Thank you. Quick question, does this code make it so that each time it activates and deactivates, the default speech sounds play? – SIMMORSAL Jul 25 '20 at 20:10
  • @SIMMORSAL I didn't hear any sound when playing with it. Maybe the phone volume was off :) – Chris Jul 26 '20 at 12:44
  • 1
    Chris could you please provide your full code cause am getting alot of errors, am working on my final school project and i need the feature of able to run speech recognition continues on the background for android and ios, could you please help me and thanks in advance :) – Show Time May 11 '21 at 01:07
  • It missing two files that i need them to test your logic please provide them. – Show Time May 12 '21 at 01:05
2

https://pub.dev/packages/speech_recognition is the best choice. it is based on SpeechRecognizer and offers offline speech to text.

Continuous listening is not possible. Even the paid, online Cloud Speech-to-Text API doesn't allow this, because it is to dangerous (misuse etc).

On iOS, by default the plugin is configured for French, English, Russian, Spanish, Italian, but you can add a missing language into the swift source file.

So finally you will not found a better plugin for speech recognition, even if it is not perfect.

Codr
  • 368
  • 3
  • 12
0

If you're willing to write custom platform specific code https://flutter.dev/docs/development/platform-integration/platform-channels, for Android you can use https://github.com/alphacep/vosk-android-demo to get exactly what you want.

Another alternative is to build your own keyboard IME to use voice typing as suggested in How Can I Implement Google Voice Typing In My Application?.

dionode
  • 26
  • 5