0

I am trying to make a react native app in which I want to check if the user pronounced the word/words correctly or not. For this task, I am using react-native-voice library and the user's speech is converted into text using it and the results are stored in presults (useState). But I do not know how and where in the code I should put the if else condition to check if the text in presults match the required string and display the message accordingly.

Here's the code App.js:

import React, {useState, useEffect} from 'react';

import {
  SafeAreaView,
  StyleSheet,
  Text,
  View,
  Alert,
  Image,
  TouchableHighlight,
  ScrollView,
} from 'react-native';

import Voice from 'react-native-voice';

const App = () => {

   const [error, setError] = useState('');
  const [presults, setPartialResults] = useState('');

  useEffect(() => {
    //Setting callbacks for the process status
    Voice.onSpeechStart = onSpeechStart;
    Voice.onSpeechEnd = onSpeechEnd;
    Voice.onSpeechError = onSpeechError;
   Voice.onSpeechPartialResults = onSpeechPartialResults;
 

    return () => {
      //destroy the process after switching the screen
      Voice.destroy().then(Voice.removeAllListeners);
    };
  }, []);

  const onSpeechStart = (e) => {
    //Invoked when .start() is called without error
    console.log('onSpeechStart: ', e);

  };

  const onSpeechEnd = (e) => {
    
    console.log('onSpeechEnd: ', e);
    //Invoked when SpeechRecognizer stops recognition
    
  
  };

  const onSpeechError = (e) => {
    //Invoked when an error occurs.
    console.log('onSpeechError: ', e);
    setError(JSON.stringify(e.error));
  };

  

  const onSpeechPartialResults = (e) => {
    //Invoked when any results are computed
    console.log('onSpeechPartialResults: ', e);
    setPartialResults(e.value);
   
  };


  const startRecognizing = async () => {
    //Starts listening for speech for a specific locale
    try {
      await Voice.start('tr-TURKEY');
      setError('');
     setPartialResults('');
      
    } catch (e) {
      //eslint-disable-next-line
      console.error(e);
    }
  };

  


  const destroyRecognizer = async () => {
    //Destroys the current SpeechRecognizer instance
    try {
      await Voice.destroy();
      setError('');
     setPartialResults('');
    } catch (e) {
      //eslint-disable-next-line
      console.error(e);
    }
  };

  return (
    <SafeAreaView style={styles.container}>
      <View style={styles.container}>
        <Text style={styles.titleText}>
          Speech to Text Conversion in React Native |
          Voice Recognition
        </Text>
        <Text style={styles.textStyle}>
          Press mike to start Recognition
        </Text>
      
        <View style={styles.headerContainer}>
         
          <Text style={styles.textWithSpaceStyle}>
            {`Error: \n ${error}`}
          </Text>
        </View>
        <TouchableHighlight onPress={() => {startRecognizing();}}>
          <Image
            style={styles.imageButton}
            source={{
              uri:          'https://raw.githubusercontent.com/AboutReact/sampleresource/master/microphone.png',
            }}
          />
        </TouchableHighlight>
        <Text style={styles.textStyle}>
          Partial Results
        </Text>
              <Text
                style={styles.textStyle}>
                {presults}
              </Text>
            
      <View style={styles.horizontalView}>
         
          <TouchableHighlight
            onPress={destroyRecognizer}
            style={styles.buttonStyle}>
            <Text style={styles.buttonTextStyle}>
              Cancel
            </Text>
          </TouchableHighlight>
        </View>
      </View>
    </SafeAreaView>
  );
};

export default App;

I put the if else condition in onSpeechEnd function,but the results are not accurate. Here's the modified onSpeechEnd function:

const onSpeechEnd = (e) => {
    if(presults == 'tamam'){
      Alert.alert('c','c');
      console.log('c');
    }
      else {
        Alert.alert('w','w');
        console.log('w');
      }
    
    console.log('onSpeechEnd: ', e);
    //Invoked when SpeechRecognizer stops recognition
    
  
  };

and here's the console result:

 LOG  onSpeechStart:  {"error": false}
 LOG  onSpeechStart:  {"error": false}
 LOG  onSpeechPartialResults:  {"value": [""]}
 LOG  onSpeechPartialResults:  {"value": [""]}
 LOG  onSpeechPartialResults:  {"value": [""]}
 LOG  onSpeechPartialResults:  {"value": [""]}
 LOG  onSpeechPartialResults:  {"value": ["Merhaba"]}
 LOG  w
 LOG  onSpeechEnd:  {"error": false}
 LOG  onSpeechStart:  {"error": false}
 LOG  onSpeechStart:  {"error": false}
 LOG  onSpeechPartialResults:  {"value": [""]}
 LOG  onSpeechPartialResults:  {"value": [""]}
 LOG  onSpeechPartialResults:  {"value": ["tamam"]}
 LOG  w
 LOG  onSpeechEnd:  {"error": false}
 LOG  onSpeechStart:  {"error": false}
James Z
  • 12,209
  • 10
  • 24
  • 44

1 Answers1

0

Root Case

The below line from the log

LOG  onSpeechPartialResults:  {"value": ["tamam"]}

which is the result of:

console.log('onSpeechPartialResults: ', e);

suggests that e is an object which has one prop value. This prop has a value which is an array: ["tamam"].

The line used to store this info is:

setPartialResults(e.value);

So presults will be an Array. Now, the if tries to compare a string "tamam" with the array: ["tamam"] and since these do not match, it executes the log from the else part (ie, w).

Solution

While this may be handled in a number of ways, the below is what comes to mind as being quick, simple.

Please try the below if statement in a separate useEffect:

useEffect(() => {
  console.log('presults changed to: ', presults);
  if(presults.includes('tamam')) {
    console.log('c');
  } else {
    console.log('w');
  }
}, [presults]);

This will check for the presence of the 'okay' (ie, tamam) in the presults array. So, tamam will evaluate to truthy, while the 'hi' (merhaba) will still evaluate to falsy. Since this if-else is placed within a separate useEffect (with dependency array being set to presults), it will execute when presults changes.

jsN00b
  • 3,584
  • 2
  • 8
  • 21
  • Unfortunately, it's still executing the else part even after I changed the if condition – Mariam Ikram Apr 16 '22 at 15:45
  • 1
    Hmm. In that case, let us investigate why. Would you please add to `onSpeechEnd` a `console.log('presults at onSpeechEnd bgn: ', presults);`. Let us see what is the value populated in `presults` when `onSpeechEnd` starts executing. – jsN00b Apr 16 '22 at 16:29
  • ops! there's no value in presults at the beginning of onSpeechEnd. Following is the console output: – Mariam Ikram Apr 16 '22 at 17:37
  • LOG onSpeechStart: {"error": false} LOG onSpeechPartialResults: {"value": [""]} LOG onSpeechPartialResults: {"value": [""]} LOG onSpeechPartialResults: {"value": ["tamam"]} LOG presults at onSpeechEnd bgn: LOG onSpeechEnd: {"error": false} – Mariam Ikram Apr 16 '22 at 17:37
  • Answer has been edited. Please see **_`Please try the below if statement in a separate useEffect`_** in the answer. – jsN00b Apr 17 '22 at 00:29