Google speech to text with java and twilio

Question

I am having problem while converting audio file to text using google speech to text. I am able to download the file from Twilio but when I supply that audio file to google speech then it gives me 0 length response. But if I convert this downloaded file using vlc media player and then supply it to google speech then it gives me right output. Please help me on this I am stuck for about a week now.

After getting response from Twilio I save it in a file with .wav extension

InputStream in = new URL(jsonObject.get("redirect_to").toString()).openStream();
Files.copy(in, Paths.get("src/main/resources/mp.wav"), StandardCopyOption.REPLACE_EXISTING);

Below is the google speech to text code.

Path path = Paths.get("src/main/resources/mp.wav");
        byte[] content = Files.readAllBytes(path);
        ByteString audioBytes = ByteString.copyFrom(content);

        try (SpeechClient speech = SpeechClient.create()) {
            RecognitionConfig recConfig =
                    RecognitionConfig.newBuilder()
                            .setEncoding(RecognitionConfig.AudioEncoding.LINEAR16)
                            .setLanguageCode("en-US")
                            .setSampleRateHertz(44100)
                            .setModel("default")
                            .setAudioChannelCount(2)
                            .build();


            RecognitionAudio recognitionAudio = RecognitionAudio.newBuilder().setContent(audioBytes).build();

            OperationFuture<LongRunningRecognizeResponse, LongRunningRecognizeMetadata> response =
                    speech.longRunningRecognizeAsync(recConfig, recognitionAudio);

            while (!response.isDone()) {
                System.out.println("Waiting for response...");
                Thread.sleep(10000);
            }

            List<SpeechRecognitionResult> results = response.get().getResultsList();

            for (SpeechRecognitionResult result : results) {
                // There can be several alternative transcripts for a given chunk of speech. Just use the
                // first (most likely) one here.
                SpeechRecognitionAlternative alternative = result.getAlternativesList().get(0);
                System.out.printf("Transcription: %s%n", alternative.getTranscript());
            }

        } catch (InterruptedException | ExecutionException e) {
            e.printStackTrace();
        }

Answer 1

As @philnash has suggested, by appending a .mp3 extension to the recording URL, the MP3 version of the recording can be downloaded from Twilio. The same applies to the '.wav' extension as well.

InputStream in = new URL(jsonObject.get(“redirect_to”).toString()+”.mp3”).openStream(); // or “.wav”
Files.copy(in, Paths.get(“src/main/resources/mp.wav”), StandardCopyOption.REPLACE_EXISTING);

I tested this out with a sample Twilio recording and the ffprobe results are below.

Downloaded .wav file

Input #0, **wav**, from 'from-twilio-change-extension.wav':
  Duration: 00:00:14.60, bitrate: 128 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 8000 Hz, 1 channels, s16, 128 kb/s

Downloaded .mp3 file

Input #0, **mp3**, from 'from-twilio-change-extension.mp3':
  Duration: 00:00:14.68, start: 0.000000, bitrate: 32 kb/s
    Stream #0:0: Audio: mp3, 22050 Hz, mono, fltp, 32 kb/s

As for audio encodings supported by the Speech-to-Text API, both WAV and MP3 are supported but MP3 is a Beta feature available only in the version v1p1beta1. So, the client library imports will look like com.google.cloud.speech.v1p1beta1.Packages.... The audio encoding in RecognitionConfig has to be changed according to the encoding of the audio file used. For a .wav file, RecognitionConfig.AudioEncoding.LINEAR16 has to be used, and for a .mp3 file, RecognitionConfig.AudioEncoding.MP3 has to be used.

---

An alternative would be to use the FFMPEG tool to convert audio files into one of the recognized codecs by Speech-to-Text. More information about usage of the tool can be found here. In your scenario, the .mka to .wav/.mp3 conversion can be done from the Java code as shown below.

String[] ffmpegCommand = {"ffmpeg", "-i", "/full/path/to/inputFile.mka", "/full/path/to/outputFile.wav"};

ProcessBuilder pb = new ProcessBuilder(ffmpegCommand);
pb.inheritIO();
pb.start();