0

I was able to integrate Twilio Video API and audio calling is working as a web app. However, when I try to build it for mobile via Phonegap Build, the audio calling is not working as expected. There were no errors reported via Chrome Dev Tools USB debugging method.

Works:

  • Web to Web calling Web to mobile app (I can hear the call the clearly, from either end)

Not working:

  • Mobile to web (No sound at all, I can only hear the voice of the one calling via web)
  • Mobile to mobile (Nothing at all)

At first, Twilio Video API is not working in Phonegap app and returning undefined on twilio-conversations.min.js. I was able to fix that by adding Crosswalk <plugin name="cordova-plugin-crosswalk-webview" spec="~2.1.0" source="npm" /> as I suspect it has something to do with webrtc. After adding this plugin, the Twilio can now create conversation and connect both participants.

I can't figure out what to do next. I tried adding cordova-plugin-media-capture to capture sound and activate "record audio permission", but still no luck.

Maybe I'm missing something. Any suggestion is very much appreciated. :)

Here's my Phonegap config.xml

<?xml version="1.0" encoding="UTF-8" ?>

<widget xmlns   = "http://www.w3.org/ns/widgets"
    xmlns:gap   = "http://phonegap.com/ns/1.0"
    id          = "com.app.experiment"
    versionCode = "1"
    version     = "1.0.0">

    <name>Twilio Phonegap Experiment</name>

    <description>
            Twilio Phonegap Testing
    </description>
    <author href="https://domainhere.com" email="woppi@domainhere.com">
             woppi
    </author>

    <platform name="android"/>
    <preference name="phonegap-version" value="cli-6.3.0" />
    <preference name="orientation" value="portrait" />
    <preference name="fullscreen" value="false" />

    <preference name="webviewbounce" value="false" />
    <preference name="DisallowOverscroll" value="true" />
    <preference name="disallowOverscroll" value="true" />

    <preference name="android-minSdkVersion" value="21" />
    <preference name="android-installLocation" value="auto" />

    <platform name="android">
        <icon src="res/icon/android/drawable-ldpi-icon" qualifier="ldpi"/>
        <icon src="res/icon/android/drawable-mdpi-icon" qualifier="mdpi"/>
        <icon src="res/icon/android/drawable-hdpi-icon" qualifier="hdpi"/>
        <icon src="res/icon/android/drawable-xhdpi-icon" qualifier="xhdpi"/>
        <icon src="res/icon/android/drawable-xxhdpi-icon" qualifier="xxhdpi"/>
        <icon src="res/icon/android/drawable-xxxhdpi-icon" qualifier="xxxhdpi"/>
    </platform>

      <platform name="android">
        <splash src="res/screen/android/drawable-port-ldpi-screen" qualifier="ldpi"/>
        <splash src="res/screen/android/drawable-port-mdpi-screen" qualifier="mdpi"/>
        <splash src="res/screen/android/drawable-port-hdpi-screen" qualifier="hdpi"/>
        <splash src="res/screen/android/drawable-port-xhdpi-screen" qualifier="xhdpi"/>
        <splash src="res/screen/android/drawable-port-xxhdpi-screen" qualifier="xxhdpi"/>
        <splash src="res/screen/android/drawable-port-xxxhdpi-screen" qualifier="xxxhdpi"/>
      </platform>

      <preference name="permissions" value="none"/>

      <plugin name="cordova-plugin-camera" spec="~2.1.1" source="npm"/>
      <plugin name="cordova-plugin-device" spec="~1.1.1" source="npm"/>
      <plugin name="cordova-plugin-dialogs" spec="~1.2.0" source="npm"/>
      <plugin name="cordova-plugin-geolocation" spec="~2.1.0" source="npm"/>
      <plugin name="cordova-plugin-inappbrowser" spec="~1.3.0" source="npm"/>
      <plugin name="cordova-plugin-network-information" spec="1.2.0" source="npm"/>
      <plugin name="cordova-plugin-splashscreen" spec="~3.2.1" source="npm"/>
      <plugin name="cordova-plugin-statusbar" spec="~2.1.2" source="npm"/>
      <plugin name="cordova-plugin-whitelist" spec="~1.2.1" source="npm"/>
      <plugin name="cordova-plugin-crosswalk-webview" spec="~2.1.0" source="npm" />
      <plugin name="cordova-plugin-media-capture" spec="~1.4.0" source="npm" />

      <access origin="*" subdomains="true"/>
      <allow-intent href="http://*/*"/>
      <allow-intent href="https://*/*"/>
      <allow-intent href="tel:*"/>
      <allow-intent href="sms:*"/>
      <allow-intent href="mailto:*"/>
      <allow-intent href="geo:*"/>
      <platform name="android">
        <allow-intent href="market:*"/>
      </platform>
</widget>
Woppi
  • 5,303
  • 11
  • 57
  • 81
  • Were you ever able to solve this? –  Nov 08 '16 at 21:33
  • @Alexander Pritchard No. Tried lots of combination with audio permission and couldn't think of what else to do. I also emailed video-product@twilio.com weeks ago but no reply until now. Sad to say, have to abandon whatever I accomplished for the phonegap and decided to go the native way, learn android development/java, and then use their Android SDK. I'm just starting again. Oh boy! – Woppi Nov 09 '16 at 03:28
  • I am sorry to hear that. Am currently on a Production-level project with a fixed budget. That is a shame, as this could affect whether we can release this feature for a client without having to eat the costs ourselves. –  Nov 09 '16 at 05:41
  • Yeah, I tried to look for other alternatives to Twilio so I can use the hybrid app still and just change vendor, but even Sinch isn't supporting phonegap as per my enquiry. :) I tried both Android SDK and Sinch has clearer receptionin mobile than Twilio and also stable API. I'll make my decision once I'm on that part of the project. Good luck on your project. :) Glad that my unfortunate experience would help someone out make a decision nyahaha :p – Woppi Nov 09 '16 at 05:57
  • 1
    I followed up with this. using "adb logcat" to get the log from the device, I saw that I needed additional permissions requested in order to access the audio and video devices. (Specifically, MODIFY_AUDIO_SETTINGS before RECORD_AUDIO could actually get a device.) Used this plugin to have all those permissions: https://www.npmjs.com/package/cordova-opentok-android-permissions And then this plugin to ask for permissions at runtime: https://www.npmjs.com/package/cordova-opentok-android-permissions I'll try and give a more thorough answer later. We're busy trying to get this on iOS. –  Nov 21 '16 at 21:33
  • @AlexanderPritchard Nice! Thanks for sharing, maybe i'll give it another go when I have time. I'm also busy building the android app for now. :) – Woppi Nov 22 '16 at 06:50
  • @AlexanderPritchard any update on this? Did you get it working? – cobberboy May 08 '17 at 08:00
  • @cobberboy we resulted in doing native instead of phonegap as the twilio js sdk is not working via phonegap, just web, it is connecting but audio is not audible. – Woppi May 08 '17 at 08:15
  • Thanks @Woppi, good to know. – cobberboy May 08 '17 at 12:17
  • I am really sad to report that I am in the same boat... Everything works great in browser but as soon as I build on device I lose audio. I have all of the necessary permissions set/requested at runtime but no combination of things is working for me. There must be a way! – niczak Jul 03 '17 at 18:35
  • 1
    @NicholasKreidberg yeah that's why we went native... I've tried to come up with a solution for I think two weeks... it was really stressful moment as I've made progress already and this was a major roadblock. Haha. Anyways, good luck. – Woppi Jul 05 '17 at 10:35
  • @Woppi I finally figure out my issue and have both audio/video working on Android using Cordova/Ionic/Twilio-Video-JS, info here: https://stackoverflow.com/questions/35344819/webaudio-not-working-with-ionic-using-crosswalk-on-android-device/35346432?noredirect=1#comment76795416_35346432 – niczak Jul 06 '17 at 12:13

1 Answers1

0

I believe you would need to build a Phonegap plugin for the video SDK to achieve this. There is a project on GitHub for the Twilio Client SDK. I have successfully used this plugin to develop a Cordova app using the Twilio Client for Android and IOS. Maybe you could develop a plugin yourself.

Hope this helps!

Thev
  • 59
  • 12