Include a cn1lib in a local codenameone android build?

Question

I am currently considering porting a app that I start developping with react-native to codenameone. For this, I am still checking the feasability and the amount of work it would requiere (as I would have to port or developp some native library binding from react-native to codenameone because codenameone miss some of my needs, like socket.io support for example). The free codenameone build cloud service beeing limited to app of 1Mb, I have to make my test builds locally (with only a few test classes and the use of the google maps cn1lib, my test app is already above the 1Mb limit) Sadly, there is no free documentation on codenameone on how to perform local builds and actually I couldn't find any instructions on internet on how to do it (I only found, on a blog post, some basic and deprecated instructions on how to perform a local iOS build but nothing for Android). So I had to figure it out myself... After some time spent digging into gradle configuration parametters, I finally succeed into building a basic codenameone app localy that works on my android test device. But the problem is that, when I add an external cn1lib (the google maps native cn1lib https://github.com/codenameone/codenameone-google-maps ), my app bug when oppening a screen that depends from this lib. In the android error log, I could find this message:

D/MyApplication(  551): [EDT] 0:0:0,99 - Exception: java.lang.ClassCastException - com.codename1.googlemaps.InternalNativeMapsImpl cannot be cast to com.codename1.system.NativeInterface
W/System.err(  551): java.lang.ClassCastException: com.codename1.googlemaps.InternalNativeMapsImpl cannot be cast to com.codename1.system.NativeInterface
W/System.err(  551):    at com.codename1.system.NativeLookup.create(Unknown Source)
W/System.err(  551):    at com.codename1.googlemaps.MapContainer.<init>(MapContainer.java:171)
W/System.err(  551):    at com.codename1.googlemaps.MapContainer.<init>(MapContainer.java:151)
W/System.err(  551):    at com.tbdlab.testapp.MyApplication.start(MyApplication.java:207)
W/System.err(  551):    at com.tbdlab.testapp.MyApplicationStub.run(MyApplicationStub.java:183)
W/System.err(  551):    at com.codename1.ui.Display.processSerialCalls(Unknown Source)
W/System.err(  551):    at com.codename1.ui.Display.mainEDTLoop(Unknown Source)
W/System.err(  551):    at com.codename1.ui.RunnableWrapper.run(Unknown Source)
W/System.err(  551):    at com.codename1.impl.CodenameOneThread$1.run(Unknown Source)
W/System.err(  551):    at java.lang.Thread.run(Thread.java:818)

I don't really understand why InternalNativeMapsImpl could not be cast into NativeInterface as I looked into the dex file of my compiled apk and all the necessary classes (for android) from the google maps cn1lib are correctly included (So I have com.codenameone.googlemaps.InternalNativeMaps, com.codenameone.googlemaps.InternalNativeMapsImpl and com.codenameone.googlemaps.MapContainer) and so are the codenameone native interface classes they depend on (com.codename1.system.NativeInterface, com.codename1.impl.android.LifecycleListener...). And I decompilled them and the code is correct (I do not use any obfuscation method anyway so there is no real reason why the compiled code would have differ from the source code). There is probably something that I am missing here to make a local codenameone build with the usage of a cn1lib.

So has anyone already succeed into making a local build with the usage of a cn1lib that perform native bindings? If yes, what is the exact procedure? I really hope someone would be able to help, because, at this point, I am seriously considering to stick with react-native (which I am quite pleased with, exept the fact that it is not completely native) or to jump into flutter (or kotlin native) even if I still think codenameone offers many advantages over these other solutions (but not beeing able to perform local builds during the development phase is just a complete no-go for me)

Answer 1

1mb is huge as it can fit the full google maps app and a lot more. It maps to the compiled size of the jar which starts off at 6kb. The whole cn1lib (only a portion of it is packaged) is 40kb. So I would suggest using the build servers for your tests.

Steve built some support for working with native interfaces a few years back here. He stopped maintaining it a bit after we hired him mostly due to lack of time and demand (not because we told him or anything like that). I'm not sure about the status of this but you can use it as a reference to how native interfaces work.

There is also this plugin (direct link here) which I personally didn't try.

Generally a native interface generates an intermediate class that invokes the native implementation directly. The native implementation for all platforms other than Java SE doesn't implement the native interface and shouldn't. I think I explained it somewhere in the docs but explaining it again in the case of Google Maps is super easy.

This is a method from the native interface:

public PeerComponent createNativeMap(int mapId);

This is the same method from the Android implementation class:

public android.view.View createNativeMap(int mapId);

As you can see the return value differs and we need to wrap it in a peer component to abstract that behavior. By avoiding inheritance and casting we get the flexibility of making a more sensible native API.

Here is the class our build server generates for maps, as you can see it's just "glue code":

package com.codename1.googlemaps;

import com.codename1.ui.PeerComponent;

public class InternalNativeMapsStub implements InternalNativeMaps{
    private InternalNativeMapsImpl impl = new InternalNativeMapsImpl();

    public void setShowMyLocation(boolean param0) {
        impl.setShowMyLocation(param0);
    }

    public void setRotateGestureEnabled(boolean param0) {
        impl.setRotateGestureEnabled(param0);
    }

    public void setMapType(int param0) {
        impl.setMapType(param0);
    }

    public int getMapType() {
        return impl.getMapType();
    }

    public int getMaxZoom() {
        return impl.getMaxZoom();
    }

    public int getMinZoom() {
        return impl.getMinZoom();
    }

    public long addMarker(byte[] param0, double param1, double param2, String param3, String param4, boolean param5) {
        return impl.addMarker(param0, param1, param2, param3, param4, param5);
    }

    public void addToPath(long param0, double param1, double param2) {
        impl.addToPath(param0, param1, param2);
    }

    public long finishPath(long param0) {
        return impl.finishPath(param0);
    }

    public void removeMapElement(long param0) {
        impl.removeMapElement(param0);
    }

    public void removeAllMarkers() {
        impl.removeAllMarkers();
    }

    public PeerComponent createNativeMap(int param0) {
        return PeerComponent.create(impl.createNativeMap(param0));
    }

    public void setPosition(double param0, double param1) {
        impl.setPosition(param0, param1);
    }

    public void calcScreenPosition(double param0, double param1) {
        impl.calcScreenPosition(param0, param1);
    }

    public int getScreenX() {
        return impl.getScreenX();
    }

    public int getScreenY() {
        return impl.getScreenY();
    }

    public void calcLatLongPosition(int param0, int param1) {
        impl.calcLatLongPosition(param0, param1);
    }

    public double getScreenLat() {
        return impl.getScreenLat();
    }

    public double getScreenLon() {
        return impl.getScreenLon();
    }

    public void deinitialize() {
        impl.deinitialize();
    }

    public float getZoom() {
        return impl.getZoom();
    }

    public void setZoom(double param0, double param1, float param2) {
        impl.setZoom(param0, param1, param2);
    }

    public double getLatitude() {
        return impl.getLatitude();
    }

    public double getLongitude() {
        return impl.getLongitude();
    }

    public long beginPath() {
        return impl.beginPath();
    }

    public void initialize() {
        impl.initialize();
    }

    public boolean isSupported() {
        return impl.isSupported();
    }

}

About socket.io you can probably just wrap the JavaScript version with a call to the BrowserComponent to get the native JS code working as a start. A full on native port can come later.

It seems you have cn1libs figured out otherwise but just for completeness this is how they are supposed to work:

The cn1lib is just a zip file containing other zip files for each platform. Refresh libs unzips the this and arranges the files in the appropriate directories under lib/impl. So you need to package the lib/impl directory matching the platform you are trying to compile with your distribution.

cn1libs also include two additional property files codenameone_library_appended.properties & codenameone_library_required.properties. Refresh libs will handle that automatically for you by setting these values into the build hints. The former values are appended to the existing build hint and the latter override an existing build hint.

Build hints effectively tell the build servers how to compile some things e.g. if we want to inject stuff into the plist, manifest etc. How this maps to a local build will vary a lot. In some cases like plistInject it would be trivial to understand but other cases might be odd. If you have a question about how a specific build hint maps to local build then you can ask that.

Answer 2

As said, in some of my tests (where I use the full set of cn1libs I would need + some custom libs), I am already above the 1Mb limitation (the server rejected my test builds for this reason). So using the free build cloud server during the development phase is not an option for me (Anyway I won't use a solution if I am not sure I can be completely independent if necessary. To make my release builds I would certainly take a subscription and use the cloud buid server as it is far more convenient than tweeking a local server, furthermore that I don't own a Mac computer (I only have a test iphone) and need to borrow one when I want to make some iOS build ;) . But I want to be sure that if, for any reason, your service dissapear, I will still be able to make my builds. Furthermore, I don't see the point of paying a subscription during the developpment phase of my app (that could take me months) especially as I am not certain I would use codenameone as a final solution (I still have to check the amount of work it would requiere to adapt some of the libs I already have for react-native to codenameone)). That is the reason why I try to make a local build.

Concerning the socket.io library, I already started to create a cn1lib that will use native solutions (https://github.com/socketio/socket.io-client-java for android, https://github.com/socketio/socket.io-client-swift for iOS, and the original socket.io lib for javascript). This is not really an issue and it was just to give an example of libraries I would have to create in codenameone if I want to switch from react-native.

In what concerns how cn1lib works, I already figured that out, I included into my android project all the necessary class of the cn1 google-maps lib (so I included the content from main.zip, nativeand.zip and stubs.zip in my project) and checked in the .dex files of my generated apk that they are actually correctly packaged in them, as already said. So my problem doesn't seems to be that I forgot to include some class of the cn1lib in my project but something else. The error message is: Exception: java.lang.ClassCastException - com.codename1.googlemaps.InternalNativeMapsImpl cannot be cast to com.codename1.system.NativeInterface so it doesn't refer to a Class not found but to a cast exception... I don't really know what can cause this issue. I took the codenameone core classes from here https://github.com/codenameone/CodenameOne/tree/master/CodenameOne/src/ https://github.com/codenameone/CodenameOne/tree/master/Ports/Android http://github.com/codenameone/codenameone-skins

to include them in my project, so I think I didn't miss one. And when building a project that doesn't use a cn1lib (like a simple "hello word" app), it compiles and run just fine on my android test device. The problem, is really just when my app try to create a googlemap view, where it returns the cast exception (and then default to try to create an html browser mapview and fails here as it is missing some html file). So it is probably a configuration problem ( may it be a problem with the java version used by the compiler as native class files where already compiled in the cn1lib main.zip file?) Here is the gradle build file I use:

buildscript {
        repositories {
            jcenter()
            maven { url "https://plugins.gradle.org/m2/" }
            google()
        }
        dependencies {
            classpath 'com.android.tools.build:gradle:3.0.1'
            classpath 'me.tatarka:gradle-retrolambda:3.2.0'
        }
    }


    allprojects {
        repositories {
            jcenter()
            google()
        }
    }

    apply plugin: 'com.android.application'
    apply plugin: 'me.tatarka.retrolambda'


    android {
        compileSdkVersion 26
        buildToolsVersion '26.0.2'
        dexOptions {
            // Prevent OutOfMemory with MultiDex during the build phase
            javaMaxHeapSize "4g"
        }
        lintOptions {
            checkReleaseBuilds false
        }

        defaultConfig {
            applicationId "com.tbdlab.testapp"
            minSdkVersion 15
            targetSdkVersion 23
            versionCode 1
            versionName "1.0"
            multiDexEnabled true
            testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
        }
        buildTypes {
            release {
                minifyEnabled false
                proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro' //my proguard files are actually empty so no obfuscation is performed. I checked it in the generated apk
            }
        }
        compileOptions {
            sourceCompatibility JavaVersion.VERSION_1_7//.VERSION_1_8
            targetCompatibility JavaVersion.VERSION_1_7//.VERSION_1_8
        }
    }
    dependencies {
        compile fileTree(include: ['*.jar'], dir: 'libs')
        androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
                exclude group: 'com.android.support', module: 'support-annotations'
            })
        compile 'com.google.android.gms:play-services:9.4.0' //compile 'com.google.android.gms:play-services-maps:11.8.0'
        compile 'com.android.support:multidex:1.0.1'
    }

and here is my AndroidManifest.xml file where I included all the permissions defined in the cn1lib:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" /> 
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" /> 
<!--- Permissions requiered by the google maps cn1lib -->
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/> 
<uses-permission android:name="com.google.android.providers.gsf.permission.READ_GSERVICES"/>
<uses-feature android:glEsVersion="0x00020000" android:required="true"/>

<application android:allowBackup="true"  android:icon="@drawable/icon" android:label="MyApplication" android:name="android.support.multidex.MultiDexApplication">
<meta-data android:name="com.google.android.gms.version" android:value="@integer/google_play_services_version"/>
<meta-data android:name="com.google.android.maps.v2.API_KEY" android:value="...masked_it_but_put_my_correct_key_here..."/>
<activity  android:label="MyApplication" android:launchMode="singleTop" android:name="com.tbdlab.testapp.MyApplicationStub" >
    <intent-filter>
        <action android:name="android.intent.action.MAIN"/>
        <category android:name="android.intent.category.LAUNCHER"/>
    </intent-filter>
</activity>
<receiver android:name="com.codename1.impl.android.LocalNotificationPublisher"/>
<service android:exported="false" android:name="com.codename1.impl.android.BackgroundFetchHandler"/>
<activity android:name="com.codename1.impl.android.CodenameOneBackgroundFetchActivity" android:theme="@android:style/Theme.NoDisplay"/>
<activity android:name="com.codename1.location.CodenameOneBackgroundLocationActivity" android:theme="@android:style/Theme.NoDisplay"/>
<service android:exported="false" android:name="com.codename1.location.BackgroundLocationHandler"/>
<service android:exported="false" android:name="com.codename1.location.GeofenceHandler"/>
<service android:exported="false" android:name="com.codename1.media.AudioService"/>
<activity android:excludeFromRecents="true" android:exported="false" android:name="com.google.android.gms.auth.api.signin.internal.SignInHubActivity" android:theme="@android:style/Theme.Translucent.NoTitleBar"/>
<provider android:authorities="com.tbdlab.testapp.google_measurement_service" android:exported="false" android:name="com.google.android.gms.measurement.AppMeasurementContentProvider"/>
<receiver android:enabled="true" android:name="com.google.android.gms.measurement.AppMeasurementReceiver">
    <intent-filter>
        <action android:name="com.google.android.gms.measurement.UPLOAD"/>
    </intent-filter>
</receiver>
<service android:enabled="true" android:exported="false" android:name="com.google.android.gms.measurement.AppMeasurementService"/>
<activity android:name="com.google.android.gms.ads.AdActivity" android:theme="@android:style/Theme.Translucent"/>

I really don't see what can cause the cast exception, but in the lack of a basic tutorial on how to create local builds, I may have miss something without even knowing it...

For this test I made a really simple app than only display a native google map and it runs correctly in the simulator and compiles on the build cloud server and runs fine in my android test device. So the issue is either in my gradle build configuration (or maybe AndroidManifest.xml file even if I don't think it has any effect on the JVM) or in the codenameone core and cn1lib I included in my android project for the local build.