Rendering clickable mathematics on Android

Question

I'm trying to create an Android app for visually manipulating algebra. Is there any library I could use that would render selectable, as in clickable, algebra? The ideal library would take some symbolic algebra, e.g. in MathML form, render it onscreen, and return onTouch events or similar which indicated which part of the rendered algebra had been touched.

For instance, in the quadratic formula, a user could touch the 4ac in the square root, and this library would tell me that the 4ac in the square root had been touched, so I could then highlight the 4ac to let the user know it was selected, and copy it independently of the surrounding algebra. Another example would be selecting one term in an equation, so that the application could then modify or replace it.

Any equivalent would be fine; a library that could return a graphical representation of an equation, and also identify which part of that equation had been touched symbolically given click co-ordinates on the graphic, would be fine. Essentially I'm looking for a MathML (or similar) renderer that knows exactly what each bit of its rendering represents.

I've had a look at symja, sympy, and MathJax, and I haven't found any way of doing this, although I did get a bit lost. Does this already exist somewhere?

Thanks in advance.

Answer 1

The most direct route would be to add 'onClick' properties directly into the MathML you use, and use element ids & Java/JavaScript method binding from there. However as far as I know, the 'onClick' attribute is not (yet?) supported in MathML.

What I have done as a hack-around is to add links to each of your clickable MathML elements, and then overload the shouldOverrideUrlLoading() method of the WebViewClient for your WebView, to catch the links that are called. Then if you add unique ID-values somewhere in your links, you can identify which element was clicked.

Change your uploaded MathML

If your original MathML looks like:

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block">
        <msqrt>
          <msup><mi>b</mi><mn>2</mn></msup>
          <mo>−</mo>
          <mn>4</mn><mi>a</mi><mi>c</mi>
        </msqrt>
</math>

change it to something like:

<math xmlns="http://www.w3.org/1998/Math/MathML" display="block">
        <msqrt href=id1>
          <msup><mi>b</mi><mn>2</mn></msup>
          <mo>−</mo>
          <mstyle href=id2><mn>4</mn><mi>a</mi><mi>c</mi></mstyle>
        </msqrt>
</math>

Note on nesting links. When using the HTML-CSS output for MathJax, only the lowest link is actively clicked. However, when using SVG output, every link is called in order from lowest level to highest level. So for what I propose, you'll have to use HTML-CSS output from MathJax. If I find a way to make SVG work, I'll make an edit.

Overload shouldOverrideUrlLoading()

This is described in a number of StackOverflow questions on this, and you can reference the Android Devs site for specifics (here) so I won't go into too much detail. Basically, shouldOverrideUrlLoading() will have a url String with the link you specified for the MathML elements that were just clicked. Parse that for the ID value and send it to a function that can use it, and you're good to go.

Here's the code I've used in my project.

public class MathmlLinksViewClient extends WebViewClient {

    //protected MainActivity activity;
    public MathmlLinksViewClient(MainActivity context) {
        activity = context;
    }
    
    @Override
    public boolean shouldOverrideUrlLoading(WebView view, String url) {
        // Reroutes id int value to MainActivity method
        activity.onMathmlClick(
                Integer.valueOf(
                            // Method I wrote to parse ID from URL
                            HtmlIdFormat.getIdFromString(url) ) );
        return true;
    }

}

Note that the url that is passed to this function is not exactly the link that you include in your MathML elements. each of these is prepended by the host link specified when using WebView.loadDataWithBaseUrl() method for your WebView. (I'm assuming you are following this guide to render MathML.)

If your view operates any other HTML links, you'll want to add an if statement to separate the actions on MathML-element links, and website links. Again, you can research how to effectively use shouldOverrideUrlLoading().

The one downside is that it will add the blue link color to all linked objects. You should be able to override this in MathML by setting the color of all elements to black with an mstyle declaration, but I'm currently having issues doing so.

I didn't cover quite everything, but I hope this helps!