issue with NotesJSONNavigator in lotusscript

Question

I have the following agent to test the new V10 JSON parser The json in the code is retrieved from the darksky weather api

Option Public
Option Declare

Sub Initialize
    Dim json As String
    Dim session As New NotesSession

    json = |{
"latitude": 51.2747748,
"longitude": 4.4433923,
"timezone": "Europe/Brussels",
"daily": {
    "summary": "Rain today, with high temperatures falling to 3øC next Sunday.",
    "icon": "rain",
    "data": [{
        "time": 1547334000,
        "summary": "Rain in the afternoon and breezy starting in the afternoon.",
        "icon": "rain",
        "sunriseTime": 1547365378,
        "sunsetTime": 1547395251,
        "moonPhase": 0.23,
        "precipIntensity": 0.4115,
        "precipIntensityMax": 1.5621,
        "precipIntensityMaxTime": 1547380800,
        "precipProbability": 0.97,
        "precipType": "rain",
        "temperatureHigh": 10.56,
        "temperatureHighTime": 1547391600,
        "temperatureLow": 5.5,
        "temperatureLowTime": 1547449200,
        "apparentTemperatureHigh": 10.56,
        "apparentTemperatureHighTime": 1547391600,
        "apparentTemperatureLow": 2.06,
        "apparentTemperatureLowTime": 1547427600,
        "dewPoint": 6.77,
        "humidity": 0.87,
        "pressure": 1009.48,
        "windSpeed": 7.24,
        "windGust": 17.26,
        "windGustTime": 1547395200,
        "windBearing": 285,
        "cloudCover": 0.93,
        "uvIndex": 1,
        "uvIndexTime": 1547377200,
        "visibility": 12.59,
        "ozone": 311.57,
        "temperatureMin": 7.17,
        "temperatureMinTime": 1547416800,
        "temperatureMax": 10.56,
        "temperatureMaxTime": 1547391600,
        "apparentTemperatureMin": 2.64,
        "apparentTemperatureMinTime": 1547416800,
        "apparentTemperatureMax": 10.56,
        "apparentTemperatureMaxTime": 1547391600
    }]
},
"offset": 1
}|

    json = removeCRLF(json)
    Dim jsnav As NotesJSONNavigator 
    Set jsnav = session.CreateJSONNavigator(json)
    Dim el As NOTESJSONELEMENT
    Set el = jsnav.getelementbypointer("/latitude")
    Print CStr(el.value)
End Sub
Function removeCRLF(json) As String
    removeCRLF =Replace(Replace(json, Chr(13), ""),Chr(10),"")
End Function

I get this error when running the agent:

Unable to Parse JSON string: Missing a comma or '}' after an object member. offset 1791

After some testing, I found out that the error is coming from a special character in the json (ø in '... falling to 3øC next ...').

Can anybody help me out on how to avoid/convert characters that could cause issues when parsing the JSON?

PS: The openntf JSON parser handles the json correctly.

Answer 1

If you can save the JSON to the file system you should be able to read it in as UTF-8 and then can use the JSONNav to get the value you are looking for.

Sub Initialize
    Dim json As Variant
    Dim session As New NotesSession
    Dim inbuf As NotesStream
    Dim path As String

    path = "c:\\jsontest.json"  
    Set inbuf = session.Createstream()
    If Not(inbuf.Open(path, "UTF-8")) Then
        Print "Unable to open JSON file"
        Exit Sub
    End If
    json = inbuf.Read()

    If IsArray(json) then
        Dim jsnav As NotesJSONNavigator
        Set jsnav = session.CreateJSONNavigator(json)
        Dim el As NotesJSONElement
        Set el = jsnav.getelementbypointer("/latitude")
        Print CStr(el.value)
    Else
        Print "JSON is nothing"
    End If
End Sub

Working to get a better answer on how to do it with inline JSON.

Answer 2

10.0.1 FP2 has the fix for this and is now available for download: https://www-01.ibm.com/support/docview.wss?uid=ibm10871936. You’ll want to use the new PreferJSONNavigator property on the request. See this technote for details https://www-01.ibm.com/support/docview.wss?uid=ibm10875724

Answer 3

It looks like you can solve this problem using NotesMIMEEntity for the UTF-8 conversion which can be done in memory and avoids the need to resort to a file-system call.

Sub Initialize
    Dim session As New NotesSession
    Dim db As NotesDatabase
    Dim stream As NotesStream
    Dim doc As NotesDocument
    Dim jsNav As NotesJSONNavigator
    Dim mime As NotesMIMEEntity
    Dim js$

    js = |{"summary": "Rain today, with high temperatures falling to 3øC next Sunday."}|

    Set db = session.CurrentDatabase
    Set doc = db.CreateDocument
    Set mime = doc.CreateMIMEEntity
    Set stream = session.CreateStream

    ' Write the JSON to a stream
    Call stream.WriteText(js)

    ' Write the stream into a NotesMIMEEntity with UTF-8 character set specified
    Call mime.SetContentFromText(stream, {text/plain;charset="UTF-8"}, ENC_NONE)

    ' Clear the existing stream and read the NotesMIMEEntity contents back in as binary
    Call stream.Truncate
    Call mime.GetContentAsBytes(stream)

    ' Finally read in the corrected JSON
    stream.Position = 0
    Set jsNav = session.CreateJSONNavigator(stream.Read)
    Call stream.Close

    MsgBox jsNav.GetFirstElement.Value
End Sub

Answer 4

This has got to be a charset issue, and sure enough... there's a property that seems to be relevant. Check the value of the jsnav.PreferUTF8. It is documented here: link. They don't say what the default is. If it's true, set it to false. If it's false, set it to true.