Elm decoding unknown json structure

Question

I've just started working with Elm to do some front-end prototyping using a Rest API I'm working on. In general, the API returns "reasonable" data structures that can be decoded because the keys and value-types are well-known, but several resource types return a data entry that just has raw json that has no predetermined structure.

Everything I've read so far seems to assume you know the structure of the data you're decoding, whereas in plain js it's relatively easy to loop over the keys and reflect on the types in order to determine how they should be handled at runtime. I'm not seeing a clear path toward handling this kind of data in Elm yet.

E.g.,

{
  "name":"foo",
  "data": {
    "bar": [{"baz":123}, "quux"]
  },
  ...
}

I'd like to know if it is currently possible to parse the value of the data entry with something akin to

function go(obj)
    for key in keys(foo)
        if foo[key] is an object
            go(foo[k])
        else if foo[key] is an array
            map(go, foo[k])
        ...

Specifically:

Is it currently possible to handle unknown, possibly deeply nested and heterogeneous json data in Elm?
If so, can you give me the key concept or high level intuition on how the author(s) intended data like this to be decoded?

I'm afraid it's not “Elm-like” to receive a structure you don't know. In Elm, you always expect an object to contain certain properties, and Elm even does a check in run-time for whether all the properties you expect exist in that object. — rishat, Nov 27 '16 at 03:45

Chad Gilbert · Accepted Answer · 2016-11-27T16:12:48.403

Yes, it is possible to write a general purpose decoder. You can first define a union type that holds all possible Json types:

type JsVal
  = JsString String
  | JsInt Int
  | JsFloat Float
  | JsArray (List JsVal)
  | JsObject (Dict String JsVal)
  | JsNull

And now you can use Json.Decode.oneOf to try every possibility.

import Json.Decode as D exposing (Decoder)
import Dict exposing (Dict)

jsValDecoder : Decoder JsVal
jsValDecoder =
  D.oneOf
    [ D.string |> D.andThen (D.succeed << JsString)
    , D.int |> D.andThen (D.succeed << JsInt)
    , D.float |> D.andThen (D.succeed << JsFloat)
    , D.list (D.lazy (\_ -> jsValDecoder)) |> D.andThen (D.succeed << JsArray)
    , D.dict (D.lazy (\_ -> jsValDecoder)) |> D.andThen (D.succeed << JsObject)
    , D.null JsNull
    ]

Json.Decode.lazy is necessary for the JsArray and JsObject constructors because they are defined recursively.

This structure should handle anything you throw at it, and it will be up to the rest of your program to decide what to do with such a flexible type.

Edit

As @Tosh pointed out, this decoder can be cleaned up by using map instead of an andThen followed by a succeed:

jsValDecoder : Decoder JsVal
jsValDecoder =
  D.oneOf
    [ D.map JsString D.string
    , D.map JsInt D.int
    , D.map JsFloat D.float
    , D.list (D.lazy (\_ -> jsValDecoder)) |> D.map JsArray
    , D.dict (D.lazy (\_ -> jsValDecoder)) |> D.map JsObject
    , D.null JsNull
    ]

Just want to comment that you could use a form: e.g. `D.map JsString D.string`. A little bit easier to read for me, at least. — Tosh, Nov 27 '16 at 15:19
Thanks, @Tosh! Good point, A single cased `success` monadic bind is a good smell that a simple `map` should work just as well. — Chad Gilbert, Nov 27 '16 at 16:14

score 4 · Answer 2 · answered Jun 21 '18 at 06:49

In Chad's excellent answer, the boolean type is missing. Here's a full module able to handle booleans as well:

module Data.JsonValue exposing (JsonValue(..), decoder)

import Dict exposing (Dict)
import Json.Decode as Decode
    exposing
        ( Decoder
        , dict
        , string
        , int
        , float
        , list
        , null
        , oneOf
        , lazy
        , map
        , bool
        )


type JsonValue
    = JsonString String
    | JsonInt Int
    | JsonFloat Float
    | JsonBoolean Bool
    | JsonArray (List JsonValue)
    | JsonObject (Dict String JsonValue)
    | JsonNull


decoder : Decoder JsonValue
decoder =
    oneOf
        [ map JsonString string
        , map JsonInt int
        , map JsonFloat float
        , map JsonBoolean bool
        , list (lazy (\_ -> decoder)) |> map JsonArray
        , dict (lazy (\_ -> decoder)) |> map JsonObject
        , null JsonNull
        ]

Thank you for the answer, but how to use it? Could you provide an example? I mean I have that data, but how to build HTML with that data? — Ivan Pruchai, Apr 22 '22 at 13:20

Elm decoding unknown json structure

2 Answers2