14

I am using a key value store as the backend for my golang application, with the date serving as the key (to keep entries sorted) and json documents as the values. The top level namespace of the json (foo) and the type and date are present in each json document that I'm storing but otherwise there are some differences (especially with respect to some nested json data), so when keyI'm pulling from the database, I don't really know what I'm pulling out at any time that I'm looping through . Here is a sample of the json data 

{
  "foo": {
    "id": "124",
    "type": "baz",
    "rawdata": [
      123,
      345,
      345
    ],
    "epoch": "1433120656704"
  }
}

{
  "foo": {
    "id": "234",
    "type": "bar",
    "rawdata": [
      {
        "key": "dog",
        "values": [
          123,
          234
        ]
      },
      {
        "key": "cat",
        "values": [
          23,
          45
        ]
      }
    ],
    "epoch": "1433120656705"
  }
}

when I'm pulling from the database, the first thing I do is unmarshal each entry into a map[string]*json.RawMessage to deal with the foo namespace

//as I'm looping through the entries in the database
   var objmap map[string]*json.RawMessage
   if err := json.Unmarshal(dbvalue, &objmap); err !=nil{
       return err
   }

which I do thanks to this SO answer

However, unlike in that SO answer, when I have to unmarshal again whatever is contained under the foo namespace I don't know which struct to unmarshal into

   if err :=json.Unmarshal(*objmap["foo"], &bazorbar; err != nil{
         return err
   }

 type Baz struct{
  Id string `json:"id"`
  Type string `json:"type"`
  RawData []int `json:"rawdata"`
  Epoch string  `json:"epoch"`
}

type Bar struct{
  Id string `json:"id"`
  Type string `json:"type"`
  RawData []*Qux `json:"rawdata"`
  Epoch string  `json:"epoch"`
}
//nested inside Bar
type Qux struct{
  Key string `json:"key"`
  Values []int `json:"values`
}

Two part Question:

  1. Is there a way to avoid repeated unmarshals (or is that something I shouldn't even care about)
  2. how can I figure out which struct to unmarshal the json.RawMessage into (which also allows for nested json data)

Update: the initial answer provided by @chendesheng enables me to find out the type but not to unmarshal again into a struct once that type has been determined (which I need to do), so based on a conversation in the comments to his/her answer, I would be interested in either of these possibilities

a) make a copy of the json.RawMessage, unmarshal into the interface as you shown (by chendesheng's answer), and then unmarshal the copy itno the struct once you know the type (from having unmarshaled into the interface)?

b) use a regular expression to determine the type and then unmarshal into a struct of that type once it's known

alessiosavi
  • 2,753
  • 2
  • 19
  • 38
BrainLikeADullPencil
  • 11,313
  • 24
  • 78
  • 134
  • 1
    No offense but I think you should reconsider your design because you should be able to avoid a decision like that at run time or at the very least use a type flag of some sort to avoid this more loose introspective means of deciding how to deserialize. You're treating data from YOUR db as if it's from some uncontrollable and unpredictable third party, I recommend against that. – evanmcdonnal Jun 01 '15 at 15:47
  • @evanmcdonnal thanks for the input. I'll take it into consideration and if I find a better way to do it I will however, I don't believe I'm treating t as if it's from an unpredictable third party. It's just that the structure of the json documents are not the same and therefore have to be unmarshalled into different structs and I therefore have to figure out what the json looks like before I choose which struct to unmarshal into – BrainLikeADullPencil Jun 01 '15 at 15:50
  • Yes, the json is representing different types. You can store the type info in your database as well and use it to make a 100% conclusive decision about which type it is prior the calling Unmarshal, you're using unmarshal like it's a try/catch rather than using type info + select to unmarshal into the correct type with no indeterminate runtime behavior. – evanmcdonnal Jun 01 '15 at 15:53

1 Answers1

11

Two ways to check struct type:

  1. Unmarshal json.RawMessage to a map[string]interface{}
  2. Use a regular expression to extract type string

http://play.golang.org/p/gfP6P4SmaC

Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
chendesheng
  • 1,969
  • 18
  • 14
  • I'm interested in this answer, but I think you misunderstand the structure of my data. Every one of my documents has a "foo" field (it is the top level namespace of each json document). What I need to determine is the value of the type field when I unmarshal a second time. Can you redo your play.golang.org to show me that? – BrainLikeADullPencil Jun 01 '15 at 13:19
  • but after I unmarshal it into the interface, (and figure out what the value of the type field is), I can't unmarshal it again into the Bar struct (now that I know the Bar struct would be the appropriate struct), can I? I need to be able to get the document into the appropriate struct – BrainLikeADullPencil Jun 01 '15 at 13:32
  • I don't think you can. If you want use a typed struct object, you need to create the object and copy fields to it. – chendesheng Jun 01 '15 at 13:36
  • 1
    is it possible to make a copy of the json.RawMessage, unmarshal into the interface as you showed, and then unmarshal the copy itno the struct once you know the type? – BrainLikeADullPencil Jun 01 '15 at 13:48
  • 1
    Yes, but you can always check the object type by regular expression instead, no need to unmarshal into a interface. What I suggest is the way go handle dynamic json type. – chendesheng Jun 01 '15 at 13:53
  • 1
    ok, can you update your answer to show how to do both? I've given you an upvote and will accept your answer if you can show me. Thank you so much for your assistance. – BrainLikeADullPencil Jun 01 '15 at 13:57
  • Thank you for updating your answer with the regular expression (I didn't notice). I will have a look at it. By saying that there are two ways to check the struct type, are you saying that copying the json.RawMessage is not a way? – BrainLikeADullPencil Jun 01 '15 at 17:22
  • This was very helpfull – Jimmy Obonyo Abor Apr 24 '17 at 05:11