0

Using Embarcadero C++Builder XE3 with Indy TIdHTTP component.

From some API, I get a String response:

{"Items":[{"Id":"jp-lp|o4nfz4IB6j9TcSaTQGxD","Type":"Container","Text":"106-0031 \u6771\u4EAC\u90FD \u6E2F\u533A \u897F\u9EBB\u5E03","Highlight":"0-3,4-8","Description":" - 991 Addresses"}]}

So, I guess either Indy or C++Builder itself convert it wrongly.

I read the string and it contains what's above.

When I parse it with TJSONObject and get the value of Text, eg:

String val = myJSONObject.get("Text")

And show it by the ShowMessage() function, everything works fine (also, under the debugger by inspecting the value), and I see a correctly presented message:

106-0031 東京都 港区 西麻布

As soon as I pass it to a TField of my TDataSet using the AsString or AsWideString property, and using ShowMessage(TField.AsString), I see question marks instead of real Unicode characters.

Also, when attaching my TDataSet to TDBGrid, I see the same question marks in the fields.

My string now looks like that:

106-0031 ??? ?? ???

I was trying to use different datasets, different versions of grids, numerous conversions.

It only worked fine for TStringGrid when I provide cell values using TStringGrid.Cells[][], so I assume that might be the problem. However, presenting the result with TStringGrid is not an option. I need to present my result in either TDBGrid / TJvDBUltimGrid / TDBAdvGrid, so it has to go through TDataSet.

All DataSets I use are in-memory sets, ie TClientDataSet, etc.

Here is the code:

String response = "{\"Items\":[{\"Id\":\"jp-lp|o4nfz4IB6j9TcSaTQGxD\",\"Type\":\"Container\",\"Text\":\"106-0031 \\u6771\\u4EAC\\u90FD \\u6E2F\\u533A \\u897F\\u9EBB\\u5E03\",\"Highlight\":\"0-3,4-8\",\"Description\":\" - 991 Addresses\"}]}";
TJSONObject *jsResponse = static_cast<TJSONObject *>(TJSONObject::ParseJSONValue(response));
String Key("Items");
TJSONArray *jsItems;
for(int i = 0; i < jsResponse->Size(); ++i){
  String CurrKey(jsResponse->Get(i)->JsonString->Value());
  if(CurrKey == Key){
    jsItems = ((TJSONArray *)(NativeJSON->Get(i)->JsonValue));
    break;
  }
}
for (int i = 0; i < jsItems->Size(); ++i){
  TJSONObject* currItem = ((TJSONObject *)jsItems->Get(i));
  String correctlyDisplayedValue;
  for(int i = 0; i < currItem->Size(); ++i){
    String CurrKey(currItem->Get(i)->JsonString->Value());
    if(CurrKey == "Text"){
      correctlyDisplayedValue = ((String)((TJSONString *)(NativeJSON- >Get(i)->JsonValue)));
      break;
    }
  }
  ShowMessage(correctlyDisplayedValue); //prints out correctly
  memoDS->Active = true;
  memoDS->FieldByName("Text")->AsString = correctlyDisplayedValue;
  ShowMessage(memoDS->FieldByName("Text")->AsString); // displayed message is one with "??? ?? ???"
}

And there is my memoDS definition in .dfm unit:

 object memoDS: TkbmMemTable
     DesignActivation = True
     AttachedAutoRefresh = True
     AttachMaxCount = 1
     FieldDefs = <     
       item
         Name = 'TEXT'
         DataType = ftWideString
       end>
     IndexDefs = <>
     SortOptions = []
     PersistentBackup = False
     ProgressFlags = [mtpcLoad, mtpcSave, mtpcCopy]
     LoadedCompletely = False
     SavedCompletely = False
     FilterOptions = []
     Version = '7.76.00 Standard Edition'
     LanguageID = 0
     SortID = 0
     SubLanguageID = 1
     LocaleID = 1024
     AutoUpdateFieldVariables = False
     Left = 566
     Top = 6
     object strngfldAddrSearchText: TStringField
       FieldName = 'TEXT'
       Size = 255
     end
   end
Tomasz Sipel
  • 1
  • 1
  • "*either Indy or C++Builder itself convert it wrongly*" - what do you mean? The JSON you have shown looks perfectly fine to me. The Unicode characters are simply being escaped in the JSON itself, and `TJSONObject` will unescape them during parsing, which is why `ShowMessage(myJSONObject.get("Text"))` displays the correct text. So, the problem is not with the JSON itself. It is with the database instead. Did you setup the database to support Unicode text? Are you using a Unicode/UTF charset/collation on your DB field(s)? What kind of database are you using, and how are you accessing it? – Remy Lebeau Mar 16 '23 at 20:08
  • @RemyLebeau well I use only in memory datasets, I edited question to involve that as if its important. I did try to set field def as `WideString` or `String' as well as using other in-memory `DataSets` with numerous configurations since DataSnap might not be the best, but nothing seems to work, might be I skipped some configuration? I'd be super glad for any advise. – Tomasz Sipel Mar 16 '23 at 23:30
  • It would help if you provided your actual DataSet setup, and your actual code that is assigning the JSON to the DataSet. – Remy Lebeau Mar 16 '23 at 23:33
  • @RemyLebeau Thank You for Your interest in the issue, I extracted the code to provide just the lines being used when processing my issue. As mentioned, I did try to use different in-memory DataSets so DataSet setup should not be a matter in that case, however I provided with latest one that I tried. – Tomasz Sipel Mar 17 '23 at 08:17
  • Not related to the DB issue, but what is `NativeJSON`? Looks like that should be `jsResponse` in the 1st loop and `currItem` in the 3rd loop. Also, those loops are completely unnecessary, as you should use the `TJSONObject.Values[]` property, `TJSONObject.FindValue()` method, or `TJSONObject.(Try)GetValue()` method instead. – Remy Lebeau Mar 17 '23 at 14:50

0 Answers0