0

Our application uses Redemption 4.4.0.1026 to save .MSG and .TXT versions of an .EML file. I get Unicode characters being replaced with question marks. I have made sure I installed all the language packs for Windows. It happens on both Windows 2003 and XP (both with all updates installed).

Two things that are strange/interesting:

  1. On regular desktop machines, the Unicode is not being stripped. But on my VMware Workstation 6.5.5 (latest in the 6.x series), I get question mark replacement.

  2. It doesn't happen for .MSG files. They are not perfect, but considerably better... I get Chiense, Arabic, Russian, etc. showing up as real characters.

Any ideas for what might be going on here?

Thanks!

P.S. I wanted to post a screenshot of what's going on, but this will have to do:

Chinese (simplified):

????????????????????????????????????????

Spanish:

En marzo de 1997, CNN lanz CNN en Espaol, la cadena de 24 horas de noticias en ese idioma dirigida al mercado latinoamericano. CNN en Espaol brinda reportajes de los principales acontecimientos mundiales, cobertura en vivo complementada con anlisis, noticias internacionales del mundo empresarial y financiero, deportes y notas de inters las 24 horas del da, los 365 das del ao.

Hindi:

???????? 1  ??? ???????? ?? ???? ?? ???????? ?? ????? ??? ??????? ??????????? ??????? ??? ?????? ?????? ?? ?????????? ?? ??? ??????? ?? ?? ?????? ?????? ??????? ?? ??? ?? ?????? ???? ???????

Arabic:

???????????????????????????? ????????? ??????????????????? ?????????? ?????? ??????? ???????????????????????????????????????

Portuguese:

Sustentava contra ele Vnus bela, Afeioada  gente Lusitana, Por quantas qualidades via nela Da antiga to amada sua Romana; Nos fortes coraes, na grande estrela, Que mostraram na terra Tingitana, E na lngua, na qual quando imagina, Com pouca corrupo cr que  a Latina. 

Bengali:

?? ??? ??? ?? ?? ??? ??? ???? ????????? ???????? ???????????? ????????? 

Russian:

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

Japanese:

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?  ? ? ?

German:

Er kam am Freitagabend nach einem harten Arbeitstag und dem blichen rger, der ihn schon seit Jahren immer wieder an seinem Arbeitsplatz plagt, mit fraglicher Freude auf ein Mahl, das seine Frau ihm, wie er hoffte, bereits aufgetischt hatte, endlich zu Hause an.

Korean:

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
Aaron
  • 1
  • 1
  • 1

1 Answers1

2

'?' characters appear when you perform a Unicode->Ansi conversion to an Ansi charset that does not support the Unicode characters that are being converted.

Remy Lebeau
  • 555,201
  • 31
  • 458
  • 770
  • Remy, that's understood. But given that it works on some machines and not others, I suspect an environmental cause at work. I don't know what it is, but there's got to be some difference in installed components between the systems. – Aaron Dec 13 '10 at 23:41
  • Hard to say since you did not show any code, but keep in mind that different machines can have different Ansi language settings at the OS layer, so if the saving is reliant on the OS default encoding, you can get different results on different machines. – Remy Lebeau Dec 14 '10 at 19:47
  • Remy, thanks. Do you know where I can check that setting? Our app runs on Windows XP SP3 and Windows 2003 SP2. – Aaron Dec 15 '10 at 17:45
  • Control Panel > Regional and Language Options – Remy Lebeau Dec 23 '10 at 06:33