1

Possible Duplicate:
Javascript: Unicode string split by chars

I have a javascript string which contains some tamil characters. I need to split them into individual unicode characters. The split method does not understand the http://en.wikipedia.org/wiki/Complex_text_layout Complex text layout.

For example:

Calling split("") on "கதிரவன்" returns:

,க,த,ி,ர,வ,ன,்

when I expected:

க,தி,ர,வ,ன்

What should be done to split unicode characters from a string properly ?

Edit: I can navigate these letters just fine in the browser (chrome). I am trying to use this js in a chrome extension. So I am fine if there is a chrome-specific solution too.

Community
  • 1
  • 1
Sankar
  • 6,192
  • 12
  • 65
  • 89

4 Answers4

3

This is totally doable.

First off, you create a set/dictionary which includes all diacritic-like characters as keys, we could name it as diacritics, and implement it just with a object literal:

var diacritics = {'\u0bbf':true,'\u0bcd':true,...};

Then do this:

var tempList = "கதிரவன்".split('');
var targetList = [];
for(var idx in tempList){
  if(diacritics[tempList[idx]])
    targetList[targetList.length - 1] +=  tempList[idx];
  else
    targetList.push(tempList[idx]);
}

We don't even need a tempList, just loop over the str char by char will do the job:

for(var i = 0; i != str.length; ++i){
  var ch = str[i];
  diacritics[ch] ? (targetList[targetList.length - 1] += ch) : targetList.push(ch);
}
Need4Steed
  • 2,170
  • 3
  • 22
  • 30
  • I was just trying to avoid this and hoping to have a native way :) For anyone referring to this in future: the diacritics should be: var diacritics = {'\u0B82':true,'\u0BBE':true, '\u0BBF':true, '\u0BC0':true, '\u0BC1':true, '\u0BC2':true, '\u0BC6':true, '\u0BC7':true, '\u0BC8':true, '\u0BCA':true, '\u0BCB':true, '\u0BCC':true, '\u0BCD':true, '\u0BD7':true}; – Sankar Oct 21 '12 at 14:10
  • It's actually unavoidable. It's not a flaw of javascript, no other language would ever support this natively. If we try to work out a solution in C/C++ it would be implemented more or less the same way. Because to split the string we either need a specified seperator or know the natural boundary of every part, since we don't have any seperator to use, we have to implement a boundary detection mechanism, that's exactly what the code above does. We could use `charCodeAt` instead of `str[i]` so that the code might run a little bit faster, if you concern about the performance, do some profiling. – Need4Steed Oct 21 '12 at 15:24
  • IIUC Google's go programming language supports parsing runes from a utf-8 string, which takes care of this. I have not verified it though. – Sankar Oct 22 '12 at 16:07
1

Have you tried a unicode library like https://github.com/reyesr/javascript-unicode that gives methods related to unicode types, for instance testing for punctuation or separator chars and split according to it (you won't be able to use the String.split() method though I guess). Or, make a big regex with all the separator chars from the unicode table, and use it to split your text. I think you're not short of options, although you're right about the lack of native support.

qSirassi
  • 61
  • 1
0

I fear that your best solution will be to build and use a web service to do the job. Porting the necessary data and algorithm into javascript would be a daunting project.

bmargulies
  • 97,814
  • 39
  • 186
  • 310
0

This would be quiet bulky to do manually from javascript as javascript itself, though being unicode, interprets strings as ascii. For information on why this is not an option and a possible work around see this post.

Community
  • 1
  • 1
SReject
  • 3,774
  • 1
  • 25
  • 41
  • Nope. Your example does not have the accent/dot over the `வ`, and using `"கதிரவன்"` instead of `"கதிரவ"` shows the same problem that the OP already has. – Matt Ball Oct 21 '12 at 13:09