Strange behavior of the Perl script (additional slash)

Question

For the localization of my project I use gettext. And I use po2json script to convert my translation file from .po format to .json format. Everything works well except one thing - if the line I want to convert contain control character, in the .json file these characters are escaped and eventually the string does not match the code (thus this phrase doesnt translated). I tried to understand the script, but I do not know Perl and I do not understand the origin of the extra slash.

Here is the code and parts of original .po file and converted .json:

The script:

my $pos = Locale::PO->load_file_asarray($src) or die "Can't parse po file [$src].";

foreach my $po (@$pos)
{
    my $qmsgid1 = $po->msgid;
    my $msgid1 = $po->dequote( $qmsgid1 );


    my $qmsgctxt = $po->msgctxt;
    my $msgctxt = $po->dequote($qmsgctxt) if $qmsgctxt;

    # build the new msgid key
    my $msg_ctxt_id = defined($msgctxt) ? join($gettext_context_glue, ($msgctxt, $msgid1)) : $msgid1;

    # build translation side
    my @trans;

    # msgid plural side
    my $qmsgid_plural = $po->msgid_plural;
    my $msgid2 = $po->dequote( $qmsgid_plural ) if $qmsgid_plural;
    push(@trans, $msgid2);

     # translated string
     # this shows up different if we're plural
     if (defined($msgid2) && length($msgid2))
     {
        my $plurals = $po->msgstr_n;
        for (my $i=0; $i<$plural_form_count; $i++)
        {
            my $qstr = ref($plurals) ? $$plurals{$i} : undef;
            my $str  = $po->dequote( $qstr ) if $qstr;
            push(@trans, $str);
        }

    # singular
    } else {
        my $qmsgstr = $po->msgstr;
        my $msgstr = $po->dequote( $qmsgstr ) if $qmsgstr;
        push(@trans, $msgstr);
    }

    $$json{$msg_ctxt_id} = \@trans;
}


my $jsonobj = new JSON;
my $basename = basename($src);
$basename =~ s/\.pot?$//;
if ($pretty)
{
    print $jsonobj->pretty->encode( { $basename => $json });
} else {
    print $jsonobj->encode( { $basename => $json } );
}

Source .po file example:

#: some/path/to/file1.php:37
msgid "Original string without command character"
msgstr "Translated string without command character"

#: some/path/to/file2.php:73
msgid "Original string with\ncommand character"
msgstr "Translated string with\ncommand character"

Converted .json file:

{"Original string without command character":[null,"Original string without command character"],"Translated string with\\ncommand character":[null,"Translated string with\\ncommand character"]}

I would be grateful to any advice or tips!

And I'm sorry, I realy do not familiar with Perl...

score 2 · Accepted Answer · answered Jan 17 '14 at 16:50

2

If you had a newline, you would get "\n".

>perl -MJSON -E"say encode_json [ chr(0x0A) ]"
["\n"]

You don't have a newline. You have \ followed by n.

>perl -MJSON -E"say encode_json [ chr(0x5C).chr(0x6E) ]"
["\\n"]

The \ must be escaped, or two string above would both return the same thing, which means that one of them would get decoded wrong.

You can verify the encoding is correct by seeing if you get back the original string after decoding.

>perl -MJSON -E"print encode_json [ chr(0x0A) ]" |
    perl -MJSON -nE"say sprintf '%v02X', decode_json($_)->[0]"
0A

>perl -MJSON -E"print encode_json [ chr(0x5C).chr(0x6E) ]" |
    perl -MJSON -nE"say sprintf '%v02X', decode_json($_)->[0]"
5C.6E

answered Jan 17 '14 at 16:50

ikegami

367,544
15
269
518

Thanks alot, now I understand my issue. But what if I can NOT edit original string? Is there any method to avoid escaping of '\' character? – Sergey Novikov Jan 19 '14 at 10:17
"The `\ ` must be escaped, or two string above would both return the same thing, which means that one of them would get decoded wrong." – ikegami Jan 19 '14 at 19:24
OTOH, if you want to convert `\n` to newlines before encoding it using JSON, that's an entirely different issue/question. – ikegami Jan 19 '14 at 19:25
See http://stackoverflow.com/questions/16556130/perl-how-to-use-command-line-special-characters-newline-tab-in-list-separat – ikegami Jan 19 '14 at 19:30

Strange behavior of the Perl script (additional slash)

1 Answers1