How to disassemble, process, and reassemble a string in perl (using regex?)?

Question

I am working on documenting some mathematical research in a Trac wiki. I have set up the Trac installation with a MathJax plugin and everything has been working great. As the documents have gotten longer, I wanted to be able to use TextMate to do syntax highlighting and make generating previews easier. I found a Trac bundle and installed it. Inside the bundle it has the following perl script for generating an HTML preview:

# Preview command contributed by Tudor Marghidanu 
#
# Requires the Text::Trac perl module:
#
# sudo perl -MCPAN -e 'install Text::Trac'
#
#!/usr/bin/env perl

use strict;
use warnings;

use Text::Trac;

my $parser = Text::Trac->new();
$parser->parse( join( '', <STDIN> ) );

print $parser->html();

(For anyone else using this bundle, the #! line needed to be moved to the top of the file for it to work properly)

This did a good job of generating the Trac wiki markup into HTML, but, obviously, did not do anything about the MathJax markup. I simply added a line

print '<script type="text/javascript"
  src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>';

to load the javascript from the CDN. Now the problem is that the MathJax code symbols, such as the ^ are being translated into HTML because they are also part of the TracWiki syntax. I tried using the TracWiki markups to essentially "comment out" the MathJax code segments, but the Perl Trac library does not seem to care about those.

It seems like there should be a way to regex match all the mathjax code segments, stash them into an array, replace them with placeholder tokens (like mj1, mj2, ...), process the replaced wiki text into HTML, then regex replace the placeholders from the array of stashed values.

If this is the right way to do it, how is this done in Perl?

If this is not the right way to do it, what is?

There is more then one way to reach youre goal. Plan and act (and later react) — lordkain, Oct 14 '14 at 13:34
Your approach seems valid. Can you add an example of such a code segment and also some stuff around it? — simbabque, Oct 14 '14 at 13:50
I am not too familiar with how the code in Perl would look as Perl is not my language of choice. The dumb way I could think of doing it would require a lot of for loops and I get the feeling that Perl is built to handle this in a much cleaner way. — cheepychappy, Oct 14 '14 at 14:13
I think @simbabque meant an example of the MathJax code symbols and the text surrounding them. It's very difficult to advise on parsing/transforming text without seeing the text to be parsed/transformed! — i alarmed alien, Oct 14 '14 at 14:31
Exactly. I'll help with the Perl part as you obviously know the algorithm you want, but for that I need to see what you want to parse. ;) — simbabque, Oct 14 '14 at 14:32
Oh, thanks @simbabque. The mathjax library I am using always has it's code wrapped in `\(` and `\)` — cheepychappy, Oct 14 '14 at 15:12
@cheepychappy for future questions, you should include samples of the input and the desired output. The code you posted wasn't relevant to the core of your question, which was about parsing text input and preventing certain parts from being transformed. Your answer is also uninformative because there's no input and output given. — i alarmed alien, Oct 14 '14 at 15:17
Note from the future: cdn.mathjax.org is nearing its end-of-life, check https://mathjax.org/cdn-shutting-down for migration tips. — Peter Krautzberger, Apr 13 '17 at 13:09

score 2 · Accepted Answer · answered Oct 15 '14 at 04:48

Taking your solution as a starting point, the following is a much simpler solution.

It relies on the ordered nature of the cut snippets and s///eg to do the replacement in a single go instead of two steps:

#!/usr/bin/env perl

#
# Preview command contributed by Tudor Marghidanu
#
# Requires the Text::Trac perl module:
#
# sudo perl -MCPAN -e 'install Text::Trac'
#

use strict;
use warnings;

use Text::Trac;

my $parser = Text::Trac->new();

my $tractext = join '', <STDIN>;

my @mathjax_snippets;

$tractext =~ s{(\\\(.*\\\))}{
    push @mathjax_snippets, $1;
    "math_jax_snippet"
}eg;

$parser->parse($tractext);
my $html = $parser->html();

$html =~ s/math_jax_snippet/shift @mathjax_snippets/eg;

print '<script type="text/javascript"
  src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>';

print $html;

Many thanks @Miller, the s///eg is exactly the kind of solution I was hoping to learn about. — cheepychappy, Oct 15 '14 at 05:59

score 1 · Answer 2 · answered Oct 14 '14 at 15:11

Ok, so I was able to get together some code. It is probably not the best Perl-y way to do this, but it does work:

#!/usr/bin/env perl

#
# Preview command contributed by Tudor Marghidanu 
#
# Requires the Text::Trac perl module:
#
# sudo perl -MCPAN -e 'install Text::Trac'
#

use strict;
use warnings;

use Text::Trac;

my $parser = Text::Trac->new();

my $tractext = join( '', <STDIN> );
my @mathjax_snippets = ();
my @placeholders = ();

while ($tractext =~ /(\\\(.*\\\))/g) {
    push(@mathjax_snippets,"$1");
    my $pos = pos $tractext;
    push(@placeholders,"math_jax_snippet_$pos");
}

my %substitution;
@substitution{@mathjax_snippets} = @placeholders;
my $pattern = '(' . join('|', map quotemeta, @mathjax_snippets) . ')';
$tractext =~ s/$pattern/$substitution{"$1"}/g;

$parser->parse( $tractext );
my $html = $parser->html();

my %replace;
@replace{@placeholders} = @mathjax_snippets;
$pattern = '(' . join('|', map quotemeta, @placeholders) . ')';

$html =~ s/$pattern/$replace{"$1"}/g;

print '<script type="text/javascript"
  src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>';

print $html;

How to disassemble, process, and reassemble a string in perl (using regex?)?

2 Answers2