Delete line break at end of text; running a perl script inside of Automator

Question

I am writing a perl script that will be run inside of an Automator app to process documents that were previously processed by hand. I need to do this process weekly, always with the same junk data removed. These are rtf files, converted from html files on Mac OS X using another Automator script in order to maintain formatting. I have created a new droplet script to process the rtf files to remove unnecessary junk data.

My shell script is:

#!/bin/bash
# 
#    replace CR with CRLF
#     
/usr/bin/perl -CSDA -pi <<'EOF' - "$@"
s/dateformat//og;
s/text1//og;
s/text2//og;
s/text3//og;
s///og;

EOF

This takes care of 99% of what needs to be done. However, the final file comes out with excess line breaks. Is there any way to have that the substitution of text1, text2 etc includes removing the line break that follows? My only restriction is that this has to be able to be run in an Automator script shell window.

Input sample data is formatted as such:

Text1 Dateformat 
[Content1] 

Text2 Dateformat
[Content2]

Text3 Dateformat
[Content3]

The script above produces output:

[Content1]


[Content2]


[Content3]

Desired output should be formatted as:

[Content1]

[Content2]

[Content3]

In the original document, there is a single line break after a content block, then the Text1 and Dateformat.

After processing, Text1 and Dateformat are removed, but as you can see there are now two line breaks between content blocks.

Answer 1

You can match and remove the whitespace as part of your pattern. The \R is the generic line ending, which matches any of the Unicode line endings, including a bare newline or a carriage return/newline pair. Also, take a look at a hexdump of the data to see what the real line-endings are. Old Mac Classic line endings seems to show up in odd places (but \R should handle that).

The \h is horizontal whitespace:

#!/bin/bash
#
#    replace CR with CRLF
#
/usr/bin/perl -CSDA -pi <<'EOF' - "$@"
s/dateformat\R//ig;
s/text1\h+//ig;
s/text2\h+//ig;
s/text3\h+//ig;
EOF

Note that I've added the /i flag for case insensitivity since your patterns are all lowercase but the data have mixed case.

I've also removed the /o switch, which no longer does anything.

If there's some reason you're removing DateFormat by itself, you can just removing all trailing whitespace after Textn. The \s gets vertical and horizontal whitespace:

#!/bin/bash
#
#    replace CR with CRLF
#
/usr/bin/perl -CSDA -pi <<'EOF' - "$@"
s/dateformat//ig;
s/text1\s+//ig;
s/text2\s+//ig;
s/text3\s+//ig;
EOF

If you just want to skip those lines, you don't even need to do a substitution. You can just skip them whether or not they have the DateFormat bit. This uses the -n instead of -p so I can control when it outputs. I've added the \A beginning-of-string anchor for good measure:

#!/bin/sh
/usr/bin/perl -CSDA -ni -e 'print unless /\AText[123]\s+/i' "$@"

Answer 2

This script does same what does one liner

use strict;
use warnings;
use feature 'say';

my $data = do { local $/; <DATA> };

$data =~ s/Text\d+\s+Dateformat\s*//g;
say $data;

__DATA__
Text1 Dateformat 
[Content1] 

Text2 Dateformat
[Content2]

Text3 Dateformat
[Content3]

Output

[Content1]

[Content2]

[Content3]

NOTE: replace <DATA> to <> to read from pipe or a file given on command line

Answer 3

use strict;
use warnings;

use Data::Dumper;

my $record = {};
my ( $key, $val );

while ( my $row = <DATA> ) {
    chomp( $row );
    next if !$row;
    if ( $row =~ /Dateformat/ ) {
        ( $key, undef ) = split /\s+/, $row;
        print "$key\n";
    } elsif ( $row =~ /\[/ ) {
        $record->{$key} = $row;
    }  
}   
print Dumper($record);





__DATA__
Text1 Dateformat 
[Content1] 

Text2 Dateformat
[Content2]

Text3 Dateformat
[Content3]