Perl regex back reference not being greedy

Question

This is a working regex:

/(ANSI|AAMVA) (\d{6})(\d{2})(\d{2})(\d{0,2})((?:DL)|(?:ID))+(.*?)\g{-2}+([^"]+)/

This is a sample string:

"@\n\nANSI 6334290212DL00389199ZO04420478DLDAQ3572928\nDAASMITH, JOHN DOE\nDAG\nDAL4389 NE 47TH AVE\nDAIASHLAND\nDAJOR\nDAK97555      \nDARC   \nDASD         \nDATM     \nDAU504\nDAW180\nDBA12201212\nDBB19780303\n"

I am trying to match a delimiter, either DL or ID, that may be in the string a second time.

I want to match whichever of DL or IDmatched previously.

The problem is, if I use ? to accomplish this it stops being greedy and prefers 0 matches.

I'm stumped, am I missing something basic with how ? operates?

Edit: The problem isn't extracting the JSON data, it's parsing the msg bit, using JSON doesn't do anything to accomplish this. I trimmed the string to just the pertinent part.

The fix by @hobbs works because it let's me change the ? to a + and still match nothing, if nothing is there.

Works! :)

/(ANSI|AAMVA) (\d{6})(\d{2})(\d{2})(\d{0,2})((?:DL)|(?:ID))+(.*?)(?:\g{-2}|(?="))+([^"]+)/

Answer 1

The problem isn't that \g{-2}? is non-greedy, it's that the (.*?) immediately before it is non-greedy, and \g{-2}? is capable of matching nothing, which means that it can't fail. And if it can't fail, then it doesn't force the group before it to match more than 0 characters. So invariably, (.*?) will match nothing, \g{-2} will match nothing, and ([^"]+) will match everything.

I don't entirely understand the format you're trying to extract (other than that it's old and weird and reminds me of CIBER billing records), but I would suggest that you either need more anchoring to focus your regex's attention on the right place, or you need to upgrade to something like a proper parser for the format. Since you're saying that you added the ? to handle the case where the delimiter never appears, the quickest band-aid fix would possibly be (?:\g{-2}|(?=")) which asserts that you either find the delimiter, or you got to the closing quote without finding it.

Although, Borodin's observation is also valid; it would be much better to decode the JSON first and then work with the string from the decoded JSON structure, instead of trying to run a regex on the JSON directly. In that case, you should be looking for \z (end of string) rather than ".

Answer 2

Your data is JSON, and it is very wrong to try to process it using regex patterns. There are perfectly good Perl modules to convert the text to a navigable data structure

I can't understand exactly what you need because you're talking about DA and ID strings, and ID doesn't occur anywhere in your sample data. But this short program should help

use strict;
use warnings 'all';
use feature 'say';

use JSON 'decode_json';

my $json = do {
    local $/;
    <DATA>;
};

my $data = decode_json $json;

say $data->{msg};


__DATA__
{"name":"SC","hostname":"tukwila","pid":11,"level":30,"msg":"@\n\nANSI 6334290212DL00389199ZO04420478DLDAQ3572928\nDAASMITH, JOHN DOE\nDAG\nDAL4389 NE 47TH AVE\nDAIASHLAND\nDAJOR\nDAK97555      \nDARC   \nDASD         \nDATM     \nDAU504\nDAW180\nDBA12201212\nDBB19780303\n","time":"2016-04-02T01:09:07.113Z","v":0}

output

@

ANSI 6334290212DL00389199ZO04420478DLDAQ3572928
DAASMITH, JOHN DOE
DAG
DAL4389 NE 47TH AVE
DAIASHLAND
DAJOR
DAK97555      
DARC   
DASD         
DATM     
DAU504
DAW180
DBA12201212
DBB19780303