PCRE Regex - How to return matches with multiline string looking for multiple strings in any order

Question

I need to use Perl-compatible regex to match several strings which appear over multiple lines in a file.

The matches need to appear in any order (server servernameA.company.com followed by servernameZ.company.com followed by servernameD.company.com or any order combination of the three). Note: All matches will appear at the beginning of each line.

In my testing with grep -P, I haven't even been able to produce a match on simple string terms that appear in any order over new lines (even when using the /s and /m modifiers). I am pretty sure from reading I need a look-ahead assertion but the samples I used didn't produce a match for me even after analyzing each bit of the regex to make sure it was relevant to my scenario.

Since I need to support this in Production, I would like an answer that is simple and relatively straight-forward to interpret.

Sample Input

irrelevant_directive = 0

# Comment
server servernameA.company.com iburst

additional_directive = yes

server servernameZ.company.com iburst
server servernameD.company.com iburst

# Additional Comment
final_directive = true

Expectation

The regex should match and return the 3 lines beginning with server (that appear in any order) if and only if there is a perfect match for strings'serverA.company.com', 'serverZ.company.com', and 'serverD.company.com' followed by iburst. All 3 strings must be included.

Finally, if the answer (or a very similar form of the answer) can address checking for strings in any order on a single line, that would be very helpful. For example, if I have a single-line string of: preauth param audit=true silent deny=5 severe=false unlock_time=1000 time=20ms and I want to ensure the terms deny=5 and time=20ms appear in any order and if so match.

Thank you in advance for your assistance.

Thanks for the quick response and looking at this. PCRE specifically. I will be using the command with `grep -P`. Please refresh as I added a new short section on the expected output of the regex which I previously left out. — Kurt W, Nov 03 '16 at 21:42
Then please don't tag with Perl. I'll remove the tag. There are solutions to this with a full-fledged program, but doing that in one regex is going to be tough. — simbabque, Nov 03 '16 at 21:44

score 1 · Answer 1 · answered Nov 03 '16 at 21:51

1

You don't need to use the PCRE features, you can simply write in ERE:

grep -E '.*(\bdeny=5\b.*\btime=20ms\b|\btime=20ms\b.*\bdeny=5\b).*' file

The PCRE approach will be different: (however you can also use the previous pattern)

grep -P '^(?=.*\bdeny=5\b).*\btime=20ms\b.*' file

answered Nov 03 '16 at 21:51

Casimir et Hippolyte

88,009
5
94
125

Hi Casimir! That was actually my secondary question that you answered. Thank you and this is helpful, but do you have an approach to solve the main problem (multi-line)? Sorry if I made things more confusing than necessary. The single line regex was a bonus and I'm testing it now! – Kurt W Nov 03 '16 at 23:05
Hi Casimir, I confirmed your solution to the single-line problem works perfectly! Thanks. When you get a chance, can you look at my main question? Apologies for grouping two into one--I won't do that in the future. – Kurt W Nov 04 '16 at 16:51
Upvoted as this is helpful. bwoebi solved my primary question. Thank you for your time! – Kurt W Nov 08 '16 at 01:45

score 1 · Accepted Answer · edited May 23 '17 at 12:33

1

Regarding the main issue [for the secondary question see Casimir et Hippolyte answer] (using x modifier): https://regex101.com/r/mkxcap/5

(?:
  (?<a>.*serverA\.company\.com\s+iburst.*)
 |(?<z>.*serverZ\.company\.com\s+iburst.*)
 |(?<d>.*serverD\.company\.com\s+iburst.*)
 |[^\n]*(?:\n|$)
)++
(?(a)(?(z)(?(d)(*ACCEPT))))(*SKIP)(*F)

The matches are now all in the a, z and d capturing groups.

It's not the most efficient (it goes three times over each line with backtracking...), but the main takeaway is to register the matches with capturing groups and then checking for them being defined.

edited May 23 '17 at 12:33

Community

1
1

answered Nov 07 '16 at 17:34

bwoebi

23,637
5
58
79

Thank you for providing this. I put my sample text from the original question into a file and tried this regex against it but it is not returning a hit. `grep -P '(?:(?.*serverA\.company\.com\s+iburst.*)|(?.*serverZ\.company\.com\s+iburst.*)|(?.*serverD\.company\.com\s+iburst.*)|[^\n]*(?:\n|$))++(?(a)(?(z)(?(d)(*ACCEPT))))(*SKIP)(*F)' testing` returns nothing. Does this need slight tuning? If I could ensure that the 3 lines would always appear right after each other (as opposed to somewhere throughout the file), would that make the regex simpler to write? Thanks in advance!! – Kurt W Nov 07 '16 at 19:59
@KurtW The issue with `grep -P` is that it matches line per line _individually_. You cannot match anything spanning multiple lines with `grep -P`. You need to feed PCRE here the whole file. (quote man page for grep: "Obtain patterns from FILE, **one per line**.") – bwoebi Nov 07 '16 at 21:07

PCRE Regex - How to return matches with multiline string looking for multiple strings in any order

2 Answers2