0

I want to find a string (driving factor) and if found, only then look for another string with same x-request-id and extract some details out of it.

x-request-id=12345 "InterestingField=7850373" [this one is subset of very specific request]
x-request-id=12345 "veryCommonField=56789" [this one is a superSet of all kind of requests]

What I've tried:

index=myindex "InterestingField" OR "veryCommonField"
| transition x-request-id

But problem with above is this query join all those request as well which has only veryCommonField in it. I want to avoid join as they are pretty low in performance.

What I need: list InterestingField, veryCommonField

Example: Below represents beginning of all kind of request. We get thousands of such request in a day.

index=myIndex xrid=12345 "Request received for this. field1: 123 field2: test"

Out of all above request below category falls under 100.

index=myIndex xrid=12345 "I belong to blahBlah category. field3: 67583, field4: testing"

I don't want to search in a super-set of 1000k+ but only in matching 100 requests. Because with increased time span, this search query will take very long.

Hari Bisht
  • 197
  • 1
  • 2
  • 8

1 Answers1

1

If I'm understanding your use-case, the following may be helpful.

Using stats

index=myindex "InterestingField" OR "veryCommonField" | stats values(InterestingField), values(veryCommonField) by x-request-id

Using subsearch

index=myindex [ index=myindex InterestingField=* | fields x-request-id | format  ]

Depending on the number of results that match InterestingField, you can also use map, https://docs.splunk.com/Documentation/Splunk/8.0.3/SearchReference/Map

index=myindex InterestingField="*" | map maxsearches=0 "search index=myindex x-request-id=$x-request-id$ | stats values(InterestingField), values(veryCommonField) by x-request-id"

If you provide more thorough example events, we can assist you further.

Simon Duff
  • 2,631
  • 2
  • 7
  • 15
  • Thanks Simon. Subsearch is best suited to my requirement here. But how do i extract field using rex in inner search and table them in my final search. ```index=myindex "_my_search_string_" | rex "before(?.*).*" [ index=myindex "_my_another_space separated string" | rex "before(?.*).*" | table myfield | fields x-request-id | format ] | table myField1, myField2``` – Hari Bisht May 04 '20 at 12:25