Powershell script with -replace regex not working

Question

I have this file named "test.json"

"AFMU repairs": {
  "enabled": true,
  "priority": 3,
  "responder": true,
  "script": "{event.item} \r\n{if event.repairedfully:\r\n    fully repaired\r\n|else:\r\n    partially repaired \r\n    {Occasionally(2, cat(\r\n        OneOf(\"to \", \"at \"),\r\n        Humanise(event.health * 100),\r\n        \" percent functionality\"\r\n    ))}\r\n}\r\n\r\n{Occasionally(2, \r\n    cat(OneOf(\", \", \"and is\"), \" ready for re-activation\")\r\n)}.",
  "default": true,
  "name": "AFMU repairs",
  "description": "Triggered when repairing modules using the Auto Field Maintenance Unit (AFMU)"
},
"Asteroid cracked": {
  "enabled": true,
  "priority": 3,
  "responder": true,
  "script": null,
  "default": true,
  "name": "Asteroid cracked",
  "description": "Triggered when you break up a 'Motherlode' asteroid for mining"
},
"Asteroid prospected": {
  "enabled": true,
  "priority": 3,
  "responder": true,
  "script": "{set minimumPercent to 10} {_ The minimum percentage surface mineral concentration to report _}\r\n{set spokenCores to [\r\n    \"Alexandrite\": false,\r\n    \"Benitoite\": false,\r\n    \"Grandidierite\": false,\r\n    \"Low Temperature Diamonds\": true,\r\n    \"Monazite\": false,\r\n    \"Musgravite\": false,\r\n    \"Rhodplumsite\": false,\r\n    \"Serendibite\": false,\r\n    \"Void Opals\": true,\r\n]}\r\n{set spokenMinerals to [\r\n    \"Bauxite\": false,\r\n    \"Bertrandite\": false,\r\n    \"Bromellite\": false,\r\n    \"Cobalt\": false,\r\n    \"Coltan\": false,\r\n    \"Cryolite\": false,\r\n    \"Gallite\": false,\r\n    \"Gold\": false,\r\n    \"Goslarite\": false,\r\n    \"Hydrogen Peroxide\": false,\r\n    \"Indite\": false,\r\n    \"Jadeite\": false,\r\n    \"Lepidolite\": false,\r\n    \"Lithium Hydroxide\": false,\r\n    \"Liquid oxygen\": false,\r\n    \"Low Temperature Diamonds\": true,\r\n    \"Methane Clathrate\": false,\r\n    \"Methanol Monohydrate\": false,\r\n    \"Moissanite\": false,\r\n    \"Osmium\": false,\r\n    \"Painite\": true,\r\n    \"Platinum\": false,\r\n    \"Palladium\": false,\r\n    \"Praseodymium\": false,\r\n    \"Pyrophyllite\": false,\r\n    \"Rutile\": false,\r\n    \"Samarium\": false,\r\n    \"Silver\": false,\r\n    \"Taaffeite\": false,\r\n    \"Thorium\": false,\r\n    \"Tritium\": true,\r\n    \"Uraninite\": false,\r\n    \"Water\": false,\r\n]}\r\n\r\n{if len(event.motherlode) > 0 && spokenCores[event.motherlode]:\r\n   Motherlode detected: {event.motherlode}.\r\n}\r\n\r\n{set minerals to []}\r\n{for mineral in event.commodities:\r\n    {if mineral.percentage > minimumPercent && spokenMinerals[mineral.commodity]:\r\n        {set mineral_desc to: \r\n            {round(mineral.percentage)} percent {mineral.commodity}\r\n        }\r\n        {set minerals to cat(minerals, [mineral_desc])}\r\n    }\r\n}\r\n{if len(minerals) > 0:\r\n    Asteroid contains {List(minerals)}\r\n    {if event.materialcontent = \"High\":\r\n        and a high concentration of engineering materials\r\n    }.\r\n    {if event.remaining < 100:\r\n        It is {100 - event.remaining} percent depleted.\r\n    }\r\n}",
  "default": true,
  "name": "Asteroid prospected",
  "description": "Triggered when using a prospecting drone"
},

What i want is to "extract" the script string, make it readable, put the script name at the top and save a readable file "test.cottle".

-EDIT-

Below an example of the final result (converted only the first of the three script in test.json):

{_ AFMU repairs }
{event.item} 
{if event.repairedfully:
    fully repaired
|else:
    partially repaired 
    {Occasionally(2, cat(
        OneOf(\"to \", \"at \"),
        Humanise(event.health * 100),
        \" percent functionality\"
    ))}
}

{Occasionally(2, 
    cat(OneOf(\", \", \"and is\"), \" ready for re-activation\")
)}.

{_ Asteroid prospected }
{set minimumPercent to 10} {_ The minimum percentage surface mineral concentration to report _}
... 
... and so on

Here's a working sample of the first expression.

I already have done it in SublimeText using regex search/replace, but i want to make it automatically, so i tried with PowerShell 7, which is basically a totally new thing for me.

This is my test.ps1:

Write-Host "Reading $FileName..."
$content = [System.IO.File]::ReadAllText("C:\test.json")

Write-Host "..replacing..."
$content -replace '(?sm)(.|\n)*?\"script\"\: \"(?<TheScript>.*)\"\,\n.*\n.*\"name\"\: \"(?<TheScriptName>.*)\".*\n' , '\{_ ${TheScriptName} \}\n${TheScript}\n\n'
$content -replace '\\r\\n' , '\n'

Write-Host "...saving."
[System.IO.File]::WriteAllText("C:\test.json.cottle", $content)

Well, the save file is just like the original.

Why does my replace won't work? i tried it at https://regexr.com/ and there's working (altough without named capturing groups). I also added the (?sm) from this question but it does'nt work. What should i do?

If you (reader) think that this question have to be downvoted, I would like to know the reasons, as i've put all the efforts i know to ask in the correct way and if i deserve a downvote i need to learn why. — Parduz, Jul 20 '21 at 14:15
Could you show us an example of what the desired output would be? — Theo, Jul 20 '21 at 14:24

Tomalak · Answer 1 · 2021-07-20T16:49:52.937

Don't try to parse JSON with regular expressions. Independently of the fact that you're in for a bag of hurt, JSON is too complex to be handled properly with regex.

Use a parser.

PowerShell's JSON parser is called ConvertFrom-JSON.

Using a helper function from an earlier answer of mine (Iterating through a JSON file PowerShell)

# https://stackoverflow.com/a/33521853/18771
function Get-ObjectMember {
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$True, ValueFromPipeline=$True)]
        [PSCustomObject]$obj
    )
    $obj | Get-Member -MemberType NoteProperty | ForEach-Object {
        $key = $_.Name
        [PSCustomObject]@{Key = $key; Value = $obj."$key"}
    }
}

Things become easy (I'm assuming $data.events here, use the real path to your events list):

$data = Get-Content "test.json" -Raw -Encoding UTF8 | ConvertFrom-Json

$data.events | Get-ObjectMember | ForEach-Object {
    # $_.key will be "AFMU repairs", $_.value will be the associated object, etc.
    "{_ $($_.key) }"
    $_.value.script
    ""
}

Outputs

{_ AFMU repairs }
{event.item} 
{if event.repairedfully:
    fully repaired
|else:
    partially repaired 
    {Occasionally(2, cat(
        OneOf("to ", "at "),
        Humanise(event.health * 100),
        " percent functionality"
    ))}
}

{Occasionally(2, 
    cat(OneOf(", ", "and is"), " ready for re-activation")
)}.

{_ Asteroid cracked }

{_ Asteroid prospected }
{set minimumPercent to 10} {_ The minimum percentage surface mineral concentration to report _}
{set spokenCores to [
    "Alexandrite": false,
    "Benitoite": false,
    "Grandidierite": false,
    "Low Temperature Diamonds": true,
    "Monazite": false,
    "Musgravite": false,
    "Rhodplumsite": false,
    "Serendibite": false,
    "Void Opals": true,
]}
{set spokenMinerals to [
    "Bauxite": false,
    "Bertrandite": false,
    "Bromellite": false,
    "Cobalt": false,
    "Coltan": false,
    "Cryolite": false,
    "Gallite": false,
    "Gold": false,
    "Goslarite": false,
    "Hydrogen Peroxide": false,
    "Indite": false,
    "Jadeite": false,
    "Lepidolite": false,
    "Lithium Hydroxide": false,
    "Liquid oxygen": false,
    "Low Temperature Diamonds": true,
    "Methane Clathrate": false,
    "Methanol Monohydrate": false,
    "Moissanite": false,
    "Osmium": false,
    "Painite": true,
    "Platinum": false,
    "Palladium": false,
    "Praseodymium": false,
    "Pyrophyllite": false,
    "Rutile": false,
    "Samarium": false,
    "Silver": false,
    "Taaffeite": false,
    "Thorium": false,
    "Tritium": true,
    "Uraninite": false,
    "Water": false,
]}

{if len(event.motherlode) > 0 && spokenCores[event.motherlode]:
   Motherlode detected: {event.motherlode}.
}

{set minerals to []}
{for mineral in event.commodities:
    {if mineral.percentage > minimumPercent && spokenMinerals[mineral.commodity]:
        {set mineral_desc to: 
            {round(mineral.percentage)} percent {mineral.commodity}
        }
        {set minerals to cat(minerals, [mineral_desc])}
    }
}
{if len(minerals) > 0:
    Asteroid contains {List(minerals)}
    {if event.materialcontent = "High":
        and a high concentration of engineering materials
    }.
    {if event.remaining < 100:
        It is {100 - event.remaining} percent depleted.

You can pipe this directly into a file with | Set-Content test.cottle -Encoding UTF8

Thank you so much! You solved my problem and made me learn something too! I'll give the answer to @TheMadTechnician 'cause he actually answered the question i posed, but in the end i'll use your solution. Thanks again. — Parduz, Jul 21 '21 at 10:41

score 0 · Accepted Answer · answered Jul 20 '21 at 16:14

You need to use the non-greedy *? instead of the greedy *. Also, the -replace operator does not update the content of the variable, so you need to capture the output of it with something like $content = $content -replace.... Then your output will be a little unexpected since you don't need to escape the replacement text, so '\{_ ${TheScriptName} \}\n${TheScript}\n\n' should simply be '{_ ${TheScriptName} }\n${TheScript}\n\n', and the special characters won't translate so you have to use "`n" instead of \n. The next line can be updated to '(\\r)?\\n', "`n". Finally you'll almost get your expected output.

Write-Host "Reading $FileName..."
$content = [System.IO.File]::ReadAllText("C:\test.json")

Write-Host "..replacing..."
$content = $content -replace '(?sm)(.|\n)*?\"script\"\: \"(?<TheScript>.*?)\"\,\n.*?\n.*?\"name\"\: \"(?<TheScriptName>.*?)\".*?\n' , '{_ ${TheScriptName} }\n${TheScript}\n\n'
$content = $content -replace '(\\r)?\\n' , "`n"

Write-Host "...saving."
[System.IO.File]::WriteAllText("C:\test.json.cottle", $content)

I did say "almost" since that includes some trailing data you don't expect due to how you replace things:

  "description": "Triggered when using a prospecting drone"
}

Since you parse this as a multi-line string you have to use the non-greedy or else you end up consuming all remaining data on the last .*. To get around that we'll have to split the file into individual records, and then parse each one individually. Totally doable, it just takes a little more doing. Here's what I would do if it were me:

Write-Host "Reading $FileName..."
$content = Get-Content $FileName -Raw

Write-Host "..replacing..."
$content = $content -split '(?sm)(?<=},)'|
    Where-Object{$_ -match '(?sm)(.|\n)*?\"script\"\: \"(?<TheScript>.*?)\"\,\n.*?\n.*?\"name\"\: \"(?<TheScriptName>.*?)\"'} |
    ForEach-Object{"`{_ $($Matches['TheScriptName']) }`n$($Matches['TheScript'])`n`n" -replace '(\\r)?\\n', "`n"}

Write-Host "...saving."
Set-Content -Value $content -Path "$filename.cottle"

Thank you so much. Your explanation made me see all the mistakes i've made, so i learnt something useful. — Parduz, Jul 21 '21 at 10:42

Powershell script with -replace regex not working

2 Answers2